How to Build AI Agents: Planning, Tool Use, and Execution

Building an AI agent starts with a simple idea: give a model the ability to take actions, observe the results, and loop. The model reasons over a goal, decides what to do next, calls a tool, gets back a result, and continues until the task is done. That read-evaluate-act loop is the core of every agent you’ll build, whether it’s a single-step tool caller or a multi-step system that writes code, runs it, and fixes its own errors.

This tutorial walks through the four components that make an agent work: planning, tool use, code execution, and evaluation. You’ll come away with a working mental model and a minimal implementation you can run today.

What Makes Something an AI Agent

An AI agent differs from a basic LLM call in one critical way: it takes actions and observes their results.

A plain chat completion produces text. An agent uses that text to call a function, query an API, run a command, or read a file — and feeds the result back as input for the next step. This feedback loop is what makes agents useful for tasks that can’t be answered in a single response.

Three capabilities define an agent:

• Tool calling — invoking defined functions by name with structured arguments
• Context accumulation — carrying tool results forward as the conversation grows
• Goal-directed looping — continuing until a stop condition is met, not just after one response

If a system has all three, you’re building an agent. If context accumulation or looping is missing, it’s a tool-augmented chatbot.

The same architecture handles a wide range of tasks: research agents that search and synthesize, coding agents that write and run code, data agents that query databases and generate reports. The implementation pattern is nearly identical in each case.

Step 1: Planning — The Reasoning Layer

The planning layer is where the model decides what to do next. You don’t need a separate planner model — the same model that generates text can reason over a goal and produce a structured action (a tool call) on each step.

The system prompt does most of the work. A good planning prompt tells the model:

• What goal it’s working toward
• Which tools are available and what each one does
• What format to use when calling a tool
• When to stop (the finish condition)

With Novita’s LLM API — which is OpenAI-compatible — this looks like:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key=os.environ["NOVITA_API_KEY"],
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "run_code",
            "description": "Execute Python code and return stdout/stderr.",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {
                        "type": "string",
                        "description": "Python code to execute"
                    }
                },
                "required": ["code"]
            }
        }
    }
]

messages = [
    {
        "role": "system",
        "content": "You are a coding assistant. Use the run_code tool to execute Python and solve tasks. Stop when you have a final answer."
    },
    {
        "role": "user",
        "content": "Find all prime numbers under 50."
    }
]

response = client.chat.completions.create(
    model="meta-llama/llama-3.1-8b-instruct",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

The model returns a tool call — a JSON object with the function name and arguments — instead of a plain text response. Your code extracts this, executes the function, and loops back with the result appended to messages.

Step 2: Tool Use — Connecting the Model to the World

Tools are what agents actually do. Without them, the model can only produce text. With them, it can read files, call APIs, run code, search databases, or trigger any function you define.

Defining Tools

Each tool is a JSON schema describing a function. The model reads this schema to format its call correctly. A few practical rules:

• Keep descriptions short and precise. The model reads them on every step. Verbose descriptions waste tokens.
• Match parameter names to what the function actually accepts. Mismatches between the schema and your implementation cause silent failures.
• Return structured output. If your tool returns raw HTML or unformatted prose, the model wastes tokens parsing it. Return clean strings or JSON with clear labels.

The Tool Call Loop

After the model makes a tool call, your code:

1. Parses the tool name and arguments from the response
2. Looks up the matching Python function
3. Calls it with the parsed arguments
4. Appends the result to messages with role: "tool"
5. Makes another completion request

The loop continues until the model returns a plain message with no tool call (the finish condition).

import json

def run_agent_loop(client, model, messages, tools, tool_fns, max_steps=10):
    for _ in range(max_steps):
        response = client.chat.completions.create(
            model=model,
            messages=messages,
            tools=tools,
            tool_choice="auto",
        )
        choice = response.choices[0]

        # Done — no tool call
        if choice.finish_reason == "stop":
            return choice.message.content

        # Execute the tool call
        tool_call = choice.message.tool_calls[0]
        fn_name = tool_call.function.name
        fn_args = json.loads(tool_call.function.arguments)

        result = tool_fns[fn_name](**fn_args)

        # Append both the model's tool call and the result
        messages.append(choice.message)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": str(result),
        })

    return "Max steps reached without a final answer."

This is the complete agent loop. The max_steps guard prevents runaway loops — if the model gets stuck calling the same tool repeatedly, the loop terminates rather than running indefinitely. Everything else — multi-tool orchestration, memory, streaming — builds on top of this pattern.

Common Tool Patterns

File access: Read and write files within the agent’s working directory. Return file contents as strings.

Web search: Return search results as a list of {title, url, snippet} objects. Keep snippets short.

Code execution: Execute code in an isolated environment and return stdout/stderr (covered in the next section).

Structured data lookup: Query a database or API and return records as JSON. The model handles summarization.

Each tool should do one thing and return deterministic output given the same inputs. Agents that try to pack too much logic into a single tool become harder to debug and evaluate.

Step 3: Code Execution — Running Code Safely

If your agent generates and runs code, you need an isolated environment. Running model-generated code directly on your machine is a security risk: the model might produce destructive shell commands, infinite loops, or code that reads sensitive files it shouldn’t access.

The standard approach is a sandbox — a short-lived, isolated Linux environment where the agent can execute code freely without affecting the host system. Novita’s Agent Sandbox is designed for exactly this use case: a cloud-based execution environment that spins up in under 200ms, runs any code the agent produces, and discards the environment when the session ends.

Here’s how the run_code tool from the planning section would be implemented using the sandbox:

from novita_sandbox.code_interpreter import Sandbox

sandbox = Sandbox.create()

def run_code(code: str) -> str:
    result = sandbox.run_code(code)
    output = ""
    if result.logs.stdout:
        output += "\n".join(result.logs.stdout)
    if result.logs.stderr:
        output += "\nSTDERR: " + "\n".join(result.logs.stderr)
    return output or "(no output)"

When the agent calls run_code, this function executes the code in the isolated environment and returns stdout/stderr as a string. The agent sees the result and decides what to do next — whether that’s fixing a bug, continuing with the output, or declaring the task done.

This architecture cleanly separates the reasoning layer from the execution layer. The LLM plans and calls tools; the sandbox runs code without exposing anything outside the container.

What the Sandbox Handles

• Shell commands (bash, git, python, node)
• File system operations within the sandbox
• Network requests from within the container
• Long-running background tasks with async result retrieval
• Pause and resume for multi-session workflows

What it doesn’t allow: access to the host system, other sandboxes, or credentials you haven’t explicitly passed in. This containment is what makes it safe to let the model write and run arbitrary code.

Step 4: Evaluation — Knowing When the Agent Is Right

Evaluation is the most skipped step in agent development. An agent that produces output isn’t the same as an agent that produces correct output.

Three Levels of Evaluation

1. Deterministic checks

For tasks with known correct answers — parsing a date, computing a value, extracting a field — write unit tests against the agent’s output. These are cheap to run and catch regressions fast.

2. Execution-based checks

For coding agents, the most reliable signal is: does the code actually run and produce the right result? Execute the agent’s output in the sandbox and compare stdout to expected values. This is stronger than comparing code text because it tests behavior, not syntax.

def evaluate_code_task(agent_code: str, expected_stdout: str) -> bool:
    result = run_code(agent_code)
    return result.strip() == expected_stdout.strip()

3. LLM-as-judge

For open-ended tasks — writing a summary, answering a research question — use a second LLM call to grade the output against a rubric. Keep the rubric specific: “Did the answer include a working code example?” is better than “Is the answer good?” Vague rubrics produce inconsistent scores.

Tracking Failure Modes

Agents fail in a recurring set of patterns:

Failure	Cause	Fix
Tool call loop	Model keeps calling the same tool	Set max_steps; log intermediate steps
Argument mismatch	Schema doesn’t match implementation	Validate tool arguments before calling
Context overflow	Session grows past model’s context limit	Summarize or truncate older tool results
Hallucinated tool name	Model invents a tool that doesn’t exist	Use a strict tool list with tool_choice: "auto"

Evaluation isn’t a separate phase at the end — it’s part of development. Every time you extend an agent’s capabilities, write at least one test that would catch a regression in the new behavior.

Choosing a Model for Agent Reasoning

Most tutorials default to closed-source models for agent reasoning. That works well for prototyping, but in production — where an agent might make 20 LLM calls per task — model cost and rate limits become real constraints.

Open-source models hosted via Novita’s LLM API increasingly match closed-source performance on the tasks agents actually need: function calling, structured output, and multi-step reasoning. Models like Llama 3.3 70B and DeepSeek-V3 benchmark at GPT-4 class levels on coding and instruction following while running at a fraction of the cost. For high-throughput agentic workloads, this difference adds up quickly.

The Novita API is fully OpenAI-compatible, so switching from a closed-source model to an open-source one is a one-line change:

# Before
model = "gpt-4o"

# After — same API contract, lower cost
model = "meta-llama/llama-3.3-70b-instruct"

For latency-sensitive agent loops, smaller models like meta-llama/llama-3.1-8b-instruct respond faster and cost less per step, which matters when each user task involves many sequential calls. A practical pattern: use a faster, smaller model for routing and classification steps, and a larger model for complex reasoning steps that need more capacity.

The Novita models library also includes strong options for specific agentic sub-tasks — vision models for image-based tool results, embedding models for retrieval, and code-specialized models for coding agents — all accessible from the same API endpoint.

Putting It Together — A Minimal Working Agent

Here’s a complete agent that takes a math problem, generates Python code to solve it, runs it in a sandbox, and returns the verified result:

import os
import json
from openai import OpenAI
from novita_sandbox.code_interpreter import Sandbox

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key=os.environ["NOVITA_API_KEY"],
)
sandbox = Sandbox.create()

def run_code(code: str) -> str:
    result = sandbox.run_code(code)
    stdout = "\n".join(result.logs.stdout)
    stderr = "\n".join(result.logs.stderr)
    return stdout or stderr or "(no output)"

tool_fns = {"run_code": run_code}

tools = [
    {
        "type": "function",
        "function": {
            "name": "run_code",
            "description": "Execute Python code. Returns stdout or stderr.",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {"type": "string"}
                },
                "required": ["code"]
            }
        }
    }
]

messages = [
    {
        "role": "system",
        "content": "Solve the task by writing and running Python code. Use run_code to execute it. Stop when you have a verified answer."
    },
    {
        "role": "user",
        "content": "What is the sum of all even Fibonacci numbers under 1,000,000?"
    }
]

for _ in range(10):
    response = client.chat.completions.create(
        model="meta-llama/llama-3.1-8b-instruct",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )
    choice = response.choices[0]

    if choice.finish_reason == "stop":
        print(choice.message.content)
        break

    tool_call = choice.message.tool_calls[0]
    fn_args = json.loads(tool_call.function.arguments)
    result = tool_fns[tool_call.function.name](**fn_args)

    messages.append(choice.message)
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": result,
    })

sandbox.kill()

This covers the full agent loop in roughly 50 lines: planning, tool calling, execution, context accumulation, max-step guard, and clean sandbox teardown. The agent writes code, runs it, sees the output, and either continues or gives a final answer.

To extend it: add more tools to tool_fns and tools, swap in a larger model for harder tasks, or replace the sandbox run_code with any other function — a web search, a database query, an API call. The loop stays the same.

FAQ

What’s the difference between an AI agent and a chatbot?

A chatbot generates text responses. An agent takes actions — calling tools, running code, querying APIs — and uses the results to continue reasoning toward a goal. The presence of tool calls and a feedback loop is what makes something an agent.

Do I need a framework like LangChain to build an AI agent?

No. The core agent loop is simple enough to implement directly against any OpenAI-compatible API. Frameworks add value for complex orchestration (multi-agent systems, persistent memory, streaming output), but they add abstraction layers that can make debugging harder. Start with the raw API, then add a framework if you genuinely need what it provides.

How do I prevent the agent from running dangerous code?

Use a sandboxed execution environment. The agent’s code runs inside an isolated container that has no access to your host system or other processes. The sandbox is discarded after the session ends, so any changes the agent makes are scoped to that container.

Can open-source models handle function calling reliably?

Yes — with caveats on model size. Llama 3.3 70B, DeepSeek-V3, and Qwen 2.5 72B all handle structured tool calling well. Smaller models (7B–8B) work for simple, well-defined tool schemas but can struggle with ambiguous instructions or complex multi-step reasoning chains. For production agents, test on your actual task distribution before committing to a model.

How many steps should an agent take per task?

Set a hard cap — 10 to 20 steps is typical for most tasks. Log intermediate steps so you can see exactly what the agent did and debug failures. Tasks that consistently need more steps usually benefit from being broken into sub-agents or sub-tasks rather than extending the loop limit.

Recommended Articles

• Building a Coding Agent with Novita’s Agent Sandbox
• Agent vs Workflow: Key Differences and Practical Trade-Offs
• Which Inference Provider Is Right for AI Agents