A coding agent is an AI system that uses a large language model as its reasoning core to autonomously write, execute, and iterate on code. Unlike a code assistant that suggests completions in your editor, a coding agent runs a full observe-decide-act loop: it reads files, writes changes, runs commands, checks the output, and revises until the task is done.
This article explains how that loop works—the planner, the LLM inference layer, the tools, and the sandboxed execution environment—then shows how to put one together using Novita’s LLM API and Agent Sandbox.
What Makes Something a Coding Agent
The difference between a code assistant and a coding agent is execution. A code assistant generates a suggestion and stops. An AI coding agent generates code, runs it, reads the result, and keeps going until the goal is met—or until it gets stuck.
That execution capability has three concrete requirements:
- Tool access — the ability to read files, write files, and run shell commands
- A sandboxed environment — somewhere to execute code that won’t damage the host system if something goes wrong
- A persistent context — tool outputs feed back into the model’s context so it can reason about what happened
Without all three, you have a chatbot that can write code. With all three, you have an agent.
The term “code agent” gets used loosely to cover everything from IDE inline suggestions to fully autonomous systems that can take a vaguely specified task, figure out which files are involved, make changes, and verify they work—without a human in the loop for each step. When developers compare “best code agent” options, they usually mean the latter: systems that complete multi-step coding tasks reliably with minimal hand-holding.
The Four Layers of a Coding Agent
Every production-grade AI coding agent has four recognizable components. The implementation details vary—different frameworks, different LLMs, different sandbox providers—but the architecture is consistent.
1. The Planner
The planner receives the task description and breaks it into steps the agent will execute. For simple tasks this happens implicitly inside the model’s reasoning. For complex tasks—“migrate this service to use the new auth library”—an explicit planning step produces a numbered task list the model works through, updating state after each step.
Planning also determines when to stop. An agent without a completion criterion will keep adding refinements indefinitely or, worse, loop on a failed step. Most implementations encode the task success condition in the system prompt and let the model decide when it’s done.
2. The LLM Inference Layer
The LLM is the reasoning core of any AI coding agent. It decides which tool to call next, what arguments to pass, and how to interpret the result. That decision is expressed as a structured tool call—a JSON object with the function name and parameters—which the framework dispatches to the actual execution layer.
For code agents, the LLM needs to handle long contexts reliably (tool results accumulate quickly), return well-formed tool calls consistently (a badly structured JSON on step 6 of a 10-step workflow breaks the entire run), and reason about state changes across many sequential tool calls.
The inference provider matters here. You need an API that supports function calling in OpenAI-compatible format, structured outputs to enforce valid JSON at the model level, and sufficient concurrency limits for agent workloads that spawn parallel sub-tasks. Novita AI’s LLM API covers all three with an OpenAI-compatible endpoint, which means you can swap models without rewriting tool-call parsing logic.
3. The Tool Layer
Tools are the agent’s interface with the world. A minimal coding agent needs four:
| Tool | What it does |
|---|---|
read_file | Return the contents of a file at a given path |
write_file | Write a string to a file path |
run_command | Execute a shell command and return stdout + stderr |
list_directory | List files and directories at a path |
Each tool must return complete output. Truncated results or silent failures corrupt the agent’s model of the codebase and cause compounding errors later. The run_command tool especially needs to capture both stdout and stderr—the agent often learns more from error output than from success output.
Some agents add a search_files tool for grep-style lookup across a codebase, or a fetch_url tool for reading external documentation. The right set depends on the task domain. For pure code work, the four above cover most cases.
4. The Sandbox
The sandbox is a complete isolated Linux environment where the agent’s commands actually run. This matters for two reasons.
First, security. Agents generate code from user prompts, retrieved documentation, and inferred patterns. Even a well-intentioned agent can produce code that deletes files, opens network connections, or consumes unbounded resources. A sandbox keeps any damage contained within an isolated environment.
Second, statefulness. A good sandbox preserves filesystem state across tool calls within a session. If the agent creates a file in step 2, it needs to still be there in step 8. Stateless container approaches—where each command runs in a fresh environment—don’t work for real coding tasks.
Novita Agent Sandbox is built on Firecracker microVMs, which give you kernel-level isolation stronger than standard containers. Sessions can run up to 24 hours, filesystem state persists across commands, and cold start is under 200 ms. That’s fast enough that waiting for the sandbox to spin up doesn’t interrupt an interactive workflow.
How the Execution Loop Works
A concrete example makes the loop easier to follow. Say the task is: “Add rate limiting to the /login endpoint.”
-
Plan — the model reads the task and identifies what it needs: find the login route, understand the current handler, add rate-limit middleware, verify with a test run.
-
Observe — the agent calls
list_directoryto find the route files, thenread_fileon the login handler. The file contents are appended to the model context. -
Decide — the model reasons about the current code and decides what to do: install the rate-limit library, modify the handler, add a test.
-
Act — the agent calls
run_command("pip install slowapi"), thenwrite_filewith the modified handler, thenrun_command("pytest tests/test_login.py"). -
Observe again — the test output feeds back into the context. If tests fail, the model reads the traceback, identifies the error, and writes a corrected file.
-
Complete — when tests pass and the model has no pending steps, it returns a final summary.
This loop runs inside a single session. The context window is the agent’s working memory—every file read, every command output, every tool call accumulates there. This is why context length matters so much for coding agents: a real refactoring task can easily fill 100K tokens by step 15. See how agents stress inference providers differently than single-turn chat.
Building a Coding Agent with Novita
The following example wires together Novita’s LLM API and Agent Sandbox. It uses the Python OpenAI SDK pointed at Novita’s endpoint—Novita’s models use the same function-calling interface as OpenAI’s API, so the integration requires no custom parsing.
import os
import json
from openai import OpenAI
from novita_sandbox.code_interpreter import Sandbox
client = OpenAI(
base_url="https://api.novita.ai/openai",
api_key=os.environ["NOVITA_API_KEY"],
)
sandbox = Sandbox.create(timeout=1800)
def read_file(path: str) -> str:
try:
return sandbox.files.read(path)
except Exception as e:
return f"Error: {e}"
def write_file(path: str, content: str) -> str:
try:
sandbox.files.write(path, content)
return f"Written to {path}"
except Exception as e:
return f"Error: {e}"
def run_command(cmd: str) -> str:
try:
result = sandbox.commands.run(cmd)
return str(result)
except Exception as e:
return f"Error: {e}"
tools = [
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read the contents of a file",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
},
},
{
"type": "function",
"function": {
"name": "write_file",
"description": "Write content to a file",
"parameters": {
"type": "object",
"properties": {
"path": {"type": "string"},
"content": {"type": "string"},
},
"required": ["path", "content"],
},
},
},
{
"type": "function",
"function": {
"name": "run_command",
"description": "Run a shell command in the sandbox and return output",
"parameters": {
"type": "object",
"properties": {"cmd": {"type": "string"}},
"required": ["cmd"],
},
},
},
]
dispatch = {
"read_file": read_file,
"write_file": write_file,
"run_command": run_command,
}
def run_agent(task: str, model: str) -> str:
messages = [
{
"role": "system",
"content": (
"You are a coding agent with access to a Linux sandbox. "
"Complete tasks by calling tools. When done, return a plain-text summary."
),
},
{"role": "user", "content": task},
]
while True:
response = client.chat.completions.create(
model=model,
messages=messages,
tools=tools,
tool_choice="auto",
)
msg = response.choices[0].message
messages.append(msg)
if not msg.tool_calls:
return msg.content
for call in msg.tool_calls:
fn = dispatch[call.function.name]
args = json.loads(call.function.arguments)
result = fn(**args)
messages.append(
{
"role": "tool",
"tool_call_id": call.id,
"content": result,
}
)
# Replace <model-id> with a function-calling model from novita.ai/docs
result = run_agent(
task="Write a Python script that counts words in a text file and run it on a sample input",
model="<model-id>",
)
print(result)
sandbox.kill()A few things to note about this implementation:
- The
while Trueloop runs until the model returns a message with no tool calls—that’s the signal the agent considers the task done. - Tool results are appended to
messagesasrole: toolentries. This is what builds up the shared context across steps. sandbox.kill()releases the compute resources. Always call it when the session ends.
For supported function-calling model IDs, check the Novita function-calling docs. For a more complete walkthrough including a Gradio UI, see Building a Coding Agent with Novita’s Agent Sandbox.
Choosing the Right LLM for Code Agents
HumanEval and SWE-bench measure single-turn code generation. Agent workloads are different—what actually breaks production code agents is tool call formatting failures. A model that scores well on benchmarks but occasionally returns malformed JSON on complex multi-turn sessions will fail in ways that are hard to debug.
The practical evaluation criteria for AI coding agents:
- Tool call reliability — how consistently does the model return well-formed tool calls across 20+ step sessions?
- Context retention — does the model correctly reference a file it read 40 steps ago?
- Instruction following — does the agent stay on task, or does it start modifying unrelated files?
- Code correctness — does the generated code actually run, or does it require multiple correction loops?
Running a representative set of real coding tasks and measuring task completion rate is more informative than any public benchmark. Pick 20–30 tasks from your own codebase, run them against candidate models, and count how many complete without human intervention.
Inference pricing compounds quickly at agent scale. A single session might consume 200K–500K tokens across all turns. Providers that offer prompt caching and competitive per-token rates change the economics significantly when you’re running hundreds of agent sessions per day.
Open-Source Models as a Cost-Effective Path
Closed-source frontier models have led on coding benchmarks, but the gap with top open-source models has narrowed considerably. Models like DeepSeek V3 and Qwen3 now perform competitively on code generation and tool use tasks—and because they’re served through OpenAI-compatible APIs, switching is a one-line change in the model parameter.
Both are available through Novita’s LLM API. You get the same endpoint, the same function-calling interface, and the same Agent Sandbox integration—without managing GPU infrastructure yourself. This matters because GPU orchestration, batching, and reliability engineering are non-trivial; delegating them to a managed API lets you focus on the agent logic.
Why this matters for coding agents specifically: per-session token costs drive the economics of agentic workloads more than licensing fees. A team running 200 coding agent sessions per day that achieves comparable task completion rates with an open-source model can reduce inference spend substantially without changing their integration code at all.
The practical test: run 50 representative coding tasks with your target model, measure tool call success rate and task completion rate, then compare against cost per session. Benchmark numbers won’t answer this question—your actual workload will.
FAQ
What is the difference between a coding agent and a code assistant?
A code assistant (like GitHub Copilot’s inline suggestions) generates completions and stops. A coding agent executes code, reads the output, and iterates. The defining characteristic is the execution loop: read, decide, act, observe, repeat. See CLI vs IDE Coding Agent for a comparison of how different agent form factors use this loop.
Do I need a sandbox to build a coding agent?
Yes, if the agent will run code generated from user input or external sources. Without isolation, a buggy code generation can damage the host filesystem or consume unbounded resources. Even for internal-only use cases, a sandbox prevents runaway processes from affecting the host. Containers provide basic isolation; microVM-based sandboxes like Novita’s offer stronger kernel-level separation for multi-tenant or security-sensitive workloads.
Can a coding agent work without internet access?
For most pure coding tasks, yes. File read/write and local command execution cover the majority of workflows. Restricting egress inside the sandbox is actually a good default—it prevents generated code from making unexpected external requests and simplifies your threat model.
What determines the best code agent for a given task?
Tool call reliability and task completion rate on your actual workload. Public benchmark rankings are a starting point for shortlisting models, not a final answer. Run your representative tasks, measure completion rate, and factor in per-session token cost. The best code agent for a small startup doing lightweight refactoring may be very different from the best option for an enterprise team running automated PR review at scale.
How long can a coding agent session run?
That depends on the sandbox provider. Novita Agent Sandbox supports sessions up to 24 hours with filesystem state preserved across commands, which covers even extended refactoring or migration tasks without requiring checkpoint/restore logic in your agent code.
