Reducing an LLM API bill is less about finding a cheaper vendor and more about paying the right price for each call. Five levers do most of the work: prompt caching, the Batch API, model routing, prompt compression, and output-token limits. The table below maps the situations where each one applies to the saving it produces and where to find the implementation details, and a worked example further down prices all five against a real Novita AI bill.
Which cost-reduction technique fits your workload?
| Scenario | Technique | Typical saving | Where to implement |
|---|---|---|---|
| The same long system prompt, policy document, or template is sent on every request | Prompt caching | Cached input tokens are billed at a fraction of the input rate, about 92% cheaper on DeepSeek V4 Pro ($0.135 vs $1.60 per million tokens) | Prompt caching on Novita AI |
| A job can run asynchronously, such as offline classification, bulk summarization, or evaluation | Batch API | 50% off input and output tokens on supported models | Batch API on Novita AI |
| One premium model is the default for every task, including simple ones | Model routing | Up to about 98% on short, high-volume tasks where a small model clears the quality bar ($19.20 to $0.26 a month in the worked example) | Model routing |
| Prompts repeat instructions, oversized examples, or boilerplate | Prompt compression | The saving equals the share of input tokens you remove: a 1,200-token prefix cut to 400 removes about 40% of a request’s input, and LLMLingua reports up to 20x compression with limited quality loss | Prompt compression |
| Responses run long because nothing caps them | Output-token limits | Output bills at roughly 2x input (DeepSeek V4 Pro: $3.20 vs $1.60 per million tokens), so capping 900 tokens to 500 removes about 44% of the output | Output-token limits |
Cache-read rates, the batch discount, and model prices from the Novita AI pricing page, checked October 14, 2026; the compression and output-token figures are the worked examples below, with the compression ratio from LLMLingua. Cache and batch discounts apply only to the models that support them.
Read the invoice before you optimize it
An LLM bill is the sum of four things: input tokens, output tokens, cached input tokens, and the retries caused by truncated or malformed answers. Two facts shape every optimization below.
Output usually costs more than input. On DeepSeek V4 Pro, output is $3.20 per million tokens against $1.60 for input, and on Llama 3.1 8B Instruct it is $0.05 against $0.02. Every token you avoid generating is worth more than a token you avoid sending, which is why output limits and prompt compression pay off on both the length and the rate.
Most applications are also input-heavy. A support summarizer, a retrieval-augmented answerer, or a coding agent sends a long prompt (system instructions, retrieved context, conversation history) and gets back a short answer. That is why the levers that shrink or discount input, such as caching, compression, and routing, usually move the monthly number the most, while batching discounts both sides at once.
Model routing: match the model to the task
The default model in most codebases is a single string that never changes. That is convenient and usually expensive: a frontier model summarizes a short ticket as readily as it plans a multi-step migration, and you pay frontier prices for both. Routing replaces that one default with a small map from task to model. The map does not need to be clever, and three tiers cover most products.
| Task type | Tier | Example on Novita AI | Why it fits |
|---|---|---|---|
| Classification, tagging, short extraction, fixed-format rewrites | Small | Llama 3.1 8B Instruct | High volume and short, constrained outputs, so an 8B model holds up |
| Summarization, standard chat, retrieval synthesis | Balanced | DeepSeek V4.1 Flash | Long context at a low input price, so long prompts stay affordable |
| Multi-step reasoning, planning, complex coding | Frontier | DeepSeek V4 Pro | Reserve the most expensive tokens for work that needs them |
A routing map is a dictionary and one function call. This example keeps the model choice in configuration rather than scattered through the codebase, so a tier can move without a rewrite:
from openai import OpenAI
client = OpenAI(
api_key="<Your API Key>",
base_url="https://api.novita.ai/openai",
)
# Route by task, not by a single global default.
ROUTES = {
"classify": "meta-llama/llama-3.1-8b-instruct", # short labels, high volume
"summarize": "deepseek/deepseek-v4.1-flash", # long input, short output
"reason": "deepseek/deepseek-v4-pro", # multi-step planning
}
def complete(task: str, messages: list[dict]) -> str:
response = client.chat.completions.create(
model=ROUTES[task],
messages=messages,
max_tokens=512,
)
return response.choices[0].message.contentTwo cautions come with routing. It is only cheaper if the smaller model clears your quality bar, so run a golden set of real requests through the candidate before you move traffic, and keep the frontier model as a fallback for the requests the small model fails. And measure cost per completed task, not cost per token: a cheap model that returns longer answers or needs a retry can cost more than its rate card suggests.
Prompt compression: send fewer input tokens
Prompt compression is the plainest lever. Fewer input tokens at the same rate is a smaller bill, with no platform feature involved, and the saving is exactly the share of input tokens you delete. Most bloated prompts carry three kinds of removable weight:
- Repeated instructions, where the same formatting rule is stated in the system prompt and again in the user turn.
- Oversized examples, where two few-shot examples would cover the pattern that six are demonstrating.
- Verbose retrieved context, where the two relevant paragraphs answer the question that a whole document was pasted in to answer.
A system prompt that states its formatting rules three times and carries six examples can often be cut to a third of its length without changing the output. If that prefix is 60% of a request’s input, cutting it from 1,200 tokens to 400 removes about 40% of the input tokens and about 40% of the input cost. Compressing prompts this aggressively is the basis of Microsoft Research’s LLMLingua work, which shows that prompts can be shortened substantially with limited quality loss.
The tradeoff is real: compress too far and the model loses the instruction it needed. Change one thing at a time, keep a sample of real requests as a regression set, and check the output before trimming further.
Output-token limits: stop paying for words you do not need
Output tokens are the expensive half of a call, and nothing caps them unless you do. Two settings control the ceiling. max_tokens sets the hard limit on the response, so a summarizer that never needs more than 300 tokens is not allowed to generate 900. A concise instruction in the system prompt, such as “answer in at most three bullet points,” shapes the length before the limit is ever reached.
If your average response is 900 tokens but 400 would answer the question, capping output at 500 tokens removes about 44% of output tokens, and output is where the price is highest. The risk is truncation: set the limit too low and the model stops mid-sentence, which can trigger a retry that costs more than it saved. Set the limit from the observed distribution of real answers rather than a guess, and leave headroom for the longest legitimate response.
Prompt caching: pay the full rate once
Prompt caching targets the one prompt you send over and over. When a request matches a prefix the system has seen before, it returns the cached result and bills the repeated tokens at the cache-read rate instead of the full input rate. On DeepSeek V4 Pro that is $0.135 against $1.60 per million tokens, about 92% off the repeated portion, and on DeepSeek V4.1 Flash it is $0.006 against $0.30.
Two design rules make caching pay off: keep the reused prefix identical across requests, and put everything that changes at the end. The API needs no extra parameters, but a cache hit only happens when the prefix matches exactly, so a timestamp or a reordered example near the top breaks it. The full pricing table, the prompt structure that produces reliable hits, and the cases where caching does not help are covered in Prompt Caching: Use Novita AI for Stable Low-Cost Caching.
Batch API: 50% off when latency does not matter
The Batch API trades latency for price. Novita AI’s Batch API applies a 50% discount to input and output tokens on supported models, processing requests asynchronously and returning results within 24 hours. It accepts OpenAI-compatible JSONL input, runs up to 50,000 requests per batch, and offers higher rate limits than online calls.
That makes it a fit for work that does not need an answer this second: offline document summarization, dataset classification, evaluation runs, and backfills. It is not a fit for anything a user is waiting on. The supported-model list, the request and retrieval flow, and error handling are in Batch API: Reduce Bandwidth Waste and Improve API Efficiency.
Worked example: a 10-million-token support summarization bill
Put the levers on the same task. A support team summarizes customer conversations with DeepSeek V4 Pro, sending 8 million input tokens and generating 2 million output tokens a month, 10 million tokens in total. Here is what that workload costs under each approach.
| Approach | Model | Input cost | Output cost | Monthly total | vs. default |
|---|---|---|---|---|---|
| Default | DeepSeek V4 Pro | $12.80 | $6.40 | $19.20 | n/a |
| Route to a small model | Llama 3.1 8B Instruct | $0.16 | $0.10 | $0.26 | -98.6% |
| Prompt cache (60% of input cached) | DeepSeek V4 Pro | $5.77 | $6.40 | $12.17 | -36.6% |
| Batch API (50% off) | DeepSeek V4 Pro | $6.40 | $3.20 | $9.60 | -50.0% |
| Prompt cache plus Batch API | DeepSeek V4 Pro | $2.88 | $3.20 | $6.08 | -68.3% |
Prices from the Novita AI pricing page, checked October 14, 2026. Token volumes are this example’s assumption; the cache row assumes 60% of input tokens hit the cache, and the batch row assumes the model is on the Batch API supported list.
The spread is the point. Routing to a small model cuts the bill by more than 98%, but only if an 8B model summarizes support conversations well enough for your reviewers, which is a quality decision rather than a pricing one. Caching and batching leave quality untouched because the same model answers, but they apply to narrower situations: caching needs a reused prefix, and batching needs a job that can wait. Combined, caching and batching take the same model’s bill down by about 68% without changing the model at all.
There is a fourth combination worth pricing. Llama 3.1 8B Instruct has no cache-read rate, but DeepSeek V4.1 Flash does, at $0.006 per million cached tokens. Routing the summarization job to a balanced model and caching its reused prefix keeps the quality of a larger model while landing the same 10 million tokens in the low single-digit dollars.
Which lever to pull first
A practical order matters, because some changes are safer than others:
- Measure the split. Log input, output, cached, and retry tokens per task before changing anything, so you can see which lever has room to move.
- Cache the stable prefix, if you have one. It is the lowest-risk change and needs no model swap.
- Route the easy tasks to a smaller model. For a mixed workload this is usually the largest single lever.
- Move async jobs to the Batch API. Apply it to backfills, evaluations, and bulk summarization first.
- Compress prompts and cap output. Do this last, because it changes model inputs and outputs and needs its own quality checks.
The levers stack. A cached, batched, right-sized request can cost a small fraction of the same request sent to a frontier model at the full rate.
When these techniques do not help
None of the five levers is universal, and knowing where they stop is part of using them well.
- If every request is a short, unique prompt with no shared prefix, caching has nothing to reuse and will not hit.
- If users are waiting on the response, the Batch API is out, because its results arrive within 24 hours rather than seconds.
- If your traffic genuinely needs a frontier model on every call, such as safety-critical reasoning or complex coding, routing will not help without a quality regression.
- Aggressive prompt compression and tight output limits trade completeness for cost, and if the answer must be exhaustive they can cost more in retries than they save.
- Cache and batch discounts apply only to supported models, and cache hits require exact prefix matches.
Conclusion
Reducing LLM API costs comes down to five levers and the order you pull them: cache the prefix you reuse, route the easy tasks to smaller models, batch the jobs that can wait, compress the prompts that carry dead weight, and cap the outputs that run long. The same 10-million-token summarization workload costs $19.20 a month on a single frontier default and $6.08 with caching and batching applied to the same model, and less again if a smaller model clears the quality bar. Start by measuring the split between input, output, cached, and retry tokens; that tells you which lever is worth pulling first.
FAQ
What is the cheapest way to reduce LLM API costs?
There is no single switch. The largest and safest savings usually come from matching the model to the task, because routing a simple job off a frontier model can cut the bill by an order of magnitude, followed by prompt caching for any reused prefix and the Batch API for jobs that can run asynchronously. Prompt compression and output limits add more, but they change model inputs and outputs and need quality checks.
Does model routing hurt quality?
It can, if you route a task to a model that cannot do it. Routing only saves money when the smaller model clears your quality bar for that specific task, so validate candidates on a golden set of real requests and keep the frontier model as a fallback. Tasks with short, constrained answers, such as classification and fixed-format extraction, tolerate smaller models far better than multi-step reasoning does.
Can I use prompt caching and the Batch API together?
Yes. They address different parts of the cost. Caching discounts the input tokens you reuse across requests, and the Batch API applies a 50% discount to input and output tokens for asynchronous jobs. In the worked example above, combining them takes the same model’s monthly bill from $19.20 to $6.08. The one constraint is that both apply only to models that support them.
How much does the Novita AI Batch API discount?
The Batch API applies a 50% discount to input and output tokens on supported models. It processes requests asynchronously with results returned within 24 hours, accepts OpenAI-compatible JSONL input, and supports up to 50,000 requests per batch.
Is prompt compression safe for output quality?
It is safe up to a point. Removing genuinely redundant instructions and oversized examples usually leaves quality unchanged, but compressing too far strips context the model needed. Treat it like any other prompt change: adjust one element at a time and verify against a sample of real requests before rolling it out.
