AI APIs are cheap for experiments but can become expensive at scale. The good news is that a few changes often cut a bill by half or more without making the results worse. Here are the biggest levers, roughly in order of impact.
First, measure
Every API response reports how many input and output tokens it used. Log those numbers before changing anything, so you know where the money actually goes. Output tokens usually cost about four to six times as much as input tokens, so long replies add up quickly.
1. Use the smallest model that does the job well
Prices between a provider's smallest and largest models can differ by 40 times or more. Simple jobs such as classifying messages, extracting fields or short summaries often work just as well on a small model. Test the cheaper model on real examples, and only move up if quality suffers. Many apps use a small model for most requests and a large one only for the hard ones.
2. Prompt caching
If many requests start with the same long text, such as a system prompt, a product manual or a large set of instructions, prompt caching lets the provider reuse it. With Anthropic's API, reading cached input costs a small fraction of the normal input price (about a tenth on most models, and less on some newer ones). Writing to the cache costs a little extra the first time, so caching pays off when the same prefix is reused.
Caching matches the start of the prompt exactly, so put the unchanging parts first and anything that varies, like the user's question, at the end. OpenAI and Google offer their own forms of caching too. Check each provider's pricing page for the details, or work out your own savings with the prompt caching cost calculator.
3. Batch processing
If results don't have to be instant, for example tagging thousands of products overnight, batch APIs process requests in the background for half the normal price. Anthropic, OpenAI and Google all offer a batch option.
4. Ask for shorter outputs
Because output is the expensive part, ask for exactly what you need: "Reply with one word: positive, negative or neutral", or a JSON object rather than an essay. Also set max_tokens as a safety limit.
5. Send less context
- Trim or summarise long conversation histories. See context windows explained.
- Send only the relevant parts of documents, using search or RAG, rather than everything.
- Remove boilerplate, repeated instructions and unused examples from prompts.
A worked example
Imagine an app handling 1,000 requests a day, each with a 1,500-token system prompt, 500 tokens of user input and a 500-token reply. Using Claude Sonnet 5.5 prices checked on 19 November 2026 ($2 per million input tokens, $10 per million output, $0.10 per million for cache reads):
def monthly_cost(requests_per_day, input_tokens, output_tokens,
price_in, price_out, cached_tokens=0, price_cache_read=0):
fresh = input_tokens - cached_tokens
per_request = (fresh * price_in + cached_tokens * price_cache_read
+ output_tokens * price_out) / 1_000_000
return per_request * requests_per_day * 30
plain = monthly_cost(1000, 2000, 500, 2, 10)
cached = monthly_cost(1000, 2000, 500, 2, 10, cached_tokens=1500, price_cache_read=0.10)
batch = plain / 2
print(f"Plain: ${plain:,.2f} a month")
print(f"Cached: ${cached:,.2f} a month")
print(f"Batch: ${batch:,.2f} a month")
Plain: $270.00 a month
Cached: $184.50 a month
Batch: $135.00 a month
(This ignores the small one-off cost of writing the cache.) Notice that caching barely touches the output cost, which here is $150 of the $270. Shorter replies or a smaller model would save more. Moving the same workload to Claude Haiku 5.5 at $0.10 and $0.50 per million would cost about $13.50 a month, if it handles the task well enough.
Prices change often. Compare current prices for your own numbers with the AI API cost calculator.
Going deeper
When does caching pay off?
On Anthropic's API, writing to the default 5-minute cache costs 1.25 times the normal input price, and reading costs about 0.1 times on most models. Writing to the 1-hour cache costs 2 times. Compare reusing a cached prefix with sending it uncached each time:
def cache_vs_plain(requests, write_mult=1.25, read_mult=0.10):
cached = write_mult + (requests - 1) * read_mult # first request writes, the rest read
plain = requests * 1.0
return cached, plain
for n in (1, 2, 5, 20):
c, p = cache_vs_plain(n)
print(f"{n:>2} requests: cached {c:.2f}x vs plain {p:.0f}x the prefix price")
1 requests: cached 1.25x vs plain 1x the prefix price
2 requests: cached 1.35x vs plain 2x the prefix price
5 requests: cached 1.65x vs plain 5x the prefix price
20 requests: cached 3.15x vs plain 20x the prefix price
With the 5-minute cache, a prefix reused even once is already cheaper. The 5-minute timer resets each time the cache is read, so steady traffic keeps it warm. Some newer models charge even less for cache reads.
Common reasons caching silently fails
- Anything that changes in the cached part, such as a timestamp or user name near the top of the system prompt, creates a new cache entry each time.
- Prompts shorter than the model's minimum cacheable length aren't cached at all.
- Changing the list or order of tools changes the prefix.
The API's usage figures report cache reads and writes separately, so check them to confirm caching is actually working.
Routing between models
A common pattern is to send every request to a small model first, and escalate to a larger one only when needed: if the small model says it's unsure, if its output fails a check, or for request types known to be hard. Even sending 80% of traffic to a model that costs a tenth as much cuts the bill dramatically.
Stack the discounts
Batch processing and caching can be combined, and both work with smaller models. For a large nightly job, the cheapest setup is usually the smallest model that passes your tests, run through the batch API, with the shared instructions cached.
Watch the hidden costs
- Thinking tokens on reasoning models are billed as output. Lower the effort setting for simple tasks where the provider allows it.
- Agent loops resend the growing conversation on every step, so long agent runs can cost far more than their final answer suggests.
- Retries after failures repeat the full cost. Fix the cause rather than retrying repeatedly.
Sources
The main references behind this guide. Numbers we worked out ourselves, such as the code outputs, come from running the code.