LLM cost: practical ways to cut your API bill without losing quality

The first AI prototype is always cheap. Then it goes into production, volume grows, someone adds an agent loop, and the monthly bill becomes a line item that finance asks about. Most of that spend is avoidable, and you rarely have to give up quality to cut it.

This is the checklist I go through when an LLM system costs more than it should. Nothing here depends on one provider. Prices and features change often, so check your provider's current pricing page for the details.

Where LLM cost actually comes from

You pay for tokens: the tokens you send (input) and the tokens the model writes back (output). Output tokens usually cost more than input tokens. So the bill is roughly:

• how many calls you make,
• how much context you send with each call,
• how long the answers are,
• and which model you send them to.

Every technique below attacks one of those four.

1. Measure cost per task before optimising

Most teams know their monthly total and nothing else. That's not enough to act on. Log, for every call: which feature or workflow made it, the model, input and output tokens, and whether the result was used.

Then compute cost per task: per support ticket answered, per document processed, per lead qualified. You'll usually find that one or two workflows account for most of the spend, and that some calls produce results nobody uses. Fix those first.

2. Right-size the model for each step

The biggest lever is not sending every request to the most expensive model. Most pipelines contain a mix of steps:

• Easy steps: classifying a message, extracting fields from a form, detecting language, routing. Small models do these well.
• Medium steps: drafting replies in a known format, summarising a document. Mid-sized models are usually enough.
• Hard steps: multi-step reasoning, messy edge cases, writing code. This is where the largest models earn their price.

A simple router that sends each request to the cheapest model that handles it reliably often cuts costs sharply. Test it on your own data: build a small evaluation set and check that the cheaper model's answers are still good enough for that step.

3. Use prompt caching

Many requests start with the same long prefix: system instructions, tool definitions, a policy document, a product catalogue. Several major providers offer prompt caching, which charges less for a repeated prefix than for fresh input.

To benefit, structure prompts so the stable part comes first and the changing part (the user's message) comes last. Don't put timestamps or request IDs at the top of the prompt, or the prefix never matches.

4. Send less context

Long context is convenient and expensive. Common waste:

• Stuffing whole documents into the prompt when only two paragraphs are relevant. Retrieve the relevant chunks instead (that's what RAG is for; see building a RAG pipeline that works in production).
• Resending the entire conversation history on every turn. Summarise older turns or keep only what's needed.
• Tool outputs dumped raw into the context. Trim logs and search results before the model sees them.

Shorter context is often more accurate too. Models get distracted by irrelevant text.

5. Ask for less output

Output tokens cost more, so ask for exactly what you need. If your code needs three fields, ask for three fields in JSON, not a friendly paragraph that contains them. Structured outputs also cut retries: when the model returns valid, parseable data the first time, you don't pay for a second call to fix the format.

6. Batch work that isn't urgent

Not everything needs an answer in two seconds. Nightly report generation, re-tagging an archive, enriching a CRM: major providers offer batch processing at a discount, in exchange for results that arrive later. If a workflow can wait hours, it probably shouldn't run at full real-time price.

7. Cache results, not just prompts

If many users ask the same question, or the same document gets processed twice, store the answer. A plain cache keyed on the normalised input is often enough. FAQ-style traffic tends to repeat itself.

8. Put a ceiling on agents

Agent loops are where costs explode quietly. An agent that retries a failing tool call, or keeps "exploring" with a growing context, can burn through a lot of tokens on one task. Set hard limits: a maximum number of turns, a token budget per task, and a timeout. Log when a limit is hit, because that usually points to a bug. (In Claude Code, for example, subagents have a maxTurns setting; I cover it in writing Claude Code subagents.)

9. Run steady, high-volume work locally

For workloads that run all day at a predictable volume, paying per token can cost more than owning or renting the hardware. Open models running on your own GPU cost the same whether you process a hundred documents or a hundred thousand.

My own example is VidForge, an autonomous multi-agent pipeline that turns a chosen topic into a finished YouTube video. The heavy lifting (narration, generated images, the on-screen host, upscaling) runs on local models on a single 16 GB consumer GPU, queued so one model runs at a time. That's why a finished video costs almost nothing to make.

Local models also keep data in-house, which matters for privacy and data protection. I go through the trade-offs in private AI and on-premise LLMs.

10. Watch the language

Token counts depend on the language. Arabic text, for example, often takes more tokens than English for the same meaning, so the same assistant can cost more per message in Arabic. Measure it on your own traffic, and factor it into model choice. More on that in building an Arabic AI chatbot.

A sensible order

If you're starting from a large bill, this order usually gives the fastest wins:

1. Measure cost per task and kill unused calls.
2. Add limits to agent loops.
3. Route easy steps to smaller models.
4. Restructure prompts for caching and trim context.
5. Move non-urgent jobs to batch.
6. For steady high-volume work, evaluate a local model.

Re-run your evaluation set after each change. A cheaper system that gives worse answers isn't cheaper once you count the cleanup.

Working together

I'm Driss Briksine, a software and ML engineer. I build AI systems end to end and run much of my own work on local models, so cost is a design constraint from day one, not an afterthought. See my work, or send me a short brief if your LLM bill is growing faster than your usage.