The era of free AI in your IDE is done, and the bills are landing. The good news is that most teams are overpaying for reasons that are entirely fixable, and a handful of changes can cut LLM token costs by 60 to 90% without touching quality. Here are the levers, ranked by how much they save for how little effort.
1. Route by task, not by habit
The single biggest waste is sending every request to your most expensive model out of habit. A classification job, a routing decision or a simple chat does not need Opus or GPT-5.6; it needs the cheapest model that gets it right. Send simple work to a lightweight model like Claude Haiku 4.5 ($1/$5 per million tokens) and only escalate to a flagship when the task genuinely warrants it. This one change often halves a bill on its own.
2. Turn on prompt caching
If you send the same system prompt or the same reference documents over and over, caching means you pay full price once and a fraction thereafter. For anything with a stable preamble, a support bot, a coding assistant, a document Q&A tool, this is close to free money left on the table.
3. Batch anything that can wait
The Batch API gives a flat 50% discount on both input and output tokens for work that tolerates up to 24 hours of turnaround. Overnight reports, bulk classification, data enrichment: none of it needs to be instant. Better still, the batch discount stacks with caching, so the two together compound.
4. Practise context hygiene
Every unnecessary token in your prompt is a token you pay for on every single call. Teams routinely stuff entire documents, full chat histories and boilerplate into a prompt that needs a paragraph. Trim it. The model does not need the whole novel to answer a question about chapter three.
5. Track spend per team and per feature
| Lever | Effort | Typical saving |
|---|---|---|
| Model routing | Medium | Large |
| Prompt caching | Low | Large on repeat work |
| Batch API | Low | Flat 50% (stacks) |
| Context hygiene | Low | Steady, everywhere |
Untracked spend grows unchecked. Per-team, per-feature attribution is the boring lever that turns the other four from good advice into an actual line going down. You cannot cut what you cannot see.
The mindset shift
Use AI where it earns its cost: planning, architecture review, test generation, code review, first drafts. Be sceptical about paying a premium model to do well-specified work you can do faster yourself. Cost discipline is not about using AI less; it is about not paying flagship prices for economy-model jobs.
Related on Top Tool Stack: Best Local LLMs You Can Run at Home · Is ChatGPT Plus Worth It in 2026?
Did you know: the batch discount and prompt caching stack, so a cached, batched job can cost a small fraction of the same work fired off one request at a time at full price. Same output, a quarter of the invoice.