Draft-Then-Refine: Cheap Model Writes, Big Model Edits

Output tokens are where your AI bill lives. On OpenAI’s current table, gpt-5.6-sol charges $5 per million tokens in and $30 per million out, a 6x gap, and Anthropic’s Opus 5 runs the same shape at $5 in and $25 out. Every long answer a frontier model writes for you is billed at the dearest rate on the sheet. So the obvious question for anyone generating text at volume: why is the most expensive model in the building doing the typing?

The draft-then-refine pattern is the answer, and it is the sixth instalment of our AI on a Budget series. A cheap model writes the first pass, an expensive model reviews it and fixes what is wrong. Done well, it takes roughly half off a generation-heavy workload. Done lazily, it costs you more than sending everything to the flagship, and almost every tutorial on the subject skips over that second part. This one does not.

Table of Contents

Why the price sheet begs for this

Look at the spread. OpenAI’s gpt-5.6-luna is $0.20 in and $1.20 out. Its older gpt-5-nano is $0.05 in and $0.40 out, which puts sol’s output at 75x nano’s. Anthropic’s Haiku 4.5 is $1 in and $5 out against Opus 5’s $5 and $25, and Fable 5.1 at $10 in and $50 out sits another 2x above that. Both vendors also cache repeated input at roughly 10% of the base rate (Fable 5.1 cache reads are $0.25 per million, a sharper cut than the rest of the line).

Two things follow. First, the cheap tier is now good enough to produce a competent draft of most routine text: product descriptions, support replies, summaries, first-pass code, boilerplate reports. Second, reading is cheap and writing is dear, so a model that reads 3,000 tokens and writes 300 costs a fraction of one that writes 3,000. That asymmetry is the whole trick.

The pattern in one paragraph

Send the brief to the small model and let it write the full draft. Then send the brief plus the draft to the big model with a narrow instruction: find the errors, the weak claims, the tone misses, and return corrections only. Apply those corrections in code (a find-and-replace, a JSON patch, a list of numbered edits) and you have frontier-grade output at a fraction of frontier-grade output pricing. The big model reads at $5 and writes very little at $30. The small model does the expensive typing at $1.20.

The maths, with the trap included

Take a 2,000-token article from an 800-token brief. Flagship only, on gpt-5.6-sol: $0.004 of input and $0.06 of output, $0.064 per piece.

Now the lazy version of draft-then-refine. Luna drafts it for $0.0026. Sol then reads brief and draft (2,800 tokens, $0.014) and rewrites the whole thing, 2,000 tokens out at $0.06. Total: $0.077. That is 20% dearer than skipping the draft altogether. You paid twice for the typing. This is the version most people build first, and it is why some teams try the pattern and conclude it does not work.

The version that works constrains the refiner’s output. Same draft, same $0.014 read, but sol returns a 400-token list of patches instead of a rewrite: $0.012. Total: $0.029 per piece, 55% below flagship-only. At 10,000 pieces a month that is $640 versus $286, and the difference buys a reasonable laptop every quarter.

On Anthropic the numbers land similarly. Haiku 4.5 drafts for $0.011, Opus 5 patches for $0.024, total $0.035 against $0.054 for Opus alone, a 35% saving. Run the refine stage through the Batch API (50% off at both vendors) and the saving widens again for anything that can wait an hour.

Draft with the cheapest model that produces coherent text for your task. Test luna, nano, Haiku 4.5, or an open model if you followed our local models instalment.
Refine with a hard cap: max_tokens around 20% of the draft length, and a schema that forces patches rather than prose (structured outputs make this reliable).
Put the brief, style rules and any reference docs at the front of both prompts so prompt caching fires on the repeated prefix.
Log the ratio of refine output to draft output. If it creeps above 30%, your drafter is too weak for the task and you should step up a tier.

Where the idea came from

This is the applied cousin of two research threads. Stanford’s FrugalGPT paper (May 2023) framed the “LLM cascade”: ask the cheapest model first, score the answer, escalate only if it fails. On their benchmarks that matched GPT-4 with up to 98% cost reduction, though the tasks were short classification-style queries, so treat the headline number as a ceiling, not a forecast. A 2024 paper from Zhejiang University, Data Shunt+, tested the collaborative version where small and large models split the work: on Amazon sentiment analysis it beat ChatGPT alone (95.64% versus 94.43% accuracy) while cutting cost to 31% of the baseline. A more recent arXiv preprint calls the same idea “response-level speculative decoding”, which is a fair label: the small model speculates, the big model verifies.

The commercial world got there by another route. Every coding agent now runs some flavour of this, with a cheap model applying edits that a frontier model specifies. Morph, which sells exactly that, claims routing and drafting cuts spend 40 to 70% for its customers. Vendor claim, obviously, but the direction matches the independent work.

Where it goes wrong

Three failure modes are common. The refiner rewrites anyway, because you asked it to “improve” instead of “list corrections”, and your cost doubles. The drafter hallucinates a fact the refiner does not have the context to catch, so the correction step needs the same source material the draft step had, which is why the caching advice above matters. And the pipeline adds latency: two sequential calls instead of one. For interactive chat that is a real cost; for anything generated in bulk it is irrelevant.

It also does nothing for the hard 15% of tasks where the small model’s draft is structurally wrong. For those, the cascade approach (grade the draft, escalate outright) beats patching. Combine the two: patch when the draft is 80% there, escalate when it is not, and route on a cheap judge call from the same small model.

What this means

If you generate text at volume and pay a frontier model to write all of it, you are paying the dearest rate on the price list for typing that a model 25x cheaper can do. Draft-then-refine, with the refiner locked to patch-only output, takes 35 to 55% off that line. Build the constrained version first, measure the refine-to-draft ratio, and step the drafter up a tier only if the ratio tells you to.

Bull / Bear / Neutral (not investment advice): Bull: the gap between cheap and flagship output pricing is the widest it has ever been, so the pattern pays more every quarter. Bear: vendors keep cutting mid-tier prices (Sonnet 5 held at $2/$10 through August), which shrinks the saving from adding a second pipeline stage. Neutral: for most small teams the win is real but modest in absolute dollars; the pattern matters most above a few thousand generations a month.

Did you know: the word “draft” comes from the Old English for “to draw or pull”, which is roughly what the refine step is doing to the small model’s prose.

Sources

Related on Top Tool Stack: AI on a Budget: cut your AI costs · Prompt caching

The free stack. One email a week: the AI tools and moves that actually matter, hype filtered out. Subscribe free →