Structured Outputs: Stop Paying for Broken JSON

OpenAI measured its own models on a hard JSON schema benchmark. The older gpt-4-0613 scored under 40 percent. A newer model with no schema enforcement reached 93 percent. Sounds close to done, until you notice that 93 percent means roughly one response in fourteen breaks your parser, and every broken response is a retry you pay for twice.

This instalment of our AI on a Budget series is about the dullest cost lever there is, and one of the few that also makes your app more reliable: structured outputs, plus the small prompt habits that stop you paying premium prices for words nobody reads.

Table of Contents

What structured outputs actually guarantee

Asking nicely for JSON is a prompt. Structured outputs are a constraint on the decoder. OpenAI converts your JSON Schema into a context-free grammar and, after every token, masks out any token that would break it. On that same hard benchmark, the enforced version scored 100 percent schema adherence.

Anthropic offers the same idea. In the current Claude API you pass output_config.format with type: "json_schema", or set strict: true on a tool definition. The old beta header (structured-outputs-2025-11-13) is only needed for the deprecated output_format parameter. Claude Opus and Sonnet 4.5 and later support it, Haiku 4.5 too.

In OpenAI strict mode, every field must be listed as required and additionalProperties must be false. Anthropic ignores minimum, maximum, minLength and maxLength in the schema itself; the SDK moves those constraints into field descriptions, so they become polite requests again.

Why it saves money

The saving is in not retrying. If one response in fourteen fails validation and you re-send the request, you are paying roughly 7 percent more on that route, plus the latency. Add the glue code that strips markdown fences, repairs trailing commas and re-asks, and the cost is engineering time as well. Constrained decoding removes the whole category of failure: the output parses, or the call tells you why it did not.

It also removes the temptation to pad prompts with “respond ONLY with valid JSON, no commentary, I mean it” and three worked examples. Those tokens are billed on every call.

The costs and limits nobody puts in the headline

First-request latency. New schemas have to be compiled. OpenAI says typical schemas take under 10 seconds, complex ones up to a minute. Anthropic caches compiled grammars for 24 hours from last use, and times out compilation at 180 seconds.
It does not check the content. OpenAI states plainly that it cannot prevent mistakes inside field values. A perfectly shaped wrong answer is still wrong.
Truncation breaks it. Hit max_tokens and Claude returns stop_reason: "max_tokens" with an incomplete object. A refusal returns a 200 status, bills the tokens, and may not match the schema. Check the stop reason before you parse.
It touches your prompt cache. On Anthropic, structured outputs inject an extra system prompt, so input tokens run slightly higher, and changing output_config.format invalidates the cache for that conversation. Pin your schema. Our prompt caching piece explains why that matters.
Complexity caps. Claude allows 20 strict tools per request, 24 optional parameters and 16 union-type parameters across all strict schemas. Flatten nested structures and mark things required where you can.
Incompatibilities. Claude structured outputs cannot be combined with citations (you get a 400) or with message prefilling. OpenAI strict mode does not work with parallel function calls.

Prompt hygiene: shrink the expensive side

Output tokens cost 5 to 6 times more than input tokens on the big models. In April 2026 pricing, Claude Opus 4.7 was $5 in and $25 out per million tokens, GPT-5.4 $2.50 and $15, Gemini 3.1 Pro $2 and $12. Generation is slow because each token needs a pass over the model weights, which is why providers charge for it.

So the cheapest habits all attack output:

Return codes, enums and booleans instead of sentences. “category”: “billing” costs a handful of tokens; a paragraph of reasoning costs hundreds.
Keep key names short and few. Every key is generated on every response, every time.
Set max_tokens to what the schema needs, not the model maximum, and handle the truncation case on purpose.
Cut the preamble from your prompt. Long instructions and spare examples are billed per call; schema descriptions can carry most of that weight.
Structure your prompt with static content first and variable content last, so caching can discount the fixed part by roughly 90 percent.
If the job can wait, Anthropic confirms structured outputs work with the batch API at the 50 percent discount.

What this means

Any pipeline that parses model output with a regex or a prayer should move to schema-enforced output this week. Cost: an afternoon. Saving: every retry and every parser-repair branch.
Budget the first-call compile time, and keep one stable schema per route so it stays cached.
Validate the values as well as the shape. Schema compliance is not correctness.
Audit your worst route for output length. Trimming a verbose answer to a 20-token label is a bigger saving than most model downgrades, since output is the 5 to 6x side of the bill.

The honest downside: you gain reliability and give up some flexibility. Strict schemas make it awkward to let a model improvise, and tight schemas can nudge it into forcing an answer into a box that does not fit. Add an “unknown” or “other” option.

Did you know: OpenAI chose a context-free grammar over a simple state machine because finite-state approaches cannot express deeply nested, recursive JSON.

Sources

Related on Top Tool Stack: AI on a Budget: cut your AI costs · Prompt caching

The free stack. One email a week: the AI tools and moves that actually matter, hype filtered out. Subscribe free →