Context Wire
Tool Reviews

JSON Mode vs Structured Outputs: Which Should You Use?

Compare JSON mode and schema-constrained outputs by guarantees, provider limits, validation, retries, latency, cost, and production use case.

Published
  • #JSON mode
  • #structured outputs
  • #JSON Schema
  • #LLM APIs

JSON mode asks a model API for syntactically valid JSON. Structured Outputs adds a schema constraint, so the response must also have the keys, types, and permitted values your application expects. That difference determines how much validation and retry logic you need.

This article uses “Structured Outputs” as the general name for provider features that constrain a final response to a supplied schema. Product names and request fields vary. Feature, limit, and compatibility statements were checked against official OpenAI, Microsoft Azure, Google Gemini, and Anthropic documentation on 2026-07-24.

Quick answer

Use Structured Outputs when downstream code depends on a stable contract: extracting records, rendering a known UI, or feeding a typed pipeline. Use JSON mode when you need parseable JSON but cannot use the provider’s schema-constrained feature or must support an older model.

The practical distinction is simple:

QuestionJSON modeStructured Outputs
Is the response intended to parse as JSON?YesYes
Are required keys and value types enforced by the model API?NoYes, within the provider’s supported schema subset
Can values still be factually or semantically wrong?YesYes
Do you still need application validation?Yes: syntax edge cases, shape, and meaningYes: refusal or truncation handling plus meaning and business rules
Does the request depend on a provider-supported schema subset?NoYes
Is it the right interface for requesting a tool action?No; use function or tool callingNo; use function or tool calling
Is it broadly available on older models?Often broaderUsually narrower and provider-specific
Is it a good default for a typed production integration?Usually a fallbackUsually the first choice

OpenAI explicitly recommends Structured Outputs over JSON mode when supported. Its official comparison says both produce valid JSON, but only Structured Outputs provides schema adherence (OpenAI Structured Outputs documentation). Microsoft gives the same recommendation for Azure OpenAI (Azure JSON mode documentation).

For tool arguments, use the provider’s function or tool-calling interface. Use a structured response format when the model’s final answer needs a fixed shape. The schema machinery may be similar, but the control flow is different.

What JSON mode guarantees

JSON mode removes a common failure class: prose around an object or another response that cannot be parsed as JSON. You normally enable a response format such as json_object, then describe the desired object in the prompt.

The guarantee stops at JSON syntax. If you ask for:

{
  "ticket_id": "string",
  "priority": "low | medium | high",
  "needs_human": "boolean"
}

JSON mode can still return valid JSON with ticketId instead of ticket_id, omit needs_human, encode the Boolean as "yes", or invent a priority outside the allowed set. Every example is parseable; none necessarily satisfies your contract.

OpenAI says JSON mode requires an explicit instruction to generate JSON; without it, generation may continue with whitespace until the token limit. The API checks that “JSON” appears in the context. Azure also tells clients to inspect a length-related finish reason because a response can be cut off (OpenAI JSON mode notes, Azure JSON mode troubleshooting).

Treat JSON mode as a transport-format control, not a data contract. Pair it with a runtime schema validator, output limit, refusal and truncation handling, and a bounded repair strategy.

What schema-constrained output guarantees

Structured Outputs uses a supplied JSON Schema, or a provider-specific subset of it, to constrain generation. A successful normal response should therefore contain the required properties, use the declared types, obey supported enums, and reject unspecified properties when the schema requires that behavior.

That stronger contract lets typed services and UIs consume a known shape without guessing which keys arrived. It can also eliminate repair calls caused by the wrong shape.

Schema compliance does not prove that the content is correct. A model can emit a valid priority: "high" for a low-priority ticket, put an incorrect date in a correctly typed string, or produce a made-up identifier that matches a pattern. Google’s official guidance says structured output guarantees syntactically correct JSON but not semantically correct values, and tells developers to validate those values in application code (Gemini Structured Outputs documentation).

OpenAI documents refusals and token-limit truncation as cases requiring explicit handling even with Structured Outputs. Anthropic documents invalid-output and schema-complexity cases. Your integration therefore needs two branches:

  1. a normal, schema-compliant result that proceeds to semantic validation; and
  2. a non-result state such as refusal, truncation, provider error, or schema-compilation error.

Before any side effect, check authorization, resource ownership, ranges, state transitions, idempotency, and policy. The guide to machine gates for AI output covers this boundary.

Provider support and schema limits

“Supports JSON Schema” rarely means every keyword in every draft of the specification. Check the exact model, endpoint, request field, supported subset, and incompatibilities before designing a shared abstraction.

Provider pathDocumented position on 2026-07-23Important constraints
OpenAI APIStructured Outputs is available on newer models starting with GPT-4o; older models may use JSON modeSupports a subset of JSON Schema. Unsupported strict schemas return an error. OpenAI lists supported types including objects, arrays, enums, and anyOf, while excluding some composition keywords.
Azure OpenAIJSON mode and Structured Outputs are separate response formats; Microsoft recommends Structured Outputs when possibleAzure documents model and API-version compatibility, requires every field to be listed as required for its supported subset, requires additionalProperties: false on objects, and does not support Structured Outputs in bring-your-own-data scenarios. Its documented maximum is five nesting levels and 100 total object properties.
Google Gemini APIStructured output accepts a schema and JSON response MIME type through provider-specific request fieldsGemini supports a subset of JSON Schema and warns that very large or deeply nested schemas may be rejected. Supported keywords and models vary by API path.
Anthropic Claude APIJSON outputs constrain final responses; strict tool use applies schemas to tool inputsSchema grammars are compiled and cached. Anthropic documents limits including 20 strict tools, 24 optional parameters, and 16 union-type parameters across strict schemas in one request. Citations and message prefilling have documented incompatibilities with JSON outputs.

Sources: OpenAI supported schemas, Azure Structured Outputs, Gemini Structured Outputs, and Anthropic Structured Outputs.

For a multi-provider system, keep a canonical application schema, then generate and test an adapter for each endpoint. Fail during deployment if an adapter uses unsupported keywords rather than discovering the mismatch in production.

Version schemas independently from prompts, store a schema ID with each response, and test old fixtures against migrations.

Validation and retry behavior

A production pipeline should distinguish four failure layers:

  1. Transport: timeout, rate limit, authentication error, or provider outage.
  2. Completion: refusal, truncation, cancellation, or another non-normal finish state.
  3. Structure: invalid JSON or mismatch with the application schema.
  4. Semantics: structurally valid data that violates business rules or lacks evidence.

JSON mode mainly reduces layer three’s syntax failures; you still validate the whole schema. Structured Outputs reduces both syntax and supported-schema failures on normal completions, but layers one, two, and four remain.

Use a local validator even when the provider promises schema adherence. It catches adapter mistakes, regressions, and cached responses produced under an older schema. Record the provider, model identifier, schema version, finish state, validator result, and retry reason without unnecessarily logging sensitive content.

Retries should be narrow. Retry transient transport failures with backoff. For JSON mode, a repair attempt can include concise validator errors, but cap the loop. After a deterministic schema rejection, correct the schema rather than repeating the request. Do not retry refusals as malformed JSON. Make side effects idempotent.

A provider fallback also needs contract tests. If the fallback supports only JSON mode, the application must re-enable full shape validation and perhaps a repair path. The LLM fallback strategy explains why a provider switch must preserve behavior, not merely return HTTP 200.

Latency and cost trade-offs

Structured Outputs can lower end-to-end cost by preventing repair calls and downstream exceptions. It can also add request tokens, provider-side processing, or grammar compilation. The net result depends on schema complexity, failure rate, traffic, and provider implementation.

OpenAI documents additional latency on the first request with a new schema, followed by no such added latency for later requests using the same schema; one section limits the note to fine-tuned models while another states it more generally, so the exact scope should be rechecked for the chosen model before making a latency commitment [VERIFY]. Anthropic is more explicit: the first use compiles a grammar, compiled grammars are cached for 24 hours after last use, and changes to schema structure or the set of tools invalidate that cache. Anthropic also says its injected format instruction slightly increases input-token usage (OpenAI latency notes, Anthropic compilation and token-cost notes).

Do not assume JSON mode is cheaper because it has fewer controls. Measure the complete transaction:

effective cost per accepted record
= (model calls + repair calls + validator compute + failed-work handling)
  / accepted records

Benchmark representative schemas and inputs. Separate cold- and warm-schema latency, track p50 and p95 time to an accepted result, and count retries, refusals, truncations, rejections, tokens, and operator interventions. Per-token price alone cannot identify the cheaper mode.

Decision by use case

Choose according to the strictest downstream consumer:

  • Data extraction into a database: Prefer Structured Outputs, then enforce domain rules and database constraints.
  • Classification into a fixed taxonomy: Prefer Structured Outputs with enums; still test labeling accuracy.
  • A final response rendered by a typed UI: Prefer Structured Outputs, with a safe renderer for every permitted variant.
  • Tool or function arguments: Use strict function calling when available, not a final JSON response that your code treats as an implicit tool request.
  • Exploratory analysis with a flexible object: JSON mode can be sufficient if the object is inspected by a person or validated before use.
  • Older-model or endpoint compatibility: Use JSON mode plus local schema validation and a bounded repair path.
  • Multi-provider fallback: Use Structured Outputs where each provider supports the canonical contract; otherwise place a full validator at the shared boundary and test every adapter.
  • High-risk side effects: Neither mode is an authorization control. Require deterministic checks and, where appropriate, human approval after validation.

For a new typed integration, start with Structured Outputs and the smallest schema that expresses the real contract. Flatten unnecessary nesting, encode optional values in the provider-supported way, and keep response schemas separate from tool schemas. Add JSON mode only when compatibility requires it.

For an existing JSON-mode integration, migration is worthwhile when repair retries, missing keys, or type errors are measurable. Run both paths against a fixed evaluation set, compare accepted-result latency and cost, then switch behind a feature flag. If you are also moving OpenAI endpoints, use the Responses API migration guide.

The durable rule is: use generation constraints for shape, application code for meaning, and authorization controls for action. Structured Outputs narrows the model’s freedom at the right boundary; it does not turn probabilistic content into trusted business data.

Frequently asked questions

Does JSON mode guarantee the same keys every time?

No. JSON mode targets valid JSON syntax, not a specific schema. The response can still omit a required key, change a key name, return the wrong type, or add an unexpected property. Validate the full object locally and use a bounded repair path if Structured Outputs is unavailable.

Do Structured Outputs eliminate validation code?

No. They reduce shape failures for supported schemas on normal completions, but your application must still handle refusals, truncation, provider errors, and business-rule failures. A schema can prove that amount is a number; it cannot prove that the amount is authorized or correct.

Should function calling or Structured Outputs be used for tool arguments?

Use the provider’s function- or tool-calling interface when the model needs to request an action. Use a structured response format when the final answer itself must have a fixed shape. If tool execution is failing despite valid-looking JSON, see how to fix LLM tool-calling errors.

Is Structured Outputs always slower or more expensive than JSON mode?

Not necessarily. Schema transmission and first-use grammar compilation can add overhead, while fewer repair calls can reduce total latency and cost. Compare cold and warm schemas, accepted-result latency, retry rate, and total tokens on your own workload; the LLM API benchmarking guide provides a repeatable measurement framework.

Can one JSON Schema be reused unchanged across providers?

Usually not. Providers support different JSON Schema subsets, request fields, model combinations, and complexity limits. Keep one canonical application contract, but compile and test a provider-specific adapter for each API.

Primary sources

The following first-party documentation was rechecked on 2026-07-24: