Definition
Structured output — often called JSON mode when the target format is JSON — is the practice of forcing a language model to return responses that conform to a precise, machine-parseable schema rather than free-form prose. This matters whenever an LLM's answer feeds into other software: a downstream program can only act on the output if every field, type, and bracket is exactly where it expects them. Structured output turns an unreliable text generator into a dependable component of a larger pipeline.
How it is enforced
The most robust approach is constrained decoding. A schema (such as a JSON Schema) is compiled into a formal grammar, and during generation a logit processor masks out any token that would violate that grammar — setting its probability to zero before sampling. The model's weights are never touched; only its choice of next token is restricted, which guarantees the result parses while leaving the model free to fill in the content. Related mechanisms include function/tool calling, where the model returns arguments matching a declared schema.
Why self-hosters use it
Running your own model for real automation — classifying error codes, extracting miner specs, populating a database — requires output you can trust to parse on the first try. Open libraries such as Outlines, Guidance, and XGrammar provide grammar-constrained decoding for locally-hosted models.
Structured output is frequently combined with format-enforcing guardrails and is a natural fit for tasks handled by an AI agent that must call tools with well-formed arguments.
In Simple Terms
Structured output — often called JSON mode when the target format is JSON — is the practice of forcing a language model to return responses…