Structured JSON Outputs & Schema Enforcement in AI APIs

Overview #

Structured output is an inference mode that guarantees the model’s generated tokens adhere strictly to a developer-specified JSON schema using context-free grammar (CFG) decoding constraints.

Eliminating Malformed JSON in Production #

Relying on prompt engineering alone ('respond only in JSON') frequently fails: models add conversational prefixes ('Sure, here is your JSON:'), omit closing braces, or invent fields. Structured Outputs constrain token sampling at the logits level, mathematically preventing the model from producing invalid tokens.

Code Example: Enforcing Strict JSON Schema in TypeScript #

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.apihundred.com/v1",
  apiKey: process.env.API100_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Extract customer: Alice, [email protected], age 32" }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "CustomerRecord",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          email: { type: "string" },
          age: { type: "integer" }
        },
        required: ["name", "email", "age"],
        additionalProperties: false
      }
    }
  }
});

console.log(JSON.parse(completion.choices[0].message.content!));

Frequently Asked Questions #

Q: How does strict JSON schema enforcement work in LLMs?

Inference engines mask invalid tokens at each decoding step using context-free grammar masks, ensuring the model can only generate tokens that satisfy the active JSON schema.