EngineeringIntermediate
Structured JSON Outputs & Schema Enforcement in AI APIs
Direct Answer & Overview
Structured output is an inference mode that guarantees the model’s generated tokens adhere strictly to a developer-specified JSON schema using context-free grammar (CFG) decoding constraints.
1.Eliminating Malformed JSON in Production
Relying on prompt engineering alone ('respond only in JSON') frequently fails: models add conversational prefixes ('Sure, here is your JSON:'), omit closing braces, or invent fields. Structured Outputs constrain token sampling at the logits level, mathematically preventing the model from producing invalid tokens.
Enforcing Strict JSON Schema in TypeScripttypescript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.apihundred.com/v1",
apiKey: process.env.API100_API_KEY,
});
const completion = await client.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Extract customer: Alice, [email protected], age 32" }],
response_format: {
type: "json_schema",
json_schema: {
name: "CustomerRecord",
strict: true,
schema: {
type: "object",
properties: {
name: { type: "string" },
email: { type: "string" },
age: { type: "integer" }
},
required: ["name", "email", "age"],
additionalProperties: false
}
}
}
});
console.log(JSON.parse(completion.choices[0].message.content!));Frequently Asked Questions
How does strict JSON schema enforcement work in LLMs?
Inference engines mask invalid tokens at each decoding step using context-free grammar masks, ensuring the model can only generate tokens that satisfy the active JSON schema.
A100
API100 Engineering Team
Infrastructure & Latency Research