Agentic AI · Module 4 of 41 · 9 concepts · about 13 min

Structured Outputs

How to get data, not prose, out of a model: JSON, schemas, typed models with Pydantic, validation, what to do when parsing fails, and why production agents depend on it.

At work: The join between a model and the rest of your software. Every router, planner, extractor and judge in an agent system returns structured output that code then acts on.

Why agents need structured output

What it is

A structured output is a model reply in a fixed, machine-readable shape, usually JSON: named fields with typed values. Agents need it wherever code reads the reply:

tool requests: a tool name and its inputs;
routing: which category a request belongs to;
plans: a list of steps with dependencies;
extraction: fields pulled out of a document;
verdicts: a judge's score and its reason.

Why it matters

Code cannot branch on a paragraph.

'I think this is probably a billing question' needs fragile text matching. {"category": "billing"} needs one line.
A fixed shape can be checked, counted, stored and compared between versions.
One malformed reply in a chain of ten steps breaks the whole task, so the shape has to hold every time, not most of the time.

In the real world: A ticket router that searched replies for the word 'billing' misrouted every ticket that said 'this is not a billing issue'; a one-field JSON output fixed it.

Parsing free text with regular expressions because 'the model usually writes it the same way'. It will not, the first time the input is unusual, and the failure will be silent.

Practise it: Design an AI Customer Support Agent

JSON and schemas

What it is

Two terms:

JSON is a plain-text format for data: objects with named fields, lists, strings, numbers, true and false.
A JSON Schema is a description of what a valid piece of JSON looks like: which fields, which types, which are required, which values are allowed.

You give the model the schema; the reply must match it.

Why it matters

The schema is a contract both sides can rely on.

The model knows exactly what to produce.
Your code knows exactly what it will receive, and can reject anything else.
Field descriptions inside the schema act as instructions, in the place the model looks when filling that field.

In the real world: An extraction pipeline's downstream code crashed weekly on a missing currency field until the field was made required with "unknown" as an allowed value.

route_schema = {
    "type": "object",
    "properties": {
        "category": {
            "type": "string",
            "enum": ["billing", "delivery", "account", "other"],
            "description": "The team that should handle the request",
        },
        "urgent": {"type": "boolean", "description": "True if food safety or a missing order"},
        "summary": {"type": "string", "description": "One sentence, no personal data"},
    },
    "required": ["category", "urgent", "summary"],
    "additionalProperties": False,
}

Leaving fields optional 'to be flexible'. Every optional field is a branch your code must handle and the model may skip. Make fields required, and use an explicit value such as "unknown" when there is nothing to say.

Three ways to get structured output

What it is

From weakest guarantee to strongest:

Ask in the prompt: 'reply with JSON only'. Works most of the time; nothing enforces it.
Use a tool definition: describe the shape as a tool's inputs. The model fills it in as a tool request.
Constrained output: you pass a schema to the provider, which restricts the model so that only text matching the schema can be produced. Often called structured outputs or strict mode.

Why it matters

The guarantee decides how much defensive code you need.

With prompt-only JSON you must handle stray text, missing brackets and wrong types.
With constrained output the reply always parses and always has the right fields, so those failures disappear.
What no method can guarantee is that the values are correct. A valid order_id may still be the wrong order.

In the real world: A team that moved from 'please return JSON' to the provider's schema-constrained mode deleted two hundred lines of repair code and its parse-failure alert.

Visual example

Methods

Prefer the bottom row wherever the provider supports it.

Three ways to get data out of a model

method	always parses	always matches schema	use when
Ask in the prompt	No	No	Prototypes, or no other option
Tool definition	Usually	Usually	The output is naturally an action
Constrained output	Yes	Yes	Anything code will branch on

Treating 'valid JSON' as 'correct answer'. Structure is guaranteed; truth is not. Still check values against real data: does that order exist, does it belong to this user?

Typed outputs with Pydantic

What it is

Pydantic is a Python library for describing data as a class with typed fields. For structured output it does three jobs:

you write the shape once, as a class;
it generates the JSON Schema to send to the model;
it validates the reply and gives you a real Python object, or a precise error.

Why it matters

A typed output means the rest of your program works with route.category, not with data["category"] and a hope.

Your editor and type checker catch misspelt fields.
Rules that a schema cannot express, such as 'amount must not exceed the order total', go in a validator next to the field.
The same class documents the contract for people.

In the real world: A claims system defines each extraction step as a small Pydantic class; a new field is one line, and a wrong type fails loudly in tests instead of quietly in production.

from typing import Literal

import anthropic
from pydantic import BaseModel, Field

class Route(BaseModel):
    category: Literal["billing", "delivery", "account", "other"]
    urgent: bool = Field(description="True if food safety or a missing order")
    summary: str = Field(description="One sentence, no personal data")

client = anthropic.Anthropic()
reply = client.messages.parse(
    model="claude-opus-5-5",
    max_tokens=16000,
    messages=[{"role": "user", "content": "My milk arrived warm and smells off."}],
    output_format=Route,          # the SDK sends the schema and validates the reply
)

route: Route = reply.parsed_output
if route.urgent:
    escalate(route)               # ordinary code branching on a typed field

Asking for a deeply nested model with thirty fields in one call. Accuracy falls as the shape grows. Split it: one call per logical part, each with a small model.

Validation: structure and meaning

What it is

Validation is checking a reply before you act on it. There are two layers:

Structural: is it valid JSON, are the fields present, are the types right? The schema covers this.
Semantic: do the values make sense? Does the order exist? Is the amount within the limit? Is the date in the future? Only your code can check this, against real data.

Why it matters

Structural validation catches a malformed reply. Semantic validation catches a well-formed mistake, which is the more dangerous kind because nothing looks wrong.

Every value that will drive an action should be checked against the system of record.
A failed check is not a crash: it becomes feedback to the model.

In the real world: A refund agent's schema accepted any integer amount; a semantic check against the order total caught a ₹4,990 refund on a ₹499 order before it was paid.

from pydantic import BaseModel, model_validator

class RefundRequest(BaseModel):
    order_id: int
    amount_rupees: int

    @model_validator(mode="after")
    def check_against_real_order(self):
        order = ORDERS.get(self.order_id)
        if order is None:
            raise ValueError(f"Order {self.order_id} does not exist")
        if not 0 < self.amount_rupees <= order["total"]:
            raise ValueError(f"Amount must be between 1 and {order['total']}")
        return self

Stopping at 'it parsed'. A reply can match the schema perfectly and name an order that does not exist. Structural checks are necessary and never sufficient.

Parsing failures and retries

What it is

When a reply fails validation, the standard recovery is a repair loop:

catch the error;
send the model its own reply plus the exact error message;
ask it to try again;
stop after a small number of attempts, usually two, and take a safe fallback.

Why it matters

A model corrects itself well when told precisely what was wrong.

'amount_rupees: must be at most 499' is fixed on the next try.
A bare 'try again' usually reproduces the same mistake.
The attempt limit matters: without it a stubborn failure becomes an endless, costly loop.

In the real world: A router that fell back to the 'other' queue after two failed attempts kept working through a model outage that sent every strict caller into a crash loop.

Visual example

Repair loop

The error message is the lesson. The attempt limit is the brake.

What happens when a reply fails validation

Model repliesamount_rupees: 4990
ValidateFails: above the order total
Feed backReply + exact error to the model
Retry or fall backTwo tries, then a person

from pydantic import ValidationError

def structured(prompt: str, model_cls, attempts: int = 3):
    """Ask for model_cls; feed validation errors back; give up safely."""
    messages = [{"role": "user", "content": prompt}]
    for _ in range(attempts):
        text = llm(messages, schema=model_cls.model_json_schema())
        try:
            return model_cls.model_validate_json(text)
        except ValidationError as error:
            messages += [
                {"role": "assistant", "content": text},
                {"role": "user", "content": f"That was invalid:\n{error}\nReturn corrected JSON only."},
            ]
    return None          # caller falls back: default route, or hand over to a person

Retrying forever, or retrying with the identical request. Each retry must carry the error, and the loop must end in a defined fallback, not an exception at 3 a.m.

Structured output in production

What it is

Habits that keep structured output reliable at scale:

Small schemas: one decision per call.
Enums over free text for anything code branches on.
A reason field before the answer field, so the model works it out before committing.
Version your schemas: a changed field is a breaking change for stored data and for other services.
Count failures: parse failures and repair attempts are metrics worth an alert.

Why it matters

In a pipeline of many steps, small failure rates multiply.

A step that fails one time in fifty, used ten times per task, breaks about one task in five.
Constrained output plus semantic checks brings each step close to never failing structurally.
Stored outputs outlive the code: a renamed field breaks every report that reads last month's data.

In the real world: An evaluation team reorders a judge's schema from {score, reason} to {reason, score} and its agreement with human graders rises by several points with no other change.

Putting the answer field first and the reasoning after. The model commits to an answer, then writes a justification for it. Reasoning first, verdict last.

Interview questions: structured outputs

What it is

The questions interviewers ask about structured outputs, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Why it matters

Reading an idea and explaining it are different skills.

Each answer below is the two or three sentences an interviewer is listening for.
The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

In the real world: A take-home task returns a router with regex parsing; the reviewer's first note is 'what happens on a reply you did not expect?', and the candidates who used a schema have nothing to fix.

Q: Why do production agents need structured outputs?

Because code acts on the reply. Routing, tool inputs, plans and verdicts must be data with a known shape, so they can be validated, branched on and stored. Free text needs fragile parsing and fails silently.

Q: How do you get reliable JSON from a model?

Prefer the provider's schema-constrained mode, which guarantees the shape. Otherwise use a tool definition. Prompt-only JSON is the weakest and needs repair code. In all cases validate, and retry with the error message.

Q: What does a schema not guarantee?

That the values are true. A valid order id may not exist or may belong to someone else. Semantic validation against real data is still needed.

Q: What is Pydantic used for here?

Declaring the output shape as a typed class, generating the JSON Schema from it, and validating the reply into a real object. Custom validators hold rules a schema cannot express.

Q: A reply fails to parse. What happens?

Catch it, send the reply and the exact validation error back, retry at most twice, then fall back to a safe default or a person. Count the failure as a metric.

Q: How do you design a schema models fill in well?

Keep it small and flat, use enums, describe every field, make fields required with an explicit 'unknown', and put reasoning before the final answer field.

Exercise: build a validated ticket router

What it is

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

Why it matters

An idea you have only read fades in a week.

Building it once, even in fifty lines, is what makes it yours.
Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.

In the real world: A support platform's router is this function plus a queue; its 'needed a repair' counter is the team's early warning that a model update changed behaviour.

Your task. Write a function that turns a customer message into a validated Route object, with a repair loop and a safe fallback.

Steps

Define a Pydantic Route with reason (string), category (four allowed values) and urgent (boolean). Put reason first.
Write route(message) that asks a model for a Route, using your provider's structured output feature if it has one.
Add a semantic rule: if the message mentions a safety word such as 'smell', 'mould' or 'allergic', urgent must be true. Enforce it with a validator.
Wrap the call in a repair loop that feeds back the validation error, tries at most three times, and otherwise returns Route(category='other', urgent=True, ...).
Run it on twenty messages you write yourself, including sarcasm, two requests in one, and a message in another language.

You are done when

The function never raises: every input yields a Route.
You can print how many inputs needed a repair attempt.
The fallback is deliberately the safe choice, and you can say why.

Stretch. Simulate a broken model that returns not json and confirm the fallback path is taken after exactly three attempts.