Part 2 of 7 · 4 min read
Describe the shape once
Five ways to get structure from a model, then the one habit that makes the rest easy: say what you want once, as a small Python class.
Five ways to get structure
There is a ladder of methods. Each one costs more effort and gives more reliability, and you can combine them.
| Method | What it is | Cost | How reliable |
|---|---|---|---|
| Prompting | Describe the format in the instructions and show the schema | Free | Good, not guaranteed |
| Post-processing | Code that finds the JSON in the reply and tidies it | Free | Fixes harmless slips |
| Retry with feedback | Send the error back to the model and ask again | Extra calls | Fixes most content mistakes |
| Constrained decoding | The provider restricts which tokens the model may produce, so output must match your schema (you met sampling in Level 1, module 4) | A provider feature | Guarantees the shape |
| Fine-tuning | Train a model on your format (Level 13) | High effort | Strong for fixed formats |
This module builds the first three, which work with any model. Provider features such as "structured output" or forced tool calls (module 4) are the fourth method. Use them when you have them, but still validate: a reply can have the right shape and the wrong meaning, and providers support only part of what a schema can express.
For the curious Why JSON mode is not enough
Many providers offer a JSON mode. It usually promises that the output is valid JSON, and nothing more. It does not promise your field names, your allowed values, or that the reply was not cut off before it finished. A schema-based mode goes further and restricts the output to your schema. Even then, the value of priority can be "high" when the right answer was "low". Valid and correct are different things, and the section on measuring comes back to this.
Describe the shape once
Before writing any checking code, you need a way to say what a valid reply looks like. In Python the most common tool for this is a library called Pydantic. You met it in Level 0, module 1, and here it is again from the beginning.
Pydantic
A Python library for describing data and checking it. You write a class that lists the fields and their types. Pydantic can then check any dictionary against that class, and it can also produce a schema from it.
Here is the form for a support ticket:
class Priority(str, Enum):
LOW = "low"
NORMAL = "normal"
HIGH = "high"
class Ticket(BaseModel):
category: str = Field(min_length=1)
priority: Priority
summary: str = Field(min_length=3, max_length=120)
needs_human: bool
Read it line by line:
class Ticket(BaseModel)says: "Ticketis a form, and it can check itself."Ticketis just the name we chose for "one support ticket".BaseModelis what Pydantic provides to make a class checkable.category: strmeans the fieldcategorymust be text.Field(min_length=1)adds a rule: at least one character.priority: Prioritymeans the value must be one of the three words listed inPriority. A class like that, which lists the only allowed values, is called an enum (short for enumeration).summaryis text between 3 and 120 characters.needs_human: boolmust be true or false.
Enum
A short, fixed list of allowed values. Use one whenever a field can only be one of a few words. It removes a whole kind of mistake: the model can no longer invent "urgent" or "HIGH!!".
Now the class does two jobs. First, it checks replies. Here it accepts a good reply:
>>> Ticket.model_validate({"category": "billing", "priority": "high",
... "summary": "Charged twice", "needs_human": True})
Ticket(category='billing', priority=<Priority.HIGH: 'high'>, summary='Charged twice', needs_human=True)
model_validate checks a dictionary and, if it is fine, gives you a real Ticket object whose fields have the right types. If it is not fine, it raises an error that lists every problem at once:
>>> Ticket.model_validate({"category": "x", "priority": "low", "summary": "ab"})
2 validation errors for Ticket
summary
String should have at least 3 characters [type=string_too_short, input_value='ab', input_type=str]
needs_human
Field required [type=missing, input_value={'category': 'x', ...}, input_type=dict]
Second, the class writes the instructions for the model. Ticket.model_json_schema() turns the class into a JSON Schema, a standard way to write a schema as JSON. Here is the part that describes priority and summary:
"priority": {"$ref": "#/$defs/Priority"},
"summary": {"type": "string", "minLength": 3, "maxLength": 120},
"needs_human": {"type": "boolean"}
and "Priority": {"enum": ["low", "normal", "high"], "type": "string"} further up. The function schema_prompt in the example code pastes this into the system prompt, together with the sentence "Reply with ONE JSON object and nothing else". Because the same class writes the prompt and does the check, the two can never disagree. That is the main idea of this part: describe the shape once, and use it twice.
Check yourself
- What does
priority: Priorityforbid thatpriority: strwould allow? - Which method checks a reply, and which method writes the instructions?
- Why is it useful that one class does both jobs?