Part 1 of 6 · 1 min read
Overview
In this module you will
- Add examples to a prompt, choose them well, and check an example set for mistakes.
- Ask for step-by-step reasoning in a way your code can use: extract the answer and verify the steps.
- Use voting over several samples, and explain when the maths behind it holds.
- Decide when examples or reasoning are worth their extra tokens, and when they are not.
Agents classify, plan and decide at every step, and a wrong small decision early can ruin a long run. Examples fix format and edge-case mistakes cheaply. Reasoning steps help on multi-step problems and give you something to check. Both cost tokens (Level 1, module 6), so you should know what you are buying.
About the examples
The two models here are imitations. fakemodel.py classifies with keyword rules and, when no keyword matches, copies the label of the most similar example. NoisySolver returns the right answer with a set probability and otherwise makes a seeded arithmetic slip. They exist to make the mechanisms visible and testable. They say nothing about how well a real model uses examples or reasoning, which you must measure on your own task.