Part 1 of 8 · 1 min read
Overview
In this module you will
- Name the stages that turn raw text into a chat assistant, and say what each one changes.
- Explain why a base model continues your text instead of answering it.
- Describe a chat template and a loss mask, and why user text must never be able to forge special tokens.
- Explain preference tuning (RLHF and DPO) and compute a DPO loss by hand.
- Say what a reasoning model is and what it costs you.
The model you call is not "a model". It is the output of several training stages, and each leaves a fingerprint you will meet in practice: the chat format you must follow, the refusals you must work around, the habit of agreeing with you, and the extra tokens a reasoning model spends.
If you know where a behaviour comes from, you know whether a prompt can fix it, whether you need a different model, or whether you need your own fine-tuning.