Part 1 of 6 · 1 min read
Overview
In this module you will
- Say when a local model is a good idea and when an API is better.
- Estimate whether a model fits in your memory, and what quantisation and context length do to that.
- Write a client for a local, chat-style HTTP endpoint with timeouts, streaming and readiness checks.
- Compute a break-even between owning hardware and paying per request.
Local models are useful for privacy (data never leaves your machine), for offline work, for cheap high-volume tasks, and for learning how serving works. They are usually weaker than the best hosted models, and they move operational work onto you. Because a local runtime usually exposes an HTTP endpoint shaped much like a hosted API, the provider layer from module 7 can treat it as one more adapter.
About the examples
No model was installed or run. server.py is an imitation server that echoes your last message over real HTTP on 127.0.0.1, so you can practise the client side (timeouts, streaming, readiness) with nothing to download. Memory figures are planning formulas for an illustrative model shape, not measurements of any real model or runtime. Always measure on your own hardware.