Try providers in order, but only fall back on errors another provider could fix.
The router and its fallbacks
The app talks to the router, which tries adapters in order. Each adapter hides one provider.
router=Router([AdapterA(...),AdapterB(...)])resp=router.complete(request)# tries A, and moves to B only if that can help
The important design decision is which errors cause a fallback:
Server error, rate limit or network failure: fall back. Another provider may be healthy.
Auth failure: fall back. This key is broken, but another provider's key may work (and you should alert someone).
Bad request: do not fall back. Your request is wrong. Sending the same wrong request elsewhere wastes money and hides a bug. The tests check that the second provider is never called.
Other: do not fall back. An error nobody has classified should be looked at, not silently routed around.
If every provider fails, the router raises the last error, so the caller sees a real cause.
A fallback is not free
A different provider gives different answers, formats and token counts. Prompts tuned for one may behave differently on the other (Level 3 covers writing prompts, Level 9 testing them). Record which provider answered in the response and the cost ledger (module 6), so you can see how often you fell back and what it cost. Combine the router with module 5's retry and breaker: retry the same provider for brief blips, and fall back for longer failures.
Testing the layer
You never need a real key to test an adapter. Pass in a fake backend that records the payload and returns a canned reply, then assert two things: the payload is what the provider needs, and the neutral response is correct. The example does exactly that, plus a test that a new third provider needs only one new class.
The tests also cover the failure paths: an unknown finish reason, a dropped connection falling back to the second provider, and three kinds of request the neutral type must refuse.
What you will use daily at work
Practice
What it looks like
A gateway or library
Many teams use an existing library or gateway that already offers one interface over many providers, and only write adapters for what it lacks.
Contract tests
The same test suite runs against every adapter, so a provider change breaks a test, not production.
Provider and model in every log line
Each request records who answered, so fallbacks and cost per provider are visible.
Prompt tests per model
Switching the model re-runs the prompt tests (Level 9).
Draft: confirm each provider's current request and error formats, and the features of any gateway library, before relying on them.
Interview angle
"How would you avoid vendor lock-in?" A good answer is not "use a framework". It is: keep a small neutral interface you own, one adapter per provider, a single error type, tests that pin the translation, and honest limits (features unique to one provider do not fit a neutral interface, so decide deliberately when to use them). Mention that swapping the model also needs the prompts re-tested. A follow-up: "what must the adapter do with a stop reason it does not know?" (Report it as unknown, never as success.)