Testing code that calls the network: Testing

How to test without the internet: swap the outside world for a fake and keep your own code real.

Never let a unit or integration test touch the real internet. It is slow, it needs a key, it costs money, and it fails when the network does. Instead, give your code a fake service to talk to.

Replace only the outside world. Your own code runs for real.

Mock, fake and stub

A stub returns canned answers. A fake is a simple working replacement (here, a fake model service). A mock also records how it was called, so that you can check it afterwards. People use the three words loosely. The upstream fixture below is a fake that also records the calls it receives.

With httpx, the simplest tool is httpx.MockTransport: a client that never leaves your process. It takes a small function that decides what to reply with. Here is the function under test, from the example project:

def summarise(client: httpx.Client, text: str) -> str:
    """Send text to a model service and return its summary."""
    text = truncate_words(text.strip())
    if not text:
        raise ValueError("text is empty")
    response = client.post("/v1/summarise", json={"text": text})
    response.raise_for_status()
    return response.json()["summary"]

and its test:

def test_summarise_sends_text_and_returns_summary(upstream):
    make, calls = upstream
    client = make(lambda req: httpx.Response(200, json={"summary": "short"}))

    assert summarise(client, "a long text") == "short"
    assert calls[0].url.path == "/v1/summarise"      # the request we sent
    assert b"a long text" in calls[0].content

Here upstream is a fixture from conftest.py that builds the fake. The lambda is the fake service: whatever request comes in, it answers with a 200 and a summary of "short". The test then checks two things: the function returned that summary, and the request it sent had the right path and body. When I ran it, the recorded request was a POST to /v1/summarise with the body {"text":"a long text"}.

This works because summarise receives the client as a parameter instead of creating its own. A test can hand in the fake. This habit is called dependency injection, and it is the main design choice that makes code testable. FastAPI does the same with Depends(...), and tests replace the real dependency with app.dependency_overrides.

Do not mock what you are trying to test

If you replace your own function with a mock and then check that the mock was called, the test proves nothing about your code. Replace only the boundary (network, clock, randomness, the model) and let everything else run for real.

For the curious Other tools for the same job

The libraries respx and pytest-httpx patch httpx for you so that your code does not need to accept a client, and responses does the same for the requests package. The idea is identical: stand in for the outside world and return the replies you choose. Injecting the client, as above, has the advantage that it works with any library.

Check yourself

Why must a test not call the real model API?
What is dependency injection, in one sentence?
Name one thing you should replace with a fake, and one thing you should not.