pytest in practice

A test is a function whose name starts with test_. It passes if it finishes without an error. Here is the small function from the example project, and a test for it:

# summarizer.py
def truncate_words(text: str, max_words: int = 200) -> str:
    words = text.split()
    return " ".join(words[:max_words])

# tests/test_summarizer.py
from summarizer import truncate_words

def test_truncate_words_keeps_the_first_words():
    assert truncate_words("one two three", 2) == "one two"

Run pytest and it finds every test_ function in the tests folder:

$ pytest -q
.............                                                            [100%]
13 passed in 0.05s

Each dot is one passing test. Useful options: pytest tests/test_summarizer.py -k truncate runs only tests with "truncate" in their name, pytest -x stops at the first failure, and pytest -vv shows full details.

What a failure looks like

A test is useful when it fails for the right reason. Suppose someone "improves" the function and introduces a bug: words[1:max_words] instead of words[:max_words]. Running the tests prints:

$ pytest -q tests/test_summarizer.py
...
>       assert truncate_words(text, limit) == expected
E       AssertionError: assert 'two' == 'one two'
E
E         - one two
E         + two
...
FAILED tests/test_summarizer.py::test_truncate_words[one two three-2-one two]
FAILED tests/test_summarizer.py::test_truncate_words[one two-5-one two]
FAILED tests/test_summarizer.py::test_truncate_words[  spaced   out  -5-spaced out]
FAILED tests/test_summarizer.py::test_summarise_sends_text_and_returns_summary
FAILED tests/test_summarizer.py::test_summarise_surfaces_upstream_errors - Va...
5 failed, 2 passed in 0.04s

The lines starting with E say what was expected and what was found: 'two' instead of 'one two'. The first word went missing. That is the whole value of a test: it points at the bug within seconds.

Shape each test as arrange, act, assert: set up the input, run the code, check one clear thing. Give it a name that reads like a sentence about behaviour, so that a failing test tells you what broke without opening the file.

Parametrise: one test, many inputs

To run the same test with several inputs, list them:

import pytest

@pytest.mark.parametrize(
    "text, limit, expected",
    [
        ("one two three", 2, "one two"),
        ("one two", 5, "one two"),
        ("  spaced   out  ", 5, "spaced out"),
        ("", 5, ""),
    ],
)
def test_truncate_words(text, limit, expected):
    assert truncate_words(text, limit) == expected

Each row is reported as its own test, which is why the failure list above names the exact input that broke. Edge cases (empty, one item, too long, odd spacing) belong in rows like these.

Expect an error

Sometimes the correct behaviour is to fail. pytest.raises checks that the error happens:

def test_empty_text_is_rejected():
    with pytest.raises(ValueError):
        summarise(client, "   ")

Fixtures: shared set-up

Fixtures usually live in a file called tests/conftest.py, so every test can use them. Some come built in with pytest: tmp_path (a fresh temporary folder), monkeypatch (temporarily change an environment variable or attribute) and capsys (capture printed output). For example, a test that reads settings from environment variables (you will build load_settings in module 6):

def test_reads_model_name_from_environment(monkeypatch):
    monkeypatch.setenv("APP_LLM_API_KEY", "sk-test-1234567890")
    monkeypatch.setenv("APP_LLM_NAME", "big-2")
    assert load_settings().llm_name == "big-2"

monkeypatch changes the variables only for this test and puts them back afterwards, so tests cannot affect each other.