CSE Roadmap

Core tier · about 5 hours · 6 parts

Part 2 of 6 · 2 min read

Embeddings and exact search

Turning text into vectors, and why checking every vector stops scaling.

Embeddings

Embedding

A list of numbers (a vector) that represents a piece of text so that texts with similar meaning get vectors that are close together. The same model must embed both your documents and your queries.

Cosine similarity

The cosine of the angle between two vectors: a·b / (|a| |b|). It is 1 for the same direction, 0 for unrelated (perpendicular), and −1 for opposite. It ignores length, so only direction matters. If every vector is scaled to length 1, cosine similarity is just the dot product.

from embed import embed
from vectors import cosine

cosine(embed("automobile"), embed("car"))        # high: same concept
cosine(embed("automobile"), embed("password"))   # near 0: unrelated

Running demo.py (real output of the example):

'automobile money-back'      -> How to get a refund for a car rental (0.83) | Reimbursement policy for vehicle charges (0.83)
'lag when I login'           -> The app is slow and has high latency (0.55) | Reset your password and sign in again (0.38)
'holiday opening'            -> How to get a refund for a car rental (0.00) | Reimbursement policy for vehicle charges (0.00)

Three things to read in that output:

  1. The first query shares no words with the best documents, yet they win. That is the point of embeddings.
  2. The second query matched "slow" through the concept "lag", but "login" and "sign in" are not linked in the toy table, so the second hit is weaker.
  3. The third query matches nothing (score 0.00), yet the search still returns two documents. Nearest-neighbour search always returns k results, even when none are relevant. A threshold or a refusal rule is your job (module 6).

Real embedding models

The toy embedder ignores word order ("dog bites man" and "man bites dog" are identical, and a test proves it). Real models do not. They differ in dimension (often 384 to 3,072), maximum input length, language coverage, price and quality on your kind of text. Two rules that are always true:

  • Use the same model for documents and queries. Vectors from different models live in different spaces.
  • Changing the model means re-embedding everything (module 9).

Exact search and why it stops working

Exact search compares the query with every stored vector. For 3,000 vectors that is 3,000 comparisons per query, which is trivial. For 10 million vectors of 1,024 dimensions it is about 10 billion multiply-adds per query, and every query pays it. Exact search is also the ground truth you measure approximate indexes against, so keep it around.

index = Exact(vectors)
index.search(query_vector, k=10)    # always correct