How tuning works

How contrastive training works

batch:  (q1, d1) (q2, d2) (q3, d3) (q4, d4)
for q1:  d1 is the answer;  d2, d3, d4 are negatives
loss  =  -log  exp(s(q1,d1)) / (exp(s(q1,d1)) + exp(s(q1,d2)) + exp(s(q1,d3)) + exp(s(q1,d4)))

Choices that matter:

• Batch size. More pages in the batch means harder choices. In the example, a batch of 2 reaches 26% on new questions, 4 gives 38%, 8 gives 48%, and 16 to 32 gives about 50 to 53%; past that nothing improved. Real systems often need much larger batches, which is why gradient caching exists.
• Negatives. Random pages from other pairs are easy negatives. Hard negatives are pages that look right but are not: the same topic, the wrong detail. They teach finer distinctions but cost effort to find.
• False negatives. A "hard negative" mined by search may actually answer the question (a duplicate page, an equivalent answer). Training on it teaches the model something wrong.
• Data pairs. Positive pairs are usually easy to get; good negatives are the hard part.

In the example, adding the best-scoring wrong page (under the untuned model) as an extra negative did not help: new questions 51% against 57% without it, and unseen topics 3% against 8%. Two reasons to read that cautiously: five runs of about 30 test questions each is noisy, and my toy pages differ from each other in only two words, so the in-batch negatives are already close to hard. In real corpora, hard negatives are reported to help; measure on yours.

Where do you put the tuning?

Real models differ from this toy, but the operational question is the same:

• Question side only (an adapter, a small extra layer on the query vector): the documents are embedded as before, so nothing in the index changes. Cheap to deploy and to roll back.
• Both sides (fine-tuning the whole model, or adapters on both): usually more accurate (57% against 50% in the example, 71% with more epochs) but every stored vector is now from a different model and must be re-embedded in a second index, then switched together (module 9: a half-migrated index is a half-broken index).

Either way, stamp the index with the model version, and keep the previous one until the new one passes your golden questions.

What happened in the example

Four findings, all of which show up in real projects in some form:

1. It works where it was taught. 19% to 50% and above on new questions about problems it saw. (The validation column is a smaller, noisier slice and should not be compared row to row.)
2. Memorising is fast, generalising is slow. Accuracy on the training questions is 98% after 10 epochs; new questions are still at 28% then and keep creeping to about 50% by epoch 150.
3. General ability drops at once. The "ordinary language" questions fell from 82% to 48% within five epochs, before the domain scores had improved. Nobody asked the model to forget; changing a shared layer changed everything.
4. It does not transfer to topics it never saw. On the four problems that were never in training, tuning moved accuracy from 4% to 4 to 8%. The model learned the word pairs in the training data, not the idea of jargon.