---
title: "Retrieval in practice — Context Field Notes"
description: "A checklist for retrieval that holds up under real traffic."
canonical: "https://press.getalchemystai.com/books/context-field-notes/retrieval-in-practice"
last_updated: "2018-10-20"
book: "Context Field Notes"
---

# Retrieval in practice

Retrieval demos are easy. Retrieval that survives real traffic is mostly bookkeeping. This chapter is a working checklist rather than a theory.

## Before you index

Decide what a document is. A forty-page PDF is not one document; it is a set of sections with different owners and update cycles. Chunk along the structure the authors already gave you: headings, list items, table rows. Fixed-size windows are a fallback, not a strategy.

- [x] Every chunk keeps a link to its source and section
- [x] Every chunk records when its source last changed
- [ ] Deleted sources remove their chunks within a day
- [ ] Access rules are checked at query time, not only at index time

## Measuring what matters

Offline metrics such as recall@k are useful for comparing two indexes. They say little about whether users got good answers. Pair them with a small, hand-labelled set of real questions and review it every time you change chunking, embeddings or ranking.

| Signal | What it tells you | Watch out for |
| --- | --- | --- |
| Recall@k | The right chunk was retrieved | Ignores whether it was used |
| Citation rate | Answers point at sources | Can be gamed by citing everything |
| Correction rate | Users had to fix the answer | Lags behind changes |

## A small, honest pipeline

```text
query → rewrite (optional) → retrieve 30 → rerank to 6 → answer with citations
```

Start there. Add steps only when a labelled failure proves you need them. Each step you add is another place for context to go wrong, and another thing to explain when it does.

---

- Previous: [Layers of memory](https://press.getalchemystai.com/books/context-field-notes/memory-layers.md)
- Contents: [Context Field Notes](https://press.getalchemystai.com/books/context-field-notes.md)
- Next: [Operating notes](https://press.getalchemystai.com/books/context-field-notes/operating-notes.md)
