---
title: "Benchmark data — Context Infrastructure"
canonical: "https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-c-benchmark-data"
last_updated: "2018-10-20"
book: "Context Infrastructure"
label: "Appendix C"
---

# Benchmark data

Per-category memory F1 from the December 2025 benchmarks, sorted by Alchemyst's score. Higher is better; the best score in each row is in bold. Overall F1, price and latency are charted in Chapter 19.

| Category (cases) | Alchemyst | Hindsight GPT OSS 120B | Supermemory | Zep |
| --- | ---: | ---: | ---: | ---: |
| Single Session Assistant (55) | 0.964 | **0.982** | 0.964 | 0.804 |
| Single Session User (68) | 0.956 | **1.000** | 0.971 | 0.929 |
| Temporal Reasoning (131) | 0.756 | **0.857** | 0.767 | 0.624 |
| Multi Session (133) | 0.729 | **0.812** | 0.714 | 0.579 |
| Single Session Preference (30) | 0.600 | **0.867** | 0.700 | 0.567 |
| Knowledge Update (75) | 0.560 | **0.923** | 0.884 | 0.833 |

**Methodology.** The evaluations cover single-session preference, user and assistant recall, knowledge updates, temporal reasoning and multi-session memory, with 30 to 133 cases per category. Latency is the P50 of a retrieval call in the test harness, and prices are in USD as charged in that harness. Alchemyst ranks second on Multi Session and ties for second on Single Session Assistant, and is last only on Knowledge Update. The eval definitions are available from the Alchemyst team on request; reproduce the setup on your own workload before relying on any of these numbers.

---

- Previous: [Appendix B: Platform limits and error codes](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-b-platform-limits-and-error-codes.md)
- Contents: [Context Infrastructure](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business.md)
- Next: [Appendix D: Code recipes](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-d-code-recipes.md)
