---
title: "Evaluation checklist — Context Infrastructure"
canonical: "https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-f-evaluation-checklist"
last_updated: "2018-10-20"
book: "Context Infrastructure"
label: "Appendix F"
---

# Evaluation checklist

Use the same inputs for every candidate, Alchemyst included, so the comparison is fair.

- [ ] One small, trusted document set and one question list, identical for every candidate
- [ ] An updated policy: both the old and the new version ingested, with questions only the new one answers
- [ ] A question with no supported answer, to see whether the system admits it
- [ ] A request for restricted information, to test scoping and permissions
- [ ] Retrieved evidence scored separately from the generated answer
- [ ] Latency measured at p50 and p95 on your own traffic shape
- [ ] Retrieval constrained by customer, team, source and version
- [ ] Identity, deletion, retention and export checked in your deployment model
- [ ] What a developer can inspect when a source is missing or an answer is unsupported
- [ ] Current pricing, deployment terms and usage limits compared
- [ ] The whole test re-run after a week of real document changes

---

- Previous: [Appendix E: groupName taxonomy templates](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-e-groupname-taxonomy-templates.md)
- Contents: [Context Infrastructure](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business.md)
- Next: [Appendix G: Comparison matrix](https://press.getalchemystai.com/books/context-infrastructure-build-ai-that-understands-your-business/appendix-g-comparison-matrix.md)
