Metrics reference
Understand original-span sufficiency, coverage, rank, precision and N/A states.
v0.2.1Local CLI · Offline reports
On this page
Evidence metrics
| Metric | Meaning |
|---|---|
| Sufficiency | Every required original evidence item is fully covered by the final span union. |
| Coverage | Fraction of complete required evidence items recovered. |
| First relevant rank / reciprocal rank | First ranked prefix that completes any required evidence item; its reciprocal. |
| Overlap precision | Relevant returned hits divided by configured k. |
| Paired delta | Within-question intervention minus baseline, keeping gains and regressions. |
Denominators and limits
Answerable questions determine retrieval-quality rates, separately for dev/heldout/slices. Unanswerable questions have N/A metrics. Unreviewed runs cannot establish relevance. nDCG is unavailable without reviewed relevance gains. ID-only imports are limited proxies.
Body truncation or extractive compression can remove evidence; evaluation uses actual final original spans. Prefixes never establish evidence. Low raw ranking scores are not a correctness probability.
Generated answers are separate
The optional workspace generate command can produce answers for inspection using your provider. Neither answers, citations nor abstention are judged. Portable retrieval commands do not generate answers.
Was something unclear? Send documentation feedback.