Questions and evidence labels

Inspect original sources first

ragvi bm25 ./manual.txt --no-input --format html --format json

Read source IDs and the corpus hash from the source report. snapshot.json retains exact extracted bodies. Offsets are zero-based Unicode character positions with an exclusive end, relative to the original extracted source unit.

Reviewed sidecar schema

{
  "schema_version": 1,
  "corpus_hash": "COPY_FROM_SOURCE_REPORT",
  "evidence": [{"evidence_id":"RESET","spans":[
    {"source_id":"COPY_ORIGINAL_SOURCE_ID","start":0,"end":13,"text":"eight seconds"}
  ]}],
  "questions": [{"question_id":"Q1","question":"How long is reset?",
    "split":"dev","answerable":true,"required_evidence_ids":["RESET"]}]
}

The example offsets apply only if the original body begins with eight seconds. Copy actual source IDs/hashes and review each quote. One evidence item can require multiple spans across sources. A question’s complete required set must contain every condition needed for an answer. Tags and reference answers are optional evaluation data.

ragvi bm25 ./manual.txt --reviewed ./reviewed.json

Evidence credit

Full union coverage of every required span earns evidence credit, including coverage across multiple chunks. Partial text and duplicate overlap cannot inflate credit. Search prefixes never earn span credit. Changing chunk size retains the same original corpus identity.

Answerability and review quality

Dev and heldout are separate splits. Unanswerable questions have N/A retrieval metrics. Review completeness, corpus version and question wording; do not leak expected evidence into supplied variants or prefixes for heldout questions.

• Runnable reviewed synthetic example

• Reviewed span rules and complete import schemas