Know whether a bad answer came from search or generation
one move: separate retrieval from generation
Open the code pane immediately, then use the Spec tab, optional design question, tests, and artifact when you want them.
Finish each move before carrying the artifact forward.
Grounded generation is a pipeline: recall and rank evidence first, then judge whether the answer cites it.
Deliverable: State the invariant in one sentence.Compute recall@k and reciprocal rank from relevant document IDs and a retrieved list.
Deliverable: Type the smallest deterministic implementation.The metrics must ignore duplicate hits, return zero for no relevant evidence, and preserve the first relevant rank.
Deliverable: Record a normal case and a boundary case.Remove the top evidence, add a distractor, and compare retrieval failure with a generation failure.
Deliverable: Name the failure signal and the guard you would add.A retrieval benchmark with recall, MRR, failure buckets, and a groundedness handoff.
Deliverable: Save the contract, evidence, and handoff note.The metrics must ignore duplicate hits, return zero for no relevant evidence, and preserve the first relevant rank.
Remove the top evidence, add a distractor, and compare retrieval failure with a generation failure.
A retrieval benchmark with recall, MRR, failure buckets, and a groundedness handoff.
Deriva-authored extension beyond the supplied PDF. The five moves are Deriva’s implementation contract for this project.