Ship a change only when the comparison earns it
one move: compare under a guardrail
Open the code pane immediately, then use the Spec tab, optional design question, tests, and artifact when you want them.
Finish each move before carrying the artifact forward.
Online experiments need a fixed assignment, a primary metric, a minimum practical lift, and a guardrail metric before promotion.
Deliverable: State the invariant in one sentence.Summarize binary outcomes for control and treatment and return rates, lift, and ship/hold using a minimum lift threshold.
Deliverable: Type the smallest deterministic implementation.The summary is deterministic, handles empty groups, and refuses to ship when the guardrail or lift is insufficient.
Deliverable: Record a normal case and a boundary case.Change the threshold, add a treatment regression, and compare a statistically noisy win with a safe hold.
Deliverable: Name the failure signal and the guard you would add.An experiment readout with assignment policy, primary metric, guardrail, and rollout decision.
Deliverable: Save the contract, evidence, and handoff note.The summary is deterministic, handles empty groups, and refuses to ship when the guardrail or lift is insufficient.
Change the threshold, add a treatment regression, and compare a statistically noisy win with a safe hold.
An experiment readout with assignment policy, primary metric, guardrail, and rollout decision.
Deriva-authored extension beyond the supplied PDF. The five moves are Deriva’s implementation contract for this project.