Let the model see all previous characters
one move: let tokens attend
Open the code pane immediately, then use the Spec tab, optional design question, tests, and artifact when you want them.
Finish each move before carrying the artifact forward.
Queries ask what to find, keys describe what is present, and values carry information.
Deliverable: Write the mechanism in one sentence.Implement scaled dot-product causal self-attention for a tiny sequence.
Deliverable: Type the smallest runnable implementation.Future positions must receive zero attention and each row must normalize.
Deliverable: Record the expected output and one edge case.Remove the causal mask, change sequence length, and inspect attention weights.
Deliverable: Change one variable and explain the result.A causal attention function with an attention-map snapshot.
Deliverable: Save the code, output, and a short failure note.Future positions must receive zero attention and each row must normalize.
Remove the causal mask, change sequence length, and inspect attention weights.
A causal attention function with an attention-map snapshot.
Mapped from the supplied Build Everything PDF, source page 26. The five moves are Deriva’s implementation contract for this project.