Generate coherent English stories
one move: compose a causal model
Open the code pane immediately, then use the Spec tab, optional design question, tests, and artifact when you want them.
Finish each move before carrying the artifact forward.
A modern decoder combines embeddings, causal GQA, RoPE, RMSNorm, SwiGLU, and weight tying.
Deliverable: Write the mechanism in one sentence.Build a tiny decoder-only transformer and train it on a short local corpus.
Deliverable: Type the smallest runnable implementation.Masks, shapes, loss, and seeded sampling must remain finite and reproducible.
Deliverable: Record the expected output and one edge case.Remove one component at a time and record the quality or stability change.
Deliverable: Change one variable and explain the result.A tiny language-model checkpoint, sample, and failure report.
Deliverable: Save the code, output, and a short failure note.Masks, shapes, loss, and seeded sampling must remain finite and reproducible.
Remove one component at a time and record the quality or stability change.
A tiny language-model checkpoint, sample, and failure report.
Mapped from the supplied Build Everything PDF, source page 13. The five moves are Deriva’s implementation contract for this project.