Match torch.softmax while moving less memory
one move: fuse memory passes
Open the code pane immediately, then use the Spec tab, optional design question, tests, and artifact when you want them.
Finish each move before carrying the artifact forward.
Online softmax keeps a running maximum and normalizer instead of writing intermediates.
Deliverable: Write the mechanism in one sentence.Fuse max, exponentiation, sum, and normalization into one Triton kernel.
Deliverable: Type the smallest runnable implementation.Rows must sum to one and match torch within numerical tolerance.
Deliverable: Record the expected output and one edge case.Vary row widths, precision, and block sizes; measure memory and latency.
Deliverable: Change one variable and explain the result.A fused softmax kernel with numerical and performance evidence.
Deliverable: Save the code, output, and a short failure note.Rows must sum to one and match torch within numerical tolerance.
Vary row widths, precision, and block sizes; measure memory and latency.
A fused softmax kernel with numerical and performance evidence.
Mapped from the supplied Build Everything PDF, source page 33. The five moves are Deriva’s implementation contract for this project.