Deriva — Derive the Algorithm

← ProjectBuild Everything / M3FlashAttention in Triton~240 min

M3fuse attention safely120 lines · C2 + D2

Express the memory-saving idea behind FlashAttention as a tiled causal reduction: compute each output from a bounded trailing window without materializing a score matrix.

Required API

def flash_prefix_average(values, tile):
    return outputs

Behavior

never read future positions
bound the active window by tile
emit one output per input

Optional design question

Think through the mechanism first if you want the extra reasoning step. It never blocks the editor.

Visible contract tests

•
tile twoflash_prefix_average([1.0, 3.0, 5.0], 2)
a bounded tile still preserves local causal context
•
tile oneflash_prefix_average([2.0, 4.0], 1)
a one-token tile is the identity

2 hidden edge tests run after the visible contract passes.

def flash_prefix_average(values, tile):
    # Compute each causal output from at most tile recent values.
    pass

Run Tests to see the contract verdicts here.

0 runs

Opening…