Deriva — Derive the Algorithm

← Projectsmini-transformer / L3Masked Self-Attention~26 min

one move: measure behavior

Implement the causal information flow of one attention head as a prefix average: each position can read itself and the past, never the future. In the Mini Transformer system, implement this level as a deterministic contract before composing it with the next service.

Required API

def causal_attention(values):
    return [prefix_average, ...]

Behavior

include the current position in its receptive field
exclude every future position
normalize each prefix by its length

Examples — tap to reveal

Constraints

Use only the Python standard library.
Return deterministic values so every run can be compared with the contract.
Do not mutate caller-owned inputs.

Opening…