Deriva — Derive the Algorithm

← Projectsmini-transformer / L1Tokenizer + Batches~16 min

one move: define the contract

Count what token follows each token. This is the complete sufficient statistic for a tiny bigram language model before probabilities or sampling. In the Mini Transformer system, implement this level as a deterministic contract before composing it with the next service.

Required API

def bigram_counts(tokens):
    return {token: {next_token: count}}

Behavior

look only at adjacent pairs
create a row for each source token
increment repeated transitions

Examples — tap to reveal

Constraints

Use only the Python standard library.
Return deterministic values so every run can be compared with the contract.
Do not mutate caller-owned inputs.

Opening…