Project 13 · Deep-learning systems0/5 completeMini Transformer
Tokenizer and batches, embeddings and positions, masked self-attention, a train step on the Project 12 engine, and a sampling failure report.
requires artifacts from: tensor-autograd
- L1Tokenizer + Batchesdefine the contract · ~16 minstart →
- L2Embeddings + Positionspreserve an invariant · ~20 minlocked
- L3Masked Self-Attentionmeasure behavior · ~26 minlocked
- L4Block + Train Stepbound a failure · ~30 minlocked
- L5Sampling + Failure Reportmake an operational decision · ~32 minlocked
Artifacts this project producesL1 · Tokenizer + Batchesoutputs: mini-transformer:l1-tokenizer-batches:artifact
L2 · Embeddings + Positionsoutputs: mini-transformer:l2-embeddings-positions:artifact
L3 · Masked Self-Attentionoutputs: mini-transformer:l3-masked-attention:artifact
L4 · Block + Train Stepoutputs: mini-transformer:l4-block-train-step:artifact
L5 · Sampling + Failure Reportoutputs: mini-transformer:l5-sampling-report:artifact