Mechanistic Interpretability

Reading the Framework paper, final part. Whose shoulders this paper stands on, and how its approach differs from the lines of work before it.

Reading the Framework paper, part 6. What the framework secured and what it did not: the wall called MLP, and the follow-up research this paper announces.

Reading the Framework paper, part 5. With two layers, heads compose with heads: Q-, K-, and V-composition, and the paper’s biggest discovery, the mechanism of the induction head.

Reading the Framework paper, part 4. Fully expanding a one-attention-layer model into paths: skip-trigrams, copying heads and positive eigenvalues, and the characteristic bugs the factored structure produces.

Reading the Framework paper, part 3. What does a transformer with no attention at all learn? Bigram statistics, and that answer turns out to be the key to the direct path in large models.

Reading the Framework paper, part 2. Redrawing the residual stream as a communication channel and attention heads as independent information movers, through virtual weights and the QK/OV split.

Reading the Framework paper, part 1. Treating tiny attention-only models as model organisms for fully reverse engineering transformers, and a map of the key results.

Reverse-engineering neural networks, circuit by circuit. Anthropic’s Transformer Circuits thread and a few companion resources are all free. Where to start, to open this series.

Finding emotion-related features in GPT-2 using OpenAI’s pretrained SAE, then training one from scratch. Feature patching turns ‘good person’ into ‘shit’.