Summary of Residual Stream Analysis with Multi-layer Saes, by Tim Lawson et al.
Residual Stream Analysis with Multi-Layer SAEs
by Tim Lawson, Lucy Farnik, Conor Houghton, Laurence Aitchison
First submitted to arxiv on: 6 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces the multi-layer sparse autoencoder (MLSAE), a novel approach to understanding how transformer language models process information across layers. The MLSAE is trained on residual stream activation vectors from every transformer layer, allowing for the study of information flow across layers. The authors find that individual latents are often active at a single layer for a given token or prompt, but the layer at which an individual latent is active may differ for different tokens or prompts. They quantify these phenomena by defining a distribution over layers and considering its variance. The results show that the variance of the distributions of latent activations over layers increases when aggregating over tokens compared to a single token. For larger underlying models, the degree to which latents are active at multiple layers also increases. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us understand how transformers work by training a special kind of autoencoder on every layer’s information. The autoencoder finds patterns in this information that tell us where it’s being used and what it means. Surprisingly, these patterns are often specific to one layer or another, even for the same sentence. This might be because each layer is looking at the information from different angles. The researchers found a way to measure how much these patterns change as they move through the layers and saw that it depends on whether you’re looking at one sentence or many sentences together. This helps us understand how transformers can get better at understanding longer texts. |
Keywords
» Artificial intelligence » Autoencoder » Prompt » Token » Transformer
