Summary of Can Sparse Autoencoders Be Used to Decompose and Interpret Steering Vectors?, by Harry Mayne et al.
Can sparse autoencoders be used to decompose and interpret steering vectors?
by Harry Mayne, Yushi Yang, Adam Mahdi
First submitted to arxiv on: 13 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel investigation into the mechanisms underlying steering vectors, a promising approach for controlling large language models, reveals two critical limitations that hinder their interpretation using sparse autoencoders (SAEs). The study finds that SAE-reconstructed steering vectors often lack the original’s steering properties due to these limitations: (1) steering vectors falling outside the input distribution designed for SAEs and (2) negative projections in feature directions not accommodated by SAEs. This work provides insights into the reasons behind these misleading decompositions, paving the way for improved methods to interpret steering vectors. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Steering vectors are a new way to control big language models. But scientists don’t fully understand how they work. They thought that using special math (sparse autoencoders) could help them figure it out. However, when they tried this approach, the results were not what they expected. They found two reasons why this didn’t work: first, steering vectors are different from what these math tools are designed to handle, and second, some of these vectors have important negative parts that these tools can’t capture. This makes it harder to use these math tools to understand how steering vectors work. |
