Sparse autoencoders (SAEs) are popular for explaining what large language models are doing, but this paper shows they often don’t learn real, meaningful features.
ROCKET is a fast, training-free way to shrink big AI models while keeping most of their smarts.