Get Started
Home
Topics
Search
Library
Research questionHow can sparse autoencoder features be shared across language models without per-model retraining?Per-model sparse autoencoders require separate training and latent labelling, making feature comparison and reuse across language models expensive. A shared representation must still preserve activation information and reconstruct each model’s activations effectively.
AI
Machine Learning
Mechanistic Interpretability
Natural Language Processing
Latest papersRecent research connected to this question, newest first.SharedSAE: One Feature Dictionary Across Language ModelsThe source studies a shared sparse-autoencoder dictionary with model-specific encoder-decoder pairs across four 1B-scale base language models from distinct families and tokenizers. Evidence covers reconstruction quality, cross-model latent correlations, transferred latent descriptions, and adaptation of additional models after the dictionary is frozen; broader model scales and deployment conditions are not established.research paper · Sep 3, 2026
Related questions
How can sparse autoencoders capture language-model features that persist across token sequences?How can multilingual representation sharing be measured without confusing anisotropy with genuine cross-lingual structure?How can we compare feature contributions to language-model representations when those features are correlated?How can language models personalize for many users without costly per-user model storage?