Get Started
Home
Topics
Search
Library
Research questionHow can sparse autoencoders capture language-model features that persist across token sequences?Token-wise sparse autoencoders can miss signals whose meaning unfolds or remains relevant across many tokens. The persistence of such signals varies by feature, complicating interpretation and monitoring of language-model behavior.
AI
AI Memory
Alignment & Safety
Mechanistic Interpretability
Natural Language Processing
Latest papersRecent research connected to this question, newest first.Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model RepresentationsApplies to language-model activation analysis with sparse autoencoders. The evidence covers naturally persistent features, learned feature-specific timescales, reconstruction quality, and a prompt-injection monitoring case study in which slow features retain injection-related signals over long contexts.research paper · Sep 2, 2026
Related questions
How can sparse autoencoder features be shared across language models without per-model retraining?How can language models sparsely access a growing internal state without limiting it to fixed-size memory?How can language-model attention remain reliable beyond its training context?How can autoregressive language models add recurrent cross-token memory without breaking KV caching or one-forward-per-token decoding?