Get Started
Home
Topics
Search
Library
Research questionWhich routing and mixing behaviors in trained four-stream residual pathways materially affect model performance?Expanding a residual pathway to multiple streams creates additional representational capacity, but it is unclear whether blocks use all streams, how information mixes across depth, and which interactions are functionally necessary.
AI
Evaluation & Benchmarks
Machine Learning
Mechanistic Interpretability
Research Paper
Technology
Latest papersRecent research connected to this question, newest first.How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity MixingThe evidence comes from the four-stream mHC pathway in DeepSeek-V4-Flash, using effective stream counts, cross-stream residual weights, inter-stream cosine similarity, and targeted interventions. Effects are measured with C4 perplexity and a six-task average score, so the findings are limited to this model and these evaluations.research paper · Sep 4, 2026
Related questions
How can multi-stream Transformer residual mixing remain stable without amplifying signals or collapsing stream diversity?How can we tell whether cross-layer routing predictability reflects shared routing dynamics or generic hidden-state smoothness?How should experts be pruned in over-dispersed MoE routing when router importance and perplexity mislead?Why do pretrained language models collapse continuous mixtures of hypotheses during latent-state reasoning?