Get Started
Home
Topics
Search
Library
Research questionHow can Transformers reuse local static patterns through lookup memory without tying memory capacity to backbone width?Transformers repeatedly recompute local patterns instead of directly retrieving them. Existing latent n-gram memory can reduce this repetition, but its capacity and cost are constrained by the backbone’s width.
AI
AI Memory
Inference Optimization
LLM Pretraining & Post-training
Machine Learning
Multimodal Models
Natural Language Processing
Latest papersRecent research connected to this question, newest first.Lngram v2: Latent N-Gram Memory with Interpretable Discrete RepresentationsThe evidence concerns tokenizer-independent latent conditional memory with discrete n-gram addressing in vision-language models of different scales, including a 30B-parameter model. It reports effects on total and activated memory parameters, language-modeling performance, and the semantic structure of discrete IDs; broader deployment behavior is not established.research paper · Sep 3, 2026
Related questions
Does reusing Transformer layers improve language-model quality when parameter, compute, and KV-cache budgets are matched?How can Vision Transformers avoid LayerNorm’s global reduction on edge accelerators without expensive retraining?How can full-graph visibility improve fusion and hardware utilization during Transformer training?How can neural networks retain capacity while fitting within fixed parameter and memory budgets?