Get Started
Home
Topics
Search
Library
Research questionHow can full-graph visibility improve fusion and hardware utilization during Transformer training?Tensor-framework abstractions can obscure relationships across a complete Transformer computation. This makes it difficult to combine attention, element-wise, and normalization operations while mapping training efficiently to hardware.
AI
LLM Pretraining & Post-training
Machine Learning
Technology
Latest papersRecent research connected to this question, newest first.Nova: An End-to-End MLIR Compiler for Deep LearningThe source describes an end-to-end MLIR JIT compiler that captures eager execution, unifies forward and backward passes, and synthesizes fused kernels for full Transformers. Evidence comes from GPT-2 training on Ada 6000 GPUs and includes numerical-parity and throughput results.research paper · Sep 2, 2026
Related questions
Does reusing Transformer layers improve language-model quality when parameter, compute, and KV-cache budgets are matched?How can Vision Transformers avoid LayerNorm’s global reduction on edge accelerators without expensive retraining?How can high-resolution diffusion Transformers prune tokens without sacrificing image fidelity or predictable compute?How can Transformers reuse local static patterns through lookup memory without tying memory capacity to backbone width?