Get Started
Home
Topics
Search
Library
Research questionHow does gradient training build hierarchical representations beyond the lazy kernel regime?Gradient training can change a network’s representations, so lazy or kernel descriptions do not explain how features are selected across layers. It remains difficult to connect these layerwise changes to the emergence of higher-level concepts and the sample complexity required for them.
AI
Machine Learning
Mechanistic Interpretability
Neural and Evolutionary Computing
Research Paper
Statistical Machine Learning
Latest papersRecent research connected to this question, newest first.Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature LearningThe source studies Neural LoFi, a stylized limit of gradient-based training in which each layer selects directions with maximal accessible low-degree label correlation. It provides a kernel-space interpretation and analyzes fully connected and convolutional architectures through theory and mechanistic experiments, with evidence of structured filters and alignment with early gradient-descent feature discovery on real datasets.research paper · Sep 4, 2026
Related questions
How should reference baselines be chosen to make gradient-based attributions meaningful and reliable?How can multilayer RNNs learn nonlinear quadrotor dynamics without vanishing or exploding gradients?How can we select essential hidden neurons from activations alone without labels or gradients while preserving discrimination?How can LoRA initialization preserve full-rank training gradients despite its low-rank bottleneck?