Get Started
Home
Topics
Search
Library
Research questionHow do initialization correlation decay and feature-function Hermite rank determine residual networks’ scaling and continuous-depth limit?As depth grows, dependence among layer weights changes the aggregate fluctuations driving residual updates. The appropriate depth scaling and resulting continuous-depth dynamics therefore depend on the initialization’s temporal correlation structure and feature transformation.
AI
Machine Learning
Neural and Evolutionary Computing
Research Paper
Statistical Machine Learning
Technology
Latest papersRecent research connected to this question, newest first.Correlated initialization of deep residual networksThe results concern initializations formed by applying a feature function to a stationary Gaussian sequence with regularly varying correlations. They characterize a critical scaling and Young differential equation driven by a Hermite process, with fractional Brownian motion as the Hermite-rank-one case, and contrast this with the Brownian limit under finite-variance iid initialization.research paper · Sep 3, 2026
Related questions
How can residual networks approximate high-dimensional semilinear heat-equation solutions without exponential parameter growth?How can LoRA initialization preserve full-rank training gradients despite its low-rank bottleneck?How does downstream learning error scale with depth when partially informed agents pass only predictions along M-covered paths?How does the leaky-ReLU parameter alter critical-point bifurcations and symmetry-breaking stability in shallow networks?