Get Started
Home
Topics
Search
Library
Research questionHow do stochastic optimizers progressively steer deep networks into simpler invariant subnetworks?The way stochastic optimization reorganizes a deep network over training time is not well understood. Architectural symmetries can produce simultaneous structural transitions whose fluctuations appear as variance spikes in a macroscopic order parameter.
AI
Machine Learning
Neural and Evolutionary Computing
Research Paper
Statistical Machine Learning
Technology
Latest papersRecent research connected to this question, newest first.Percolation Dynamics in Optimization : Variance Cascades and Discrete Scale InvarianceThe analysis models stochastic gradient flow as a percolation process in which subnetworks merge in discrete blocks. It treats SGD and extends the trapping and scaling-cascade analysis to Adam and AdamW under an explicit heavy-tailed noise model.research paper · Sep 2, 2026
Related questions
How can neural-network optimization make additive updates produce consistent relative changes across differently sized weights?How can scientific machine-learning models overcome optimization plateaus when their local linearized subspace permits greater accuracy?How can Adam’s error be bounded for strongly convex stochastic optimization without assuming bounded iterates?How does the leaky-ReLU parameter alter critical-point bifurcations and symmetry-breaking stability in shallow networks?