Get Started
Home
Topics
Search
Library
Research questionHow does the leaky-ReLU parameter alter critical-point bifurcations and symmetry-breaking stability in shallow networks?Changing the leaky-ReLU parameter can cause symmetry-related critical points to bifurcate or change stability. The key difficulty is determining when these transitions occur and whether the global minimum becomes symmetry-breaking unstable.
AI
Machine Learning
Neural and Evolutionary Computing
Research Paper
Technology
Latest papersRecent research connected to this question, newest first.Nonlinear Dynamics In Optimization Landscape of Shallow Neural Networks with Tunable Leaky ReLUThis theoretical analysis considers shallow leaky-ReLU networks with Gaussian inputs, equal layer width k, and mean-squared loss, for k ≥ 4. It studies variation in the leaky parameter α, identifies associated symmetries and multi-mode degeneracy at critical number 0, and includes an explicit k = 5 example. The reported results cover width-independent bifurcations for nonnegative α and no further symmetry-breaking instability of the global minimum for α in (0, 1).research paper · Sep 1, 2026
Related questions
How can we characterize when deep ReLU networks with different architectures or parameters compute the same function on the unit cube?How do initialization correlation decay and feature-function Hermite rank determine residual networks’ scaling and continuous-depth limit?How do stochastic optimizers progressively steer deep networks into simpler invariant subnetworks?Can non-smooth or quantized activations support stable echo-state dynamics beyond conventional spectral-radius expectations?