Search Results for author:
Found 3 papers, 0 papers with code
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD
We further demonstrate that, SGD-trained ReLU NNs can learn a single-index target of the form $y=f(\langle\boldsymbol{u},\boldsymbol{x}\rangle) + \epsilon$ by recovering the principal direction, with a sample complexity linear in $d$ (up to log factors), where $f$ is a monotonic function with at most polynomial growth, and $\epsilon$ is the noise.
Convergence Analysis and Implicit Regularization of Feedback Alignment for Deep Linear Networks
We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks.
A Study of Condition Numbers for First-Order Optimization
Since smoothness and strong convexity are not continuous, we propose a comprehensive study of existing alternative metrics which we prove to be continuous.
