no code implementations • 22 May 2024 • Liam Madden
The minimal number of neurons required for a feedforward neural network to interpolate $n$ generic input-output pairs from $\mathbb{R}^d\times \mathbb{R}$ is $\Theta(\sqrt{n})$.
no code implementations • 22 May 2024 • Liam Madden, Curtis Fox, Christos Thrampoulidis
Given a sequence of tokens, such as words, the task of next-token prediction is to predict the next-token conditional probability distribution.
no code implementations • 3 Aug 2023 • Liam Madden, Christos Thrampoulidis
In order to analyze general activations, we derive the precise generic rank of the network's Jacobian, which can be written in terms of Hadamard powers and the Khatri-Rao product.
no code implementations • 17 Oct 2019 • Emiliano Dall'Anese, Andrea Simonetto, Stephen Becker, Liam Madden
Approaches for the design of time-varying or online first-order optimization methods are discussed, with emphasis on algorithms that can handle errors in the gradient, as may arise when the gradient is estimated.