Search Results for author:
Found 14 papers, 6 papers with code
Discrete error dynamics of mini-batch gradient descent for least squares regression
We study the discrete dynamics of mini-batch gradient descent for least squares regression when sampling without replacement.
Near-Interpolators: Rapid Norm Growth and the Trade-Off between Interpolation and Generalization
Under a random matrix theoretic assumption on the data distribution and an eigendecay assumption on the data covariance matrix $\boldsymbol{\Sigma}$, we demonstrate that any near-interpolator exhibits rapid norm growth: for $\tau$ fixed, $\boldsymbol{\beta}$ has squared $\ell_2$-norm $\mathbb{E}[\|{\boldsymbol{\beta}}\|_{2}^{2}] = \Omega(n^{\alpha})$ where $n$ is the number of samples and $\alpha >1$ is the exponent of the eigendecay, i. e., $\lambda_i(\boldsymbol{\Sigma}) \sim i^{-\alpha}$.
Spectral Neural Networks: Approximation Theory and Optimization Landscape
There is a large variety of machine learning methodologies that are based on the extraction of spectral geometric information from data.
Double Descent and Overfitting under Noisy Inputs and Distribution Shift for Linear Denoisers
Motivated by this, we study supervised denoising and noisy-input regression under distribution shift.
Least Squares Regression Can Exhibit Under-Parameterized Double Descent
The relationship between the number of training data points, the number of parameters, and the generalization capabilities has been widely studied.
Supermodular Rank: Set Function Decomposition and Optimization
We define the supermodular rank of a function on a lattice.
Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network
For that, we use more than 100, 000 research papers and build up a knowledge network with more than 64, 000 concept nodes.
How can classical multidimensional scaling go wrong?
While $D_l$ is not metric, when given as input to cMDS instead of $D$, it empirically results in solutions whose distance to $D$ does not increase when we increase the dimension and the classification accuracy degrades less than the cMDS solution.
An Analysis of COVID-19 Knowledge Graph Construction and Applications
The construction and application of knowledge graphs have seen a rapid increase across many disciplines in recent years.
Training Data Size Induced Double Descent For Denoising Neural Networks and the Role of Training Noise Level
In fact the generalization error versus number of of training data points is a double descent curve.
Project and Forget: Solving Large-Scale Metric Constrained Problems
Given a set of dissimilarity measurements amongst data points, determining what metric representation is most "consistent" with the input measurements or the metric that best captures the relevant geometric features of the data is a key step in many machine learning algorithms.
Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding
Given data, finding a faithful low-dimensional hyperbolic embedding of the data is a key method by which we can extract hierarchical information or learn representative geometric features of the data.
Project and Forget: Solving Large Scale Metric Constrained Problems
Given a set of distances amongst points, determining what metric representation is most “consistent” with the input distances or the metric that captures the relevant geometric features of the data is a key step in many machine learning algorithms.
Unsupervised Metric Learning in Presence of Missing Data
Here, we present a new algorithm MR-MISSING that extends these previous algorithms and can be used to compute low dimensional representation on data sets with missing entries.
