A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data

1 code implementation • 26 Feb 2024 • Antonio Sclocchi, Alessandro Favero, Matthieu Wyart

We find that the backward diffusion process acting after a time $t$ is governed by a phase transition at some threshold time, where the probability of reconstructing high-level features, like the class of an image, suddenly drops.

On the different regimes of Stochastic Gradient Descent

1 code implementation • 19 Sep 2023 • Antonio Sclocchi, Matthieu Wyart

For small $B$ and large $\eta$, SGD corresponds to a stochastic evolution of the parameters, whose noise amplitude is governed by the ''temperature'' $T\equiv \eta/B$.

Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning

no code implementations • 31 Jan 2023 • Antonio Sclocchi, Mario Geiger, Matthieu Wyart

They show that SGD noise can be detrimental or instead useful depending on the training regime.

Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data

no code implementations • 7 Feb 2022 • Umberto M. Tomasini, Antonio Sclocchi, Matthieu Wyart

Recently, several theories including the replica method made predictions for the generalization error of Kernel Ridge Regression.

Surfing on minima of isostatic landscapes: avalanches and unjamming transition

no code implementations • 5 Oct 2020 • Silvio Franz, Antonio Sclocchi, Pierfrancesco Urbani

This algorithm allows to "surf" between isostatic marginally stable configurations and to investigate some properties of such landscape.

Disordered Systems and Neural Networks Statistical Mechanics