no code implementations • 26 Sep 2023 • Thijs Bos, Johannes Schmidt-Hieber
Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods.
no code implementations • 26 Sep 2023 • Johannes Schmidt-Hieber, Wouter M Koolen
In this work, we study a variation of this Hebbian learning rule to recover the regression vector in the linear regression model.
no code implementations • 18 Jun 2023 • Gabriel Clara, Sophie Langer, Johannes Schmidt-Hieber
We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model.
no code implementations • 27 Jan 2023 • Johannes Schmidt-Hieber
In particular, the locality in the updating rule of the connection parameters in biological neural networks (BNNs) makes it biologically implausible that the learning of the brain is based on gradient descent.
no code implementations • 16 May 2022 • Matteo Giordano, Kolyan Ray, Johannes Schmidt-Hieber
We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure.
1 code implementation • 12 Jan 2022 • Masaaki Imaizumi, Johannes Schmidt-Hieber
We argue that under reasonable assumptions, the local geometry forces SGD to stay close to a low dimensional subspace and that this induces another form of implicit regularization and results in tighter bounds on the generalization error for deep neural networks.
no code implementations • 2 Aug 2021 • Thijs Bos, Johannes Schmidt-Hieber
For classification problems, trained deep neural networks return probabilities of class memberships.
no code implementations • 31 Jul 2020 • Johannes Schmidt-Hieber
There is a longstanding debate whether the Kolmogorov-Arnold representation theorem can explain the use of more than one hidden layer in neural networks.
no code implementations • 30 May 2020 • Alexis Derumigny, Johannes Schmidt-Hieber
In a second part of the article, the abstract lower bounds are applied to several statistical models including the Gaussian white noise model, a boundary estimation problem, the Gaussian sequence model and the high-dimensional linear regression model.
no code implementations • 2 Aug 2019 • Johannes Schmidt-Hieber
Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known.
no code implementations • 6 Apr 2018 • Konstantin Eckle, Johannes Schmidt-Hieber
Deep neural networks (DNNs) generate much richer function spaces than shallow networks.
no code implementations • 22 Aug 2017 • Johannes Schmidt-Hieber
It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log n$-factors) under a general composition assumption on the regression function.