no code implementations • 6 May 2024 • Emre Onal, Klemens Flöge, Emma Caldwell, Arsen Sheverdin, Vincent Fortuin
Fine-tuned Large Language Models (LLMs) often suffer from overconfidence and poor calibration, particularly when fine-tuned on small datasets.
no code implementations • 28 Feb 2024 • Laura Manduchi, Kushagra Pandey, Robert Bamler, Ryan Cotterell, Sina Däubener, Sophie Fellenz, Asja Fischer, Thomas Gärtner, Matthias Kirchler, Marius Kloft, Yingzhen Li, Christoph Lippert, Gerard de Melo, Eric Nalisnick, Björn Ommer, Rajesh Ranganath, Maja Rudolph, Karen Ullrich, Guy Van Den Broeck, Julia E Vogt, Yixin Wang, Florian Wenzel, Frank Wood, Stephan Mandt, Vincent Fortuin
The field of deep generative modeling has grown rapidly and consistently over the years.
no code implementations • 25 Feb 2024 • Rayen Dhahri, Alexander Immer, Betrand Charpentier, Stephan Günnemann, Vincent Fortuin
Neural network sparsification is a promising avenue to save computational time and memory costs, especially in an age where many successful AI models are becoming too large to na\"ively deploy on consumer hardware.
no code implementations • 1 Feb 2024 • Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David Dunson, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, José Miguel Hernández-Lobato, Aliaksandr Hubin, Alexander Immer, Theofanis Karaletsos, Mohammad Emtiyaz Khan, Agustinus Kristiadi, Yingzhen Li, Stephan Mandt, Christopher Nemeth, Michael A. Osborne, Tim G. J. Rudner, David Rügamer, Yee Whye Teh, Max Welling, Andrew Gordon Wilson, Ruqi Zhang
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
no code implementations • 30 Nov 2023 • Alexander Möllers, Alexander Immer, Elvin Isufi, Vincent Fortuin
Graph contrastive learning has shown great promise when labeled data is scarce, but large unlabeled datasets are available.
no code implementations • 30 Oct 2023 • Szilvia Ujváry, Gergely Flamich, Vincent Fortuin, José Miguel Hernández Lobato
An important yet underexplored question in the PAC-Bayes literature is how much tightness we lose by restricting the posterior family to factorized Gaussian distributions when optimizing a PAC-Bayes bound.
1 code implementation • 28 Sep 2023 • Julyan Arbel, Konstantinos Pitas, Mariia Vladimirova, Vincent Fortuin
Neural networks have achieved remarkable performance across various problem domains, but their widespread applicability is hindered by inherent limitations such as overconfidence in predictions, lack of interpretability, and vulnerability to adversarial attacks.
no code implementations • 14 Sep 2023 • Alexander Möllers, Alexander Immer, Vincent Fortuin, Elvin Isufi
We leverage this decomposition to develop a contrastive self-supervised learning approach for processing simplicial data and generating embeddings that encapsulate specific spectral information. Specifically, we encode the pertinent data invariances through simplicial neural networks and devise augmentations that yield positive contrastive examples with suitable spectral properties for downstream tasks.
no code implementations • 29 Jun 2023 • Eliot Wong-Toi, Alex Boyd, Vincent Fortuin, Stephan Mandt
Deep, overparameterized regression models are notorious for their tendency to overfit.
no code implementations • 26 May 2023 • Kouroche Bouchiat, Alexander Immer, Hugo Yèche, Gunnar Rätsch, Vincent Fortuin
Neural additive models (NAMs) enhance the transparency of deep neural networks by handling input features in separate additive sub-networks.
1 code implementation • 17 Apr 2023 • Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Vincent Fortuin
The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks.
no code implementations • 4 Apr 2023 • Mrinank Sharma, Tom Rainforth, Yee Whye Teh, Vincent Fortuin
Conventional Bayesian Neural Networks (BNNs) cannot leverage unlabelled data to improve their predictions.
no code implementations • 14 Nov 2022 • Jonas Rothfuss, Martin Josifoski, Vincent Fortuin, Andreas Krause
Meta-Learning aims to speed up the learning process on new tasks by acquiring useful inductive biases from datasets of related learning tasks.
1 code implementation • 22 Feb 2022 • Alexander Immer, Tycho F. A. van der Ouderaa, Gunnar Rätsch, Vincent Fortuin, Mark van der Wilk
We develop a convenient gradient-based method for selecting the data augmentation without validation data during training of a deep neural network.
no code implementations • pproximateinference AABI Symposium 2022 • Marcello Massimo Negri, Vincent Fortuin, Jan Stuehmer
Variational auto-encoders have proven to capture complicated data distributions and useful latent representations, while advances in meta-learning have made it possible to extract prior knowledge from data.
1 code implementation • ACL 2022 • Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin, Ryan Cotterell
Such performance improvements have motivated researchers to quantify and understand the linguistic information encoded in these representations.
no code implementations • NeurIPS Workshop ICBINB 2021 • Tristan Cinquin, Alexander Immer, Max Horn, Vincent Fortuin
In recent years, the transformer has established itself as a workhorse in many applications ranging from natural language processing to reinforcement learning.
1 code implementation • 7 Oct 2021 • James Urquhart Allingham, Florian Wenzel, Zelda E Mariet, Basil Mustafa, Joan Puigcerver, Neil Houlsby, Ghassen Jerfel, Vincent Fortuin, Balaji Lakshminarayanan, Jasper Snoek, Dustin Tran, Carlos Riquelme Ruiz, Rodolphe Jenatton
Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models.
no code implementations • 6 Oct 2021 • Vincent Fortuin, Mark Collier, Florian Wenzel, James Allingham, Jeremiah Liu, Dustin Tran, Balaji Lakshminarayanan, Jesse Berent, Rodolphe Jenatton, Effrosyni Kokiopoulou
Uncertainty estimation in deep learning has recently emerged as a crucial area of interest to advance reliability and robustness in safety-critical applications.
2 code implementations • pproximateinference AABI Symposium 2022 • Lauro Langosco di Langosco, Vincent Fortuin, Heiko Strathmann
Particle-based approximate Bayesian inference approaches such as Stein Variational Gradient Descent (SVGD) combine the flexibility and convergence guarantees of sampling methods with the computational benefits of variational inference.
no code implementations • 20 Jul 2021 • Nikolaos Mourdoukoutas, Marco Federici, Georges Pantalos, Mark van der Wilk, Vincent Fortuin
We propose a novel Bayesian neural network architecture that can learn invariances from data alone by inferring a posterior distribution over different weight-sharing schemes.
1 code implementation • NeurIPS 2021 • Francesco D'Angelo, Vincent Fortuin
Deep ensembles have recently gained popularity in the deep learning community for their conceptual simplicity and efficiency.
no code implementations • 20 Jun 2021 • Francesco D'Angelo, Vincent Fortuin, Florian Wenzel
Ensembles of deep neural networks have achieved great success recently, but they do not offer a proper Bayesian justification.
no code implementations • 10 Jun 2021 • Seth Nabarro, Stoil Ganev, Adrià Garriga-Alonso, Vincent Fortuin, Mark van der Wilk, Laurence Aitchison
Here, we provide several approaches to developing principled Bayesian neural networks incorporating data augmentation.
no code implementations • 14 May 2021 • Vincent Fortuin
While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians.
1 code implementation • 14 May 2021 • Vincent Fortuin, Adrià Garriga-Alonso, Mark van der Wilk, Laurence Aitchison
Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance.
1 code implementation • 11 Apr 2021 • Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, Mohammad Emtiyaz Khan
Marginal-likelihood based model-selection, even though promising, is rarely used in deep learning due to estimation difficulties.
1 code implementation • NeurIPS Workshop ICBINB 2020 • Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W. Ober, Florian Wenzel, Gunnar Rätsch, Richard E. Turner, Mark van der Wilk, Laurence Aitchison
Isotropic Gaussian priors are the de facto standard for modern Bayesian neural network inference.
no code implementations • pproximateinference AABI Symposium 2022 • Simon Bing, Vincent Fortuin, Gunnar Rätsch
While many models have been introduced to learn such disentangled representations, only few attempt to explicitly exploit the structure of sequential data.
no code implementations • pproximateinference AABI Symposium 2021 • Adrià Garriga-Alonso, Vincent Fortuin
Stochastic gradient Markov Chain Monte Carlo algorithms are popular samplers for approximate inference, but they are generally biased.
no code implementations • pproximateinference AABI Symposium 2021 • Francesco D'Angelo, Vincent Fortuin
Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution.
1 code implementation • pproximateinference AABI Symposium 2021 • Metod Jazbec, Michael Pearce, Vincent Fortuin
Variational autoencoders often assume isotropic Gaussian priors and mean-field posteriors, hence do not exploit structure in scenarios where we may expect similarity or consistency across latent variables.
1 code implementation • 26 Oct 2020 • Metod Jazbec, Matthew Ashman, Vincent Fortuin, Michael Pearce, Stephan Mandt, Gunnar Rätsch
Conventional variational autoencoders fail in modeling correlations between data points due to their use of factorized priors.
1 code implementation • 20 Oct 2020 • Matthew Ashman, Jonathan So, Will Tebbutt, Vincent Fortuin, Michael Pearce, Richard E. Turner
Large, multi-dimensional spatio-temporal datasets are omnipresent in modern science and engineering.
no code implementations • ICLR 2020 • Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, Richard Turner
Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks.
3 code implementations • ICML Workshop LifelongML 2020 • Jonas Rothfuss, Vincent Fortuin, Martin Josifoski, Andreas Krause
Meta-learning can successfully acquire useful inductive biases from data.
1 code implementation • 17 Oct 2019 • Andreas Kopf, Vincent Fortuin, Vignesh Ram Somnath, Manfred Claassen
Clustering high-dimensional data, such as images or biological measurements, is a long-standingproblem and has been studied extensively.
no code implementations • pproximateinference AABI Symposium 2019 • Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, Stephan Mandt
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction Multivariate Time Series Imputation +3
2 code implementations • 3 Oct 2019 • Laura Manduchi, Matthias Hüser, Julia Vogt, Gunnar Rätsch, Vincent Fortuin
We show that DPSOM achieves superior clustering performance compared to current deep clustering methods on MNIST/Fashion-MNIST, while maintaining the favourable visualization properties of SOMs.
1 code implementation • 28 Sep 2019 • Andreas Georgiou, Vincent Fortuin, Harun Mustafa, Gunnar Rätsch
We therefore aim to develop a more memory-efficient technique for taxonomic classification.
1 code implementation • 27 Sep 2019 • Margherita Rosnati, Vincent Fortuin
With a mortality rate of 5. 4 million lives worldwide every year and a healthcare cost of more than 16 billion dollars in the USA alone, sepsis is one of the leading causes of hospital mortality and an increasing concern in the ageing western world.
no code implementations • 25 Sep 2019 • Laura Manduchi, Matthias Hüser, Gunnar Rätsch, Vincent Fortuin
There are very performant deep clustering models on the one hand and interpretable representation learning techniques, often relying on latent topological structures such as self-organizing maps, on the other hand.
no code implementations • 25 Sep 2019 • Andreas Kopf, Vincent Fortuin, Vignesh Ram Somnath, Manfred Claassen
MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture.
no code implementations • 25 Sep 2019 • Andreas Georgiou, Vincent Fortuin, Harun Mustafa, Gunnar Rätsch
Of particular interest is the determination of the distribution of the taxa of microbes in metagenomic samples.
3 code implementations • 9 Jul 2019 • Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, Stephan Mandt
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction Multivariate Time Series Imputation +2
no code implementations • 23 Jan 2019 • Vincent Fortuin, Heiko Strathmann, Gunnar Rätsch
When it comes to meta-learning in Gaussian process models, approaches in this setting have mostly focused on learning the kernel function of the prior, but not on learning its mean function.
no code implementations • 24 Oct 2018 • Vincent Fortuin, Gideon Dresdner, Heiko Strathmann, Gunnar Rätsch
We explore different techniques for selecting inducing points on discrete domains, including greedy selection, determinantal point processes, and simulated annealing.
6 code implementations • ICLR 2019 • Vincent Fortuin, Matthias Hüser, Francesco Locatello, Heiko Strathmann, Gunnar Rätsch
We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set.