1 code implementation • 31 Jul 2023 • Gonçalo Mordido, Pranshu Malviya, Aristide Baratin, Sarath Chandar
In this work, we increase the efficiency of the maximization and minimization parts of SAM's objective to achieve a better loss-sharpness trade-off.
1 code implementation • 18 Jul 2023 • Pranshu Malviya, Gonçalo Mordido, Aristide Baratin, Reza Babanezhad Harikandeh, Jerry Huang, Simon Lacoste-Julien, Razvan Pascanu, Sarath Chandar
Adaptive gradient-based optimizers, particularly Adam, have left their mark in training large-scale deep learning models.
no code implementations • 18 Nov 2022 • Gonçalo Mordido, Sébastien Henwood, Sarath Chandar, François Leduc-Primeau
In this work, we show that applying sharpness-aware training, by optimizing for both the loss value and loss sharpness, significantly improves robustness to noisy hardware at inference time without relying on any assumptions about the target hardware.
1 code implementation • 3 Aug 2022 • Simon Guiroy, Christopher Pal, Gonçalo Mordido, Sarath Chandar
Specifically, we analyze the evolution, during meta-training, of the neural activations at each hidden layer, on a small set of unlabelled support examples from a single task of the target tasks distribution, as this constitutes a minimal and justifiably accessible information from the target problem.
no code implementations • 3 May 2022 • Jonathan Kern, Sébastien Henwood, Gonçalo Mordido, Elsa Dupraz, Abdeldjalil Aïssa-El-Bey, Yvon Savaria, François Leduc-Primeau
Memristors enable the computation of matrix-vector multiplications (MVM) in memory and, therefore, show great potential in highly increasing the energy efficiency of deep neural network (DNN) inference accelerators.
no code implementations • 31 Mar 2021 • Gonçalo Mordido, Matthijs Van Keirsbilck, Alexander Keller
We demonstrate that 1x1-convolutions in 1D time-channel separable convolutions may be replaced by constant, sparse random ternary matrices with weights in $\{-1, 0,+1\}$.
no code implementations • 22 Mar 2021 • Gonçalo Mordido, Haojin Yang, Christoph Meinel
The analysis of the compression effects in generative adversarial networks (GANs) after training, i. e. without any fine-tuning, remains an unstudied, albeit important, topic with the increasing trend of their computation and memory requirements.
no code implementations • COLING 2020 • Gonçalo Mordido, Christoph Meinel
We propose a family of metrics to assess language generation derived from population estimation methods widely used in ecology.
1 code implementation • 3 Feb 2020 • Julian Niedermeier, Gonçalo Mordido, Christoph Meinel
Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future.
no code implementations • 10 Jan 2020 • Gonçalo Mordido, Haojin Yang, Christoph Meinel
We propose to tackle the mode collapse problem in generative adversarial networks (GANs) by using multiple discriminators and assigning a different portion of each minibatch, called microbatch, to each discriminator.
no code implementations • 29 May 2019 • Gonçalo Mordido, Matthijs Van Keirsbilck, Alexander Keller
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption.
no code implementations • 30 Jul 2018 • Gonçalo Mordido, Haojin Yang, Christoph Meinel
We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch.