no code implementations • 1 May 2024 • Rishav Mukherji, Mark Schöne, Khaleelulla Khan Nazeer, Christian Mayr, David Kappel, Anand Subramoney
Activity and parameter sparsity are two standard methods of making neural networks computationally more efficient.
no code implementations • 9 Jan 2024 • Hector A. Gonzalez, Jiaxin Huang, Florian Kelber, Khaleelulla Khan Nazeer, Tim Langer, Chen Liu, Matthias Lohrmann, Amirhossein Rostami, Mark Schöne, Bernhard Vogginger, Timo C. Wunderlich, Yexin Yan, Mahmoud Akl, Christian Mayr
This development is accompanied by a rapid growth of the required computational demands for larger models and more data.
no code implementations • 14 Dec 2023 • Khaleelulla Khan Nazeer, Mark Schöne, Rishav Mukherji, Bernhard Vogginger, Christian Mayr, David Kappel, Anand Subramoney
In this work, we demonstrate the first-ever implementation of a language model on a neuromorphic device - specifically the SpiNNaker 2 chip - based on a recently published event-based architecture called the EGRU.
no code implementations • 13 Nov 2023 • Rishav Mukherji, Mark Schöne, Khaleelulla Khan Nazeer, Christian Mayr, Anand Subramoney
Yet, sparse activations, while omnipresent in both biological neural networks and deep learning systems, have not been fully utilized as a compression technique in deep learning.
no code implementations • 24 May 2023 • David Kappel, Khaleelulla Khan Nazeer, Cabrel Teguemne Fokam, Christian Mayr, Anand Subramoney
In addition, back-propagation relies on the transpose of forward weight matrices to compute updates, introducing a weight transport problem across the network.
1 code implementation • 13 Jun 2022 • Anand Subramoney, Khaleelulla Khan Nazeer, Mark Schöne, Christian Mayr, David Kappel
However, there is still a need to bridge the gap between what RNNs are capable of in terms of efficiency and performance and real-world application requirements.
Ranked #2 on Gesture Recognition on DVS128 Gesture (using extra training data)