Search Results for author:
Found 6 papers, 1 papers with code
Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models
Activity and parameter sparsity are two standard methods of making neural networks computationally more efficient.
Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models
While prior methods can process up to a few thousand time steps, our model, based on modern recurrent deep state-space models, scales to event streams of millions of events for both training and inference. We leverage their stable parameterization for learning long-range dependencies, parallelizability along the sequence dimension, and their ability to integrate asynchronous events effectively to scale them up to long event streams. We further augment these with novel event-centric techniques enabling our model to match or beat the state-of-the-art performance on several event stream benchmarks.
Ranked #1 on
Audio Classification
on SSC
SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning
This development is accompanied by a rapid growth of the required computational demands for larger models and more data.
Language Modeling on a SpiNNaker 2 Neuromorphic Chip
In this work, we demonstrate the first-ever implementation of a language model on a neuromorphic device - specifically the SpiNNaker 2 chip - based on a recently published event-based architecture called the EGRU.
Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference
Yet, sparse activations, while omnipresent in both biological neural networks and deep learning systems, have not been fully utilized as a compression technique in deep learning.
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
However, there is still a need to bridge the gap between what RNNs are capable of in terms of efficiency and performance and real-world application requirements.
Ranked #2 on
Gesture Recognition
on DVS128 Gesture
(using extra training data)
