Slow Momentum (SlowMo) is a distributed optimization method where workers periodically synchronize and perform a momentum update, after multiple iterations of a base optimization algorithm. Periodically, after taking some number $\tau$ of base algorithm steps, workers average their parameters using ALLREDUCE and perform a momentum update.
Source: SlowMo: Improving Communication-Efficient Distributed SGD with Slow MomentumPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Blocking | 1 | 25.00% |
Image Classification | 1 | 25.00% |
Machine Translation | 1 | 25.00% |
Translation | 1 | 25.00% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |