Activation Functions

Leaky ReLU

Leaky Rectified Linear Unit, or Leaky ReLU, is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient is determined before training, i.e. it is not learnt during training. This type of activation function is popular in tasks where we may suffer from sparse gradients, for example training generative adversarial networks.

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Image Generation 87 12.13%
Image-to-Image Translation 43 6.00%
Translation 37 5.16%
Disentanglement 26 3.63%
Face Generation 20 2.79%
Super-Resolution 19 2.65%
Style Transfer 18 2.51%
Image Manipulation 16 2.23%
Face Swapping 15 2.09%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories