Activation Functions

Sigmoid Activation

Sigmoid Activations are a type of activation function for neural networks:

$$f\left(x\right) = \frac{1}{\left(1+\exp\left(-x\right)\right)}$$

Some drawbacks of this activation that have been noted in the literature are: sharp damp gradients during backpropagation from deeper hidden layers to inputs, gradient saturation, and slow convergence.

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Language Modelling 21 3.01%
Classification 19 2.72%
Sentence 18 2.58%
Image Classification 16 2.29%
Image Generation 16 2.29%
Image-to-Image Translation 16 2.29%
Management 15 2.15%
Decision Making 14 2.01%
Translation 14 2.01%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories