Viet-Anh on Software Logo

What is: Sigmoid Activation?

Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Sigmoid Activations are a type of activation function for neural networks:

f(x)=1(1+exp(x))f\left(x\right) = \frac{1}{\left(1+\exp\left(-x\right)\right)}

Some drawbacks of this activation that have been noted in the literature are: sharp damp gradients during backpropagation from deeper hidden layers to inputs, gradient saturation, and slow convergence.