Viet-Anh on Software Logo

What is: HardELiSH?

SourceThe Quest for the Golden Activation Function
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

HardELiSH is an activation function for neural networks. The HardELiSH is a multiplication of the HardSigmoid and ELU in the negative part and a multiplication of the Linear and the HardSigmoid in the positive part:

f(x)=xmax(0,min(1,(x+12))) if x1f\left(x\right) = x\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right) \right) \text{ if } x \geq 1 f(x)=(ex1)max(0,min(1,(x+12))) if x<0f\left(x\right) = \left(e^{x}-1\right)\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right)\right) \text{ if } x < 0

Source: Activation Functions