Viet-Anh on Software Logo

What is: Hardtanh Activation?

Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Hardtanh is an activation function used for neural networks:

f(x)=1 if x<1f\left(x\right) = -1 \text{ if } x < - 1 f(x)=x if 1x1f\left(x\right) = x \text{ if } -1 \leq x \leq 1 f(x)=1 if x>1f\left(x\right) = 1 \text{ if } x > 1

It is a cheaper and more computationally efficient version of the tanh activation.

Image Source: Zhuan Lan