Viet-Anh on Software Logo

What is: Normalized Linear Combination of Activations?

SourceTrainable Activations for Image Classification
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

The Normalized Linear Combination of Activations, or NormLinComb, is a type of activation function that has trainable parameters and uses the normalized linear combination of other activation functions.

NormLinComb(x)=i=0nwiFi(x)WNormLinComb(x) = \frac{\sum\limits_{i=0}^{n} w_i \mathcal{F}_i(x)}{\mid \mid W \mid \mid}