Viet-Anh on Software Logo

What is: Attentive Normalization?

SourceAttentive Normalization
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Attentive Normalization generalizes the common affine transformation component in the vanilla feature normalization. Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging feature attention.