What is: Attentive Normalization?
Source | Attentive Normalization |
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
Attentive Normalization generalizes the common affine transformation component in the vanilla feature normalization. Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging feature attention.