Viet-Anh on Software Logo

What is: Fastformer?

SourceFastformer: Additive Attention Can Be All You Need
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Fastformer is an type of Transformer which uses additive attention as a building block. Instead of modeling the pair-wise interactions between tokens, additive attention is used to model global contexts, and then each token representation is further transformed based on its interaction with global context representations.