Viet-Anh on Software Logo

What is: Self-Attention Network?

SourceExploring Self-attention for Image Recognition
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Self-Attention Network (SANet) proposes two variations of self-attention used for image recognition: 1) pairwise self-attention which generalizes standard dot-product attention and is fundamentally a set operator, and 2) patchwise self-attention which is strictly more powerful than convolution.