Viet-Anh on Software Logo

What is: Peer-attention?

SourceAssembleNet++: Assembling Modality Representations via Attention Connections
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Peer-attention is a network component which dynamically learns the attention weights using another block or input modality. This is unlike AssembleNet which partially relies on exponential mutations to explore connections. Once the attention weights are found, we can either prune the connections by only leaving the argmax over hh or leave them with softmax.