**Routed Attention** is an attention pattern proposed as part of the [Routing Transformer](https://paperswithcode.com/method/routing-transformer) architecture.  Each attention module
considers a clustering of the space: the current timestep only attends to context belonging to the same cluster. In other word, the current time-step query is routed to a limited number of context through its cluster assignment. This can be contrasted with [strided](https://paperswithcode.com/method/strided-attention) attention patterns and those proposed with the [Sparse Transformer](https://paperswithcode.com/method/sparse-transformer).

In the image to the right, the rows represent the outputs while the columns represent the inputs. The different colors represent cluster memberships for the output token.

**Content-Aware ReAssembly of FEatures (CARAFE)** is an operator for feature upsampling in convolutional neural networks. CARAFE has several appealing properties: (1) Large field of view. Unlike previous works (e.g. bilinear interpolation) that only exploit subpixel neighborhood, CARAFE can aggregate contextual information within a large receptive field. (2) Content-aware handling. Instead of using a fixed kernel for all samples (e.g. deconvolution), CARAFE enables instance-specific content-aware handling, which generates adaptive kernels on-the-fly. (3) Lightweight and fast to compute.

CARAFE

CARAFE: Content-Aware ReAssembly of FEatures

Routing Attention

Efficient Content-Based Sparse Attention with Routing Transformers

**AlphaZero** is a reinforcement learning agent for playing board games such as Go, chess, and shogi. 

Source	Efficient Content-Based Sparse Attention with Routing Transformers
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com