**Rainbow DQN** is an extended [DQN](https://paperswithcode.com/method/dqn) that combines several improvements into a single learner. Specifically:

- It uses [Double Q-Learning](https://paperswithcode.com/method/double-q-learning) to tackle overestimation bias.
- It uses [Prioritized Experience Replay](https://paperswithcode.com/method/prioritized-experience-replay) to prioritize important transitions.
- It uses [dueling networks](https://paperswithcode.com/method/dueling-network).
- It uses [multi-step learning](https://paperswithcode.com/method/n-step-returns).
- It uses distributional reinforcement learning instead of the expected return.
- It uses noisy linear layers for exploration.

**Fastformer** is an type of [Transformer](https://paperswithcode.com/method/transformer) which uses [additive attention](https://www.paperswithcode.com/method/additive-attention) as a building block. Instead of modeling the pair-wise interactions between tokens, [additive attention](https://paperswithcode.com/method/additive-attention) is used to model global contexts, and then each token representation is further transformed based on its interaction with global context representations.

Fastformer

Fastformer: Additive Attention Can Be All You Need

Rainbow DQN

Rainbow: Combining Improvements in Deep Reinforcement Learning

In the field of scene segmentation,
encoder-decoder structures cannot make use of the global relationships 
between objects, whereas RNN-based structures 
heavily rely on the output of the long-term memorization.
To address the above problems, 
Fu et al. proposed a novel framework, 
 the dual attention network (DANet), 
for natural scene image segmentation. 
Unlike CBAM and BAM, it adopts a self-attention mechanism 
instead of simply stacking convolutions to compute the spatial attention map,
which enables the network to capture global information directly. 

DANet uses in parallel a position attention module and a channel attention module to capture feature dependencies in spatial and channel domains. Given the input feature map $X$, convolution layers are applied first in the position attention module to obtain new feature maps. Then the position attention module selectively aggregates the features at each position using a weighted sum of features at all positions, where the weights are determined by feature similarity between corresponding pairs of positions. The channel attention module has a similar form except for dimensional reduction to model cross-channel relations. Finally the outputs from the two branches are fused to obtain final feature representations. For simplicity, we reshape the feature map $X$ to $C\times (H \times W)$ whereupon the overall process can be written as 
\begin{align}
    Q,\quad K,\quad V &= W_qX,\quad W_kX,\quad W_vX
\end{align}
\begin{align}
    Y^\text{pos} &=  X+ V\text{Softmax}(Q^TK)
\end{align}
\begin{align}
    Y^\text{chn} &=  X+ \text{Softmax}(XX^T)X 
\end{align}
\begin{align}
    Y &= Y^\text{pos} + Y^\text{chn}
\end{align}
where $W_q$, $W_k$, $W_v \in \mathbb{R}^{C\times C}$ are used to generate new feature maps.   

The position attention module enables
DANet to capture long-range contextual information
and adaptively integrate similar features at any scale
from a global viewpoint,
while the channel attention module is responsible for 
enhancing useful channels 
as well as suppressing noise. 
Taking spatial and channel 
relationships into consideration explicitly
improves the feature representation for scene segmentation.
However, it is computationally costly, especially for large input feature maps.

Source	Rainbow: Combining Improvements in Deep Reinforcement Learning
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com