**DCN-V2** is an architecture for learning-to-rank that improves upon the original [DCN](http://paperswithcode.com/method/dcn) model. It first learns explicit feature interactions of the inputs (typically the embedding layer) through cross layers, and then combines with a deep network to learn complementary implicit interactions. The core of DCN-V2 is the cross layers, which inherit the simple structure of the cross network from DCN, however it is significantly more expressive at learning explicit and bounded-degree cross features.

**PP-YOLO** is an object detector based on [YOLOv3](https://paperswithcode.com/method/yolov3). It mainly tries to combine various existing tricks that almost not increase the number of model parameters and FLOPs, to achieve the goal of improving the accuracy of detector as much as possible while ensuring that the speed is almost unchanged. Some of these changes include:

- Changing the [DarkNet-53](https://paperswithcode.com/method/darknet-53) backbone with ResNet50-vd. Some of the convolutional layers in ResNet50-vd are also replaced with [deformable convolutional layers](https://paperswithcode.com/method/deformable-convolution).
- A larger batch size is used - changing from 64 to 192.
- An exponentially moving average is used for the parameters.
- [DropBlock](https://paperswithcode.com/method/dropblock) is applied to the [FPN](https://paperswithcode.com/method/fpn).
- An IoU loss is used.
- An IoU prediction branch is added to measure the accuracy of localization.
- [Grid Sensitive](https://paperswithcode.com/method/grid-sensitive) is used, similar to [YOLOv4](https://paperswithcode.com/method/yolov4).
- [Matrix NMS](https://paperswithcode.com/method/matrix-nms) is used.
- [CoordConv](https://paperswithcode.com/method/coordconv) is used for the [FPN](https://paperswithcode.com/method/fpn), replacing the 1x1 convolution layer, and also the first convolution layer in the detection head.
- [Spatial Pyramid Pooling](https://paperswithcode.com/method/spatial-pyramid-pooling) is used for the top feature map.

PP-YOLO

PP-YOLO: An Effective and Efficient Implementation of Object Detector

DCN-V2

DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems

**Multi-head Attention** is a module for attention mechanisms which runs through an attention mechanism several times in parallel. The independent attention outputs are then concatenated and linearly transformed into the expected dimension. Intuitively, multiple attention heads allows for attending to parts of the sequence differently (e.g. longer-term dependencies versus shorter-term dependencies). 

$$ \text{MultiHead}\left(\textbf{Q}, \textbf{K}, \textbf{V}\right) = \left[\text{head}\_{1},\dots,\text{head}\_{h}\right]\textbf{W}_{0}$$

$$\text{where} \text{ head}\_{i} = \text{Attention} \left(\textbf{Q}\textbf{W}\_{i}^{Q}, \textbf{K}\textbf{W}\_{i}^{K}, \textbf{V}\textbf{W}\_{i}^{V} \right) $$

Above $\textbf{W}$ are all learnable parameter matrices.

Note that [scaled dot-product attention](https://paperswithcode.com/method/scaled) is most commonly used in this module, although in principle it can be swapped out for other types of attention mechanism.

Source: [Lilian Weng](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html#a-family-of-attention-mechanisms)

Source	DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com