**RepPoints** is a representation for object detection that consists of a set of points which indicate the spatial extent of an object and semantically significant local areas. This representation is learned via weak localization supervision from rectangular ground-truth boxes and implicit recognition feedback. Based on the richer RepPoints representation, the authors develop an anchor-free object detector that yields improved performance compared to using bounding boxes.

**Multi-head Attention** is a module for attention mechanisms which runs through an attention mechanism several times in parallel. The independent attention outputs are then concatenated and linearly transformed into the expected dimension. Intuitively, multiple attention heads allows for attending to parts of the sequence differently (e.g. longer-term dependencies versus shorter-term dependencies). 

$$ \text{MultiHead}\left(\textbf{Q}, \textbf{K}, \textbf{V}\right) = \left[\text{head}\_{1},\dots,\text{head}\_{h}\right]\textbf{W}_{0}$$

$$\text{where} \text{ head}\_{i} = \text{Attention} \left(\textbf{Q}\textbf{W}\_{i}^{Q}, \textbf{K}\textbf{W}\_{i}^{K}, \textbf{V}\textbf{W}\_{i}^{V} \right) $$

Above $\textbf{W}$ are all learnable parameter matrices.

Note that [scaled dot-product attention](https://paperswithcode.com/method/scaled) is most commonly used in this module, although in principle it can be swapped out for other types of attention mechanism.

Source: [Lilian Weng](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html#a-family-of-attention-mechanisms)

Multi-Head Attention

Attention Is All You Need

RepPoints

RepPoints: Point Set Representation for Object Detection

**Generalized Mean Pooling (GeM)** computes the generalized mean of each channel in a tensor. Formally:

$$ \textbf{e} = \left[\left(\frac{1}{|\Omega|}\sum\_{u\in{\Omega}}x^{p}\_{cu}\right)^{\frac{1}{p}}\right]\_{c=1,\cdots,C} $$

where $p > 0$ is a parameter. Setting this exponent as $p > 1$ increases the contrast of the pooled feature map and focuses on the salient features of the image. GeM is a generalization of the [average pooling](https://paperswithcode.com/method/average-pooling) commonly used in classification networks ($p = 1$) and of spatial max-pooling layer ($p = \infty$).

Source: [MultiGrain](https://paperswithcode.com/method/multigrain)

Image Source: [Eva Mohedano](https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.slideshare.net%2Fxavigiro%2Fd1l5-contentbased-image-retrieval-upc-2018-deep-learning-for-computer-vision&psig=AOvVaw2-9Hx23FNGFDe4GHU22Oo5&ust=1591798200590000&source=images&cd=vfe&ved=0CA0QjhxqFwoTCOiP-9P09OkCFQAAAAAdAAAAABAD)

Source	RepPoints: Point Set Representation for Object Detection
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com