**ExtremeNet** is a a bottom-up object detection framework that detects four extreme points (top-most, left-most, bottom-most, right-most) of an object. It uses a keypoint estimation framework to find extreme points, by predicting four multi-peak heatmaps for each object category. In addition, it uses one [heatmap](https://paperswithcode.com/method/heatmap) per category predicting the object center, as the average of two bounding box edges in both the x and y dimension. We group extreme points into objects with a purely geometry-based approach. We group four extreme points, one from each map, if and only if their
geometric center is predicted in the center heatmap with a score higher than a pre-defined threshold, We enumerate all $O\left(n^{4}\right)$ combinations of extreme point prediction, and select the valid ones.

In relation-aware global attention (RGA) stresses the importance of global structural information provided by pairwise relations, and uses it to produce attention maps. 

RGA comes in two forms,  spatial RGA (RGA-S) and channel RGA (RGA-C). RGA-S first reshapes the input feature map $X$ to $C\times (H\times W)$ and the pairwise relation matrix $R \in \mathbb{R}^{(H\times W)\times (H\times W)}$ is computed using 
\begin{align}
    Q &= \delta(W^QX) 
\end{align}
\begin{align}
    K &= \delta(W^KX) 
\end{align}
\begin{align}
    R &= Q^TK
\end{align}
The relation vector $r_i$ at position $i$ is defined by stacking  pairwise relations at all positions:
\begin{align}
    r_i = [R(i, :); R(:,i)]    
\end{align}
and the spatial relation-aware feature $y_i$ can be written as
\begin{align}
    Y_i = [g^c_\text{avg}(\delta(W^\varphi x_i)); \delta(W^\phi r_i)]
\end{align}
where $g^c_\text{avg}$ denotes global average pooling in the channel domain. Finally, the spatial attention score at position $i$ is given by 
\begin{align}
    a_i = \sigma(W_2\delta(W_1y_i))
\end{align}
RGA-C has the same form as RGA-S, except for taking the input feature map as a set of $H\times W$-dimensional features.

RGA uses global relations to generate the attention score for each feature node,  so provides valuable structural information and significantly enhances the representational power. RGA-S and RGA-C are flexible enough to be used in any CNN network; Zhang et al. propose  using them jointly in sequence to better capture both spatial and cross-channel relationships.

Relation-Aware Global Attention for Person Re-identification

ExtremeNet

Bottom-up Object Detection by Grouping Extreme and Center Points

**Wavelet** is an asynchronous data parallel approach that interleaves waves of training tasks on the same group of GPUs, such that tasks belong to one wave can leverage on-device memory from tasks in another wave during their memory valley period, thus boost-up the training throughput. As shown in the Figure, Wavelet divides dataparallel training tasks into two waves, namely tick-wave and tock-wave. The task launching offset is achieved by delaying the launch time of tock-wave tasks for half of a whole forward-backward training cycle. Therefore, the tock-wave tasks can directly leverage GPU memory valley period of tick-wave tasks (e.g. 0.4s-0.6s in Figure 2(a)), since backward propagation of tick-wave tasks is compute-heavy but memory is often unused. Similarly, tick-wave tasks can leverage memory valley period of tock-wave tasks in the same way.

Source	Bottom-up Object Detection by Grouping Extreme and Center Points
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com