**IoU-Balanced Sampling** is hard mining method for object detection. Suppose we need to sample $N$ negative samples from $M$ corresponding candidates. The selected probability for each sample under random sampling is:

$$ p = \frac{N}{M} $$

To raise the selected probability of hard negatives, we evenly split the sampling interval into $K$ bins according to IoU. $N$ demanded negative samples are equally distributed to each bin. Then we select samples from them uniformly. Therefore, we get the selected probability under IoU-balanced sampling:

$$ p\_{k} = \frac{N}{K}*\frac{1}{M\_{k}}\text{ , } k\in\left[0, K\right)$$

where $M\_{k}$ is the number of sampling candidates in the corresponding interval denoted by $k$. $K$ is set to 3 by default in our experiments.

The sampled histogram with IoU-balanced sampling is shown by green color in the Figure to the right. The IoU-balanced sampling can guide the distribution of training samples close to the one of hard negatives.

**AdaSqrt** is a stochastic optimization technique that is motivated by the observation that methods like [Adagrad](https://paperswithcode.com/method/adagrad) and [Adam](https://paperswithcode.com/method/adam) can be viewed as relaxations of [Natural Gradient Descent](https://paperswithcode.com/method/natural-gradient-descent).

The updates are performed as follows:

$$ t \leftarrow t + 1 $$

$$ \alpha\_{t} \leftarrow \sqrt{t} $$

$$ g\_{t} \leftarrow \nabla\_{\theta}f\left(\theta\_{t-1}\right) $$

$$ S\_{t} \leftarrow S\_{t-1} + g\_{t}^{2} $$

$$ \theta\_{t+1} \leftarrow \theta\_{t} + \eta\frac{\alpha\_{t}g\_{t}}{S\_{t} + \epsilon} $$

AdaSqrt

Second-order Information in First-order Optimization Methods

IoU-Balanced Sampling

Libra R-CNN: Towards Balanced Learning for Object Detection

The **NVAE Generative Residual Cell** is a skip connection block used as part of the [NVAE](https://paperswithcode.com/method/nvae) architecture for the generator. The residual cell expands the number of channels $E$ times before applying the [depthwise separable convolution](https://paperswithcode.com/method/depthwise-separable-convolution), and then maps it back to $C$ channels. The design motivation was to help model long-range correlations in the data by increasing the receptive field of the network, which explains the expanding path but also the use of depthwise convolutions to keep a handle on parameter count.

Source	Libra R-CNN: Towards Balanced Learning for Object Detection
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com