**CornerNet-Squeeze Hourglass Module** is an image model block used in [CornerNet](https://paperswithcode.com/method/cornernet)-Lite that is based on an [hourglass module](https://paperswithcode.com/method/hourglass-module), but uses modified fire modules instead of residual blocks. Other than replacing the residual blocks, further modifications include: reducing the maximum feature map resolution of the hourglass modules by adding one more downsampling layer before the hourglass modules, removing one downsampling layer in each hourglass module, replacing the 3 × 3 filters with 1 x 1 filters in the prediction modules of CornerNet, and finally replacing the nearest neighbor upsampling in the hourglass network with transpose [convolution](https://paperswithcode.com/method/convolution) with a 4 × 4 kernel.

**AdaDelta** is a stochastic optimization technique that allows for per-dimension learning rate method for [SGD](https://paperswithcode.com/method/sgd). It is an extension of [Adagrad](https://paperswithcode.com/method/adagrad) that seeks to reduce its aggressive, monotonically decreasing learning rate. Instead of accumulating all past squared gradients, Adadelta restricts the window of accumulated past gradients to a fixed size $w$.

Instead of inefficiently storing $w$ previous squared gradients, the sum of gradients is recursively defined as a decaying average of all past squared gradients. The running average $E\left[g^{2}\right]\_{t}$ at time step $t$ then depends only on the previous average and current gradient:

$$E\left[g^{2}\right]\_{t} = \gamma{E}\left[g^{2}\right]\_{t-1} + \left(1-\gamma\right)g^{2}\_{t}$$

Usually $\gamma$ is set to around $0.9$. Rewriting SGD updates in terms of the parameter update vector:

$$ \Delta\theta_{t} = -\eta\cdot{g\_{t, i}}$$
$$\theta\_{t+1}  = \theta\_{t} + \Delta\theta_{t}$$

AdaDelta takes the form:

$$ \Delta\theta_{t} = -\frac{\eta}{\sqrt{E\left[g^{2}\right]\_{t} + \epsilon}}g_{t} $$

The main advantage of AdaDelta is that we do not need to set a default learning rate.

AdaDelta

ADADELTA: An Adaptive Learning Rate Method

CornerNet-Squeeze Hourglass Module

CornerNet-Lite: Efficient Keypoint Based Object Detection

OASIS is a [GAN](https://paperswithcode.com/method/gan)-based model to translate semantic label maps into realistic-looking images. The model builds on preceding work such as [Pix2Pix](https://paperswithcode.com/method/pix2pix) and SPADE. OASIS introduces the following innovations:  

1. The method is not dependent on the perceptual loss, which is commonly used for the semantic image synthesis task. A [VGG](https://paperswithcode.com/method/vgg) network trained on ImageNet is routinely employed as the perceptual loss to strongly improve the synthesis quality. The authors show that this perceptual loss also has negative effects: First, it reduces the diversity of the generated images. Second, it negatively influences the color distribution to be more biased towards ImageNet. OASIS eliminates the dependence on the perceptual loss by changing the common discriminator design: The OASIS discriminator segments an image into one of the real classes or an additional fake class. In doing so, it makes more efficient use of the label maps that the discriminator normally receives. This distinguishes the discriminator from the commonly used encoder-shaped discriminators, which concatenate the label maps to the input image and predict a single score per image. With the more fine-grained supervision through the loss of the OASIS discriminator, the perceptual loss is shown to become unnecessary.

2. A user can generate a diverse set of images per label map by simply resampling noise. This is achieved by conditioning the [spatially-adaptive denormalization](https://arxiv.org/abs/1903.07291) module in each layer of the GAN generator directly on spatially replicated input noise. A side effect of this conditioning is that at inference time an image can be resampled either globally or locally (either the complete image changes or a restricted region in the image).

Source	CornerNet-Lite: Efficient Keypoint Based Object Detection
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com