**Strip Pooling** is a pooling strategy for scene parsing which considers a long but narrow kernel, i.e., $1\times{N}$ or $N\times{1}$. As an alternative to global pooling, strip pooling offers two advantages. First, it deploys a long kernel shape along one spatial dimension and hence enables capturing long-range relations of isolated regions. Second, it keeps a narrow kernel shape along the other spatial dimension, which facilitates capturing local context and prevents irrelevant regions from interfering the label prediction. Integrating such long but narrow pooling kernels enables the scene parsing networks to simultaneously aggregate both global and local context. This is essentially different from the traditional spatial pooling which collects context from a fixed square region.

A **DQN**, or Deep Q-Network, approximates a state-value function in a [Q-Learning](https://paperswithcode.com/method/q-learning) framework with a neural network. In the Atari Games case, they take in several frames of the game as an input and output state values for each action as an output. 

It is usually used in conjunction with [Experience Replay](https://paperswithcode.com/method/experience-replay), for storing the episode steps in memory for off-policy learning, where samples are drawn from the replay memory at random. Additionally, the Q-Network is usually optimized towards a frozen target network that is periodically updated with the latest weights every $k$ steps (where $k$ is a hyperparameter). The latter makes training more stable by preventing short-term oscillations from a moving target. The former tackles autocorrelation that would occur from on-line learning, and having a replay memory makes the problem more like a supervised learning problem.

Image Source: [here](https://www.researchgate.net/publication/319643003_Autonomous_Quadrotor_Landing_using_Deep_Reinforcement_Learning)

Playing Atari with Deep Reinforcement Learning

Strip Pooling

Strip Pooling: Rethinking Spatial Pooling for Scene Parsing

**AdaSmooth** is a stochastic optimization technique that allows for per-dimension learning rate method for [SGD](https://paperswithcode.com/method/sgd). It is an extension of [Adagrad](https://paperswithcode.com/method/adagrad) and [AdaDelta](https://paperswithcode.com/method/adadelta) that seek to reduce its aggressive, monotonically decreasing learning rate. Instead of accumulating all past squared gradients, Adadelta restricts the window of accumulated past gradients to a fixed size $w$ while AdaSmooth adaptively selects the size of the window.

Given the window size  $M$, the effective ratio is calculated by 

$$e_t  = \frac{s_t}{n_t}= \frac{| x_t -  x_{t-M}|}{\sum_{i=0}^{M-1} | x_{t-i} -  x_{t-1-i}|}\\
= \frac{| \sum_{i=0}^{M-1} \Delta x_{t-1-i}|}{\sum_{i=0}^{M-1} | \Delta x_{t-1-i}|}.$$

Given the effective ratio, the scaled smoothing constant is obtained by:

$$c_t =  ( \rho_2- \rho_1) \times e_t   + (1-\rho_2),$$

The running average $E\left[g^{2}\right]\_{t}$ at time step $t$ then depends only on the previous average and current gradient:

$$ E\left[g^{2}\right]\_{t} = c_t^2 \odot g_{t}^2  +  \left(1-c_t^2 \right)\odot E[g^2]_{t-1} $$

Usually $\rho_1$ is set to around $0.5$ and $\rho_2$ is set to around 0.99. The update step the follows:

$$ \Delta x_t = -\frac{\eta}{\sqrt{E\left[g^{2}\right]\_{t} + \epsilon}} \odot  g_{t}, $$

which is incorporated into the final update:

$$x_{t+1} = x_{t} + \Delta x_t.$$

The main advantage of AdaSmooth is its faster convergence rate and insensitivity to hyperparameters.

Source	Strip Pooling: Rethinking Spatial Pooling for Scene Parsing
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com