A **Depthwise Dilated Separable Convolution** is a type of [convolution](https://paperswithcode.com/method/convolution) that combines [depthwise separability](https://paperswithcode.com/method/depthwise-separable-convolution) with the use of [dilated convolutions](https://paperswithcode.com/method/dilated-convolution).

The **Adaptive Locally Connected Neuron (ALCN)** is a topology aware, and locally adaptive -focusing neuron:

$$a =  f\:\Bigg( \sum_{i=1}^{m} w_{i}\phi\left( \tau\left(i\right),\Theta\right) x_{i} + b \Bigg) %f\:\Bigg(\mathbf{X(W \circ \Phi) + b} \Bigg) $$

ALCN

An Adaptive Locally Connected Neuron Model: Focusing Neuron

Depthwise Dilated Separable Convolution

ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network

**Demon Adam** is a stochastic optimizer where the [Demon](https://paperswithcode.com/method/demon) momentum rule is applied to the [Adam](https://paperswithcode.com/method/adam) optimizer.

$$ \beta\_{t} = \beta\_{init}\cdot\frac{\left(1-\frac{t}{T}\right)}{\left(1-\beta\_{init}\right) + \beta\_{init}\left(1-\frac{t}{T}\right)} $$

$$ m\_{t, i} = g\_{t, i} + \beta\_{t}m\_{t-1, i} $$

$$ v\_{t+1} = \beta\_{2}v\_{t}  + \left(1-\beta\_{2}\right)g^{2}\_{t} $$

$$ \theta_{t} = \theta_{t-1} - \eta\frac{\hat{m}\_{t}}{\sqrt{\hat{v}\_{t}} + \epsilon}  $$

Source	ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com