**DropBlock** is a structured form of [dropout](https://paperswithcode.com/method/dropout) directed at regularizing convolutional networks. In DropBlock, units in a contiguous region of a feature map are dropped together.  As DropBlock discards features in a correlated area, the networks must look elsewhere for evidence to fit the data.

Training a denoiser on signals gives you a powerful prior over this signal that you can then use to sample examples of this signal.

Denoising Score Matching

Generative Modeling by Estimating Gradients of the Data Distribution

DropBlock

DropBlock: A regularization method for convolutional networks

A **Projection Discriminator** is a type of discriminator for generative adversarial networks. It is motivated by a probabilistic model in which the distribution of the conditional variable $\textbf{y}$ given $\textbf{x}$ is discrete or uni-modal continuous distributions.

If we look at the original solution for the loss function $\mathcal{L}\_{D}$ in the vanilla GANs, we can decompose it into the sum of two log-likelihood ratios:

$$ f^{*}\left(\mathbf{x}, \mathbf{y}\right) = \log\frac{q\left(\mathbf{x}\mid{\mathbf{y}}\right)q\left(\mathbf{y}\right)}{p\left(\mathbf{x}\mid{\mathbf{y}}\right)p\left(\mathbf{y}\right)} = \log\frac{q\left(\mathbf{y}\mid{\mathbf{x}}\right)}{p\left(\mathbf{y}\mid{\mathbf{x}}\right)} + \log\frac{q\left(\mathbf{x}\right)}{p\left(\mathbf{x}\right)}  = r\left(\mathbf{y\mid{x}}\right) + r\left(\mathbf{x}\right) $$

We can model the log likelihood ratio $r\left(\mathbf{y\mid{x}}\right)$ and  $r\left(\mathbf{x}\right)$ by some parametric functions $f\_{1}$ and $f\_{2}$ respectively. If we make a standing assumption that $p\left(y\mid{x}\right)$ and $q\left(y\mid{x}\right)$ are simple distributions like those that are Gaussian or discrete log linear on the feature space, then the parametrization of the following form becomes natural:

$$ f\left(\mathbf{x}, \mathbf{y}; \theta\right) = f\_{1}\left(\mathbf{x}, \mathbf{y}; \theta\right) + f\_{2}\left(\mathbf{x}; \theta\right) = \mathbf{y}^{T}V\phi\left(\mathbf{x}; \theta\_{\phi}\right) + \psi\left(\phi(\mathbf{x}; \theta\_{\phi}); \theta\_{\psi}\right) $$

where $V$ is the embedding matrix of $y$, $\phi\left(·, \theta\_{\phi}\right)$ is a vector output function of $x$, and $\psi\left(·, \theta\_{\psi}\right)$ is a scalar function of the same $\phi\left(\mathbf{x}; \theta\_{\phi}\right)$ that appears in $f\_{1}$. The learned parameters $\theta = ${$V, \theta\_{\phi}, \theta\_{\psi}$} are trained to optimize the adversarial loss. This model of the discriminator is the projection.

Source	DropBlock: A regularization method for convolutional networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com