**ControlVAE** is a [variational autoencoder](https://paperswithcode.com/method/vae) (VAE) framework that combines the automatic control theory with the basic VAE to stabilize the KL-divergence of VAE models to a specified value. It leverages a non-linear PI controller, a variant of the proportional-integral-derivative (PID) control, to dynamically tune the weight of the KL-divergence term in the evidence lower bound (ELBO) using the output KL-divergence as feedback. This allows for control of the KL-divergence to a desired value (set point), which is effective in avoiding posterior collapse and learning disentangled representations.

**Gradient Normalization** is a normalization method for [Generative Adversarial Networks](https://paperswithcode.com/methods/category/generative-adversarial-networks) to tackle the training instability of generative adversarial networks caused by the sharp gradient space. Unlike existing work such as [gradient penalty](https://paperswithcode.com/method/wgan-gp-loss) and [spectral normalization](https://paperswithcode.com/method/spectral-normalization), the proposed GN only imposes a hard 1-Lipschitz constraint on the discriminator function, which increases the capacity of the network.

Gradient Normalization

Gradient Normalization for Generative Adversarial Networks

ControlVAE

ControlVAE: Tuning, Analytical Properties, and Performance Analysis

**Mish** is an activation function for neural networks which can be defined as:

$$ f\left(x\right) = x\cdot\tanh{\text{softplus}\left(x\right)}$$

where

$$\text{softplus}\left(x\right) = \ln\left(1+e^{x}\right)$$

(Compare with functionally similar previously proposed activation functions such as the [GELU](https://paperswithcode.com/method/silu) $x\Phi(x)$ and the [SiLU](https://paperswithcode.com/method/silu) $x\sigma(x)$.)

Source	ControlVAE: Tuning, Analytical Properties, and Performance Analysis
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com