**Gaussian Affinity** is a type of affinity or self-similarity function between two points $\mathbb{x\_{i}}$ and $\mathbb{x\_{j}}$ that uses a Gaussian function:

$$ f\left(\mathbb{x\_{i}}, \mathbb{x\_{j}}\right) = e^{\mathbb{x^{T}\_{i}}\mathbb{x\_{j}}} $$

Here $\mathbb{x^{T}\_{i}}\mathbb{x\_{j}}$ is dot-product similarity.

A **Sparse Transformer** is a [Transformer](https://paperswithcode.com/method/transformer) based architecture which utilises sparse factorizations of the attention matrix to reduce time/memory to $O(n \sqrt{n})$. Other changes to the Transformer architecture include: (a) a restructured [residual block](https://paperswithcode.com/method/residual-block) and weight initialization, (b) A set of sparse attention kernels which efficiently compute subsets of the attention matrix, (c) recomputation of attention weights during the backwards pass to reduce memory usage

Sparse Transformer

Generating Long Sequences with Sparse Transformers

Gaussian Affinity

**RealNVP** is a generative model that utilises real-valued non-volume preserving (real NVP) transformations for density estimation. The model can perform efficient and exact inference, sampling and log-density estimation of data points.

Year	2005
Data Source	CC BY-SA - https://paperswithcode.com