**GMVAE**, or **Gaussian Mixture Variational Autoencoder**, is a stochastic regularization layer for [transformers](https://paperswithcode.com/methods/category/transformers). A GMVAE layer is trained using a 700-dimensional internal representation of the first MLP layer. For every output from the first MLP layer, the GMVAE layer first computes a latent low-dimensional representation sampling from the GMVAE posterior distribution to then provide at the output a reconstruction sampled from a generative model.

FastGCN is a fast improvement of the GCN model recently proposed by Kipf & Welling (2016a) for learning graph embeddings. It generalizes transductive training to an inductive manner and also addresses the memory bottleneck issue of GCN caused by recursive expansion of neighborhoods. The crucial ingredient is a sampling scheme in the reformulation of the loss and the gradient, well justified through an alternative view of graph convoluntions in the form of integral transforms of embedding functions.

Description and image from: [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://arxiv.org/pdf/1801.10247.pdf)

FastGCN

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

GMVAE

Regularizing Transformers With Deep Probabilistic Layers

**ENet** is a semantic segmentation architecture which utilises a compact encoder-decoder architecture. Some design choices include:

1. Using the [SegNet](https://paperswithcode.com/method/segnet) approach to downsampling y saving indices of elements chosen in max
pooling layers, and using them to produce sparse upsampled maps in the decoder.
2.  Early downsampling to optimize the early stages of the network and reduce the cost of processing large input frames. The first two blocks of ENet heavily reduce the input size, and use only a small set of feature maps. 
3. Using PReLUs as an activation function
4. Using dilated convolutions 
5. Using Spatial [Dropout](https://paperswithcode.com/method/dropout)

Source	Regularizing Transformers With Deep Probabilistic Layers
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com