**PAUSE**, or **Positive and Annealed Unlabeled Sentence Embedding**, is an approach for learning sentence embeddings from a partially labeled dataset. It is based on a dual encoder schema that is widely adopted in supervised sentence embedding training. Each individual sample $\mathbf{x}$ contains a pair of hypothesis and premise sentences $(x\_{i},x^{\prime}_{i})$, each of which is fed into a pretrained encoder (e.g. [BERT](https://paperswithcode.com/method/bert)). As shown in Figure, the two encoders are identical during the training by sharing their weights.

**Variational Dropout** is a regularization technique based on [dropout](https://paperswithcode.com/method/dropout), but uses a variational inference grounded approach. In Variational Dropout, we repeat the same dropout mask at each time step for both inputs, outputs, and recurrent layers (drop the same network units at each time step). This is in contrast to ordinary Dropout where different dropout masks are sampled at each time step for the inputs and outputs alone.

Variational Dropout

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

PAUSE

PAUSE: Positive and Annealed Unlabeled Sentence Embedding

**FLAVR** is an architecture for video frame interpolation. It uses 3D space-time convolutions to enable end-to-end learning and inference for video frame interpolation. Overall, it consists of a [U-Net](https://paperswithcode.com/method/u-net) style architecture with 3D space-time convolutions and
deconvolutions (yellow blocks). Channel gating is used after all (de-)[convolution](https://paperswithcode.com/method/convolution) layers (blue blocks). The final prediction layer (the purple block) is implemented as a convolution layer to project the 3D feature maps into $(k−1)$ frame predictions. This design allows FLAVR to predict multiple frames in one inference forward pass.

Source	PAUSE: Positive and Annealed Unlabeled Sentence Embedding
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com