**Pattern-Exploiting Training** is a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task. These phrases are then used to assign soft labels to a large set of unlabeled examples. Finally, standard supervised training is performed on the resulting training set. 

In the case of PET for sentiment classification, first a number of patterns encoding some form of task description are created to convert training examples to cloze questions; for each pattern, a pretrained language model is finetuned. Secondly, the ensemble of trained models annotates unlabeled data. Lastly, a classifier is trained on the resulting soft-labeled dataset.

The **Softmax** output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification.  Given an input vector $x$ and a weighting vector $w$ we have:

$$ P(y=j \mid{x}) = \frac{e^{x^{T}w_{j}}}{\sum^{K}_{k=1}e^{x^{T}wk}} $$

Softmax

Pattern-Exploiting Training

Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference

**Temporally Consistent Spatial Augmentation** is a video data augmentation technique used for contrastive learning in the [Contrastive Video Representation Learning](https://paperswithcode.com/method/cvrl) framework. It fixes the randomness of spatial augmentation across frames; this prevents spatial augmentation hurting learning if applied independently across frames, because in that case it breaks the natural motion. In contrast, having temporally consistent spatial augmentation does not break the natural motion in the frames.

Source	Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com