**Temporally Consistent Spatial Augmentation** is a video data augmentation technique used for contrastive learning in the [Contrastive Video Representation Learning](https://paperswithcode.com/method/cvrl) framework. It fixes the randomness of spatial augmentation across frames; this prevents spatial augmentation hurting learning if applied independently across frames, because in that case it breaks the natural motion. In contrast, having temporally consistent spatial augmentation does not break the natural motion in the frames.

**Pattern-Exploiting Training** is a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task. These phrases are then used to assign soft labels to a large set of unlabeled examples. Finally, standard supervised training is performed on the resulting training set. 

In the case of PET for sentiment classification, first a number of patterns encoding some form of task description are created to convert training examples to cloze questions; for each pattern, a pretrained language model is finetuned. Secondly, the ensemble of trained models annotates unlabeled data. Lastly, a classifier is trained on the resulting soft-labeled dataset.

Pattern-Exploiting Training

Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference

Temporally Consistent Spatial Augmentation

Spatiotemporal Contrastive Video Representation Learning

**DELG** is a convolutional neural network for image retrieval that combines generalized mean pooling for global features and attentive selection for local features. The entire network can be learned end-to-end by carefully balancing the gradient flow between two heads – requiring only image-level labels. This allows for efficient inference by extracting an image’s global feature, detected keypoints and local descriptors within a single model.

The model is enabled by leveraging hierarchical image representations that arise in [CNNs](https://paperswithcode.com/methods/category/convolutional-neural-networks), which are coupled to [generalized mean pooling](https://paperswithcode.com/method/generalized-mean-pooling) and attentive local feature detection. Secondly, a convolutional autoencoder module is adopted that can successfully learn low-dimensional local descriptors. This can be readily integrated into the unified model, and avoids the need of post-processing learning steps, such as [PCA](https://paperswithcode.com/method/pca), that are commonly used. Finally, a procedure is used that enables end-to-end training of the proposed model using only image-level supervision. This requires carefully controlling the gradient flow between the global and local network heads during backpropagation, to avoid disrupting the desired representations.

Source	Spatiotemporal Contrastive Video Representation Learning
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com