The Global Local Attention Module (GLAM) is an image model block that attends to the feature map's channels and spatial dimensions locally, and also attends to the feature map's channels and spatial dimensions globally. The locally attended feature maps, globally attended feature maps, and the original feature maps are then fused through a weighted sum (with learnable weights) to obtain the final feature map.

Paper:

Song, C. H., Han, H. J., & Avrithis, Y. (2022). All the attention you need: Global-local, spatial-channel attention for image retrieval. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2754-2763).

**Contrastive Video Representation Learning**, or **CVRL**, is a self-supervised contrastive learning framework for learning spatiotemporal visual representations from unlabeled videos. Representations are learned using a contrastive loss, where two clips from the same short video are pulled together in the embedding space, while clips from different videos are pushed away. Data augmentations are designed involving spatial and temporal cues. Concretely, a [temporally consistent spatial augmentation](https://paperswithcode.com/method/temporally-consistent-spatial-augmentation#) method is used to impose strong spatial augmentations on each frame of the video while maintaining the temporal consistency across frames. A sampling-based temporal augmentation method is also used to avoid overly enforcing invariance on clips that are distant in time. 

End-to-end, from a raw video, we first sample a temporal interval from a monotonically decreasing distribution. The temporal interval represents the number of frames between the start points of two clips, and we sample two clips from a video according to this interval. Afterwards we apply a [temporally consistent spatial augmentation](https://paperswithcode.com/method/temporally-consistent-spatial-augmentation) to each of the clips and feed them into a 3D backbone with an MLP head. The contrastive loss is used to train the network to attract the clips from the same video and repel the clips from different videos in the embedding space.

CVRL

Spatiotemporal Contrastive Video Representation Learning

Global Local Attention Module

All the attention you need: Global-local, spatial-channel attention for image retrieval

The Triplet Entropy Loss (TEL) training method aims to leverage both the strengths of Cross Entropy Loss (CEL) and [Triplet loss](https://paperswithcode.com/method/triplet-loss) during the training process, assuming that it would lead to better generalization. The TEL method though does not contain a pre-training step, but trains simultaneously with both CEL and Triplet losses.

Source	All the attention you need: Global-local, spatial-channel attention for image retrieval
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com