The Triplet Entropy Loss (TEL) training method aims to leverage both the strengths of Cross Entropy Loss (CEL) and [Triplet loss](https://paperswithcode.com/method/triplet-loss) during the training process, assuming that it would lead to better generalization. The TEL method though does not contain a pre-training step, but trains simultaneously with both CEL and Triplet losses.

The Global Local Attention Module (GLAM) is an image model block that attends to the feature map's channels and spatial dimensions locally, and also attends to the feature map's channels and spatial dimensions globally. The locally attended feature maps, globally attended feature maps, and the original feature maps are then fused through a weighted sum (with learnable weights) to obtain the final feature map.

Paper:

Song, C. H., Han, H. J., & Avrithis, Y. (2022). All the attention you need: Global-local, spatial-channel attention for image retrieval. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2754-2763).

Global Local Attention Module

All the attention you need: Global-local, spatial-channel attention for image retrieval

Triplet Entropy Loss

Triplet Entropy Loss: Improving The Generalisation of Short Speech Language Identification Systems

**PULSE** is a self-supervised photo upsampling algorithm. Instead of starting with the LR image and slowly adding detail, PULSE traverses the high-resolution natural image manifold, searching for images that downscale to the original LR image. This is formalized through the downscaling loss, which guides exploration through the latent space of a generative model. By leveraging properties of high-dimensional Gaussians, the authors aim to restrict the search space to guarantee realistic outputs.

Source	Triplet Entropy Loss: Improving The Generalisation of Short Speech Language Identification Systems
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com