**PULSE** is a self-supervised photo upsampling algorithm. Instead of starting with the LR image and slowly adding detail, PULSE traverses the high-resolution natural image manifold, searching for images that downscale to the original LR image. This is formalized through the downscaling loss, which guides exploration through the latent space of a generative model. By leveraging properties of high-dimensional Gaussians, the authors aim to restrict the search space to guarantee realistic outputs.

The Triplet Entropy Loss (TEL) training method aims to leverage both the strengths of Cross Entropy Loss (CEL) and [Triplet loss](https://paperswithcode.com/method/triplet-loss) during the training process, assuming that it would lead to better generalization. The TEL method though does not contain a pre-training step, but trains simultaneously with both CEL and Triplet losses.

Triplet Entropy Loss

Triplet Entropy Loss: Improving The Generalisation of Short Speech Language Identification Systems

PULSE

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

Spatial pooling usually operates on a small region which limits its capability to capture long-range dependencies and focus on distant regions. To overcome this, Hou et al. proposed  strip pooling, a novel pooling method capable of encoding long-range context in either horizontal or vertical spatial domains.  

Strip pooling has two branches for horizontal  and vertical strip pooling. The horizontal strip pooling part first pools the input feature $F \in \mathcal{R}^{C \times H \times W}$ in the horizontal direction:
\begin{align}
y^1 = \text{GAP}^w (X) 
\end{align}
Then a 1D convolution with kernel size 3 is applied in $y$ to capture the relationship between different rows and channels. This is repeated $W$ times to make  the output $y_v$  consistent with the input shape:
\begin{align}
    y_h = \text{Expand}(\text{Conv1D}(y^1))
\end{align}
Vertical strip pooling is performed in a similar way. Finally, the outputs of the two branches are fused using element-wise summation to produce the attention map:
\begin{align}
s &= \sigma(Conv^{1\times 1}(y_{v} + y_{h}))
\end{align}
\begin{align}
Y &= s  X
\end{align}

The strip pooling module (SPM) is further developed in the mixed pooling module (MPM). Both consider  spatial  and channel relationships to overcome the locality of convolutional neural networks.  SPNet achieves  state-of-the-art results for several complex semantic segmentation benchmarks.

Source	PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com