**Sparsemax** is a type of activation/output function similar to the traditional [softmax](https://paperswithcode.com/method/softmax), but able to output sparse probabilities. 

$$ \text{sparsemax}\left(z\right) = \arg\_{p∈\Delta^{K−1}}\min||\mathbf{p} - \mathbf{z}||^{2} $$

**Pretext-Invariant Representation Learning (PIRL, pronounced as “pearl”)** learns invariant representations based on pretext tasks. PIRL is used with a commonly used pretext task that involves solving [jigsaw](https://paperswithcode.com/method/jigsaw) puzzles. Specifically, PIRL constructs image representations that are similar to the representation of transformed versions of the same image and different from the representations of other images.

PIRL

Self-Supervised Learning of Pretext-Invariant Representations

Sparsemax

From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

**Drafting Network** is a style transfer module designed to transfer global style patterns in low-resolution, since global patterns can be transferred easier in low resolution due to larger receptive field and less local details. To achieve single style transfer, earlier work trained an encoder-decoder module, where only the content image is used as input. To better combine the style feature and the content feature, the Drafting Network adopts the [AdaIN module](https://paperswithcode.com/method/adaptive-instance-normalization).

The architecture of Drafting Network is shown in the Figure, which includes an encoder, several AdaIN modules and a decoder. (1) The encoder is a pre-trained [VGG](https://paperswithcode.com/method/vgg)-19 network, which is fixed during training. Given $\bar{x}\_{c}$ and $\bar{x}\_{s}$, the VGG encoder extracts features in multiple granularity at 2_1, 3_1 and 4_1 layers. (2) Then, we apply feature modulation between the content and style feature using AdaIN modules after 2_1, 3_1 and 4_1 layers, respectively. (3) Finally, in each granularity of decoder, the corresponding feature from the AdaIN module is merged via a [skip-connection](https://paperswithcode.com/methods/category/skip-connections). Here, skip-connections after AdaIN modules in both low and high levels are leveraged to help to reserve content structure, especially for low-resolution image.

Source	From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com