**MixText** is a semi-supervised learning method for text classification, which uses a new data augmentation method called TMix. TMix creates a large amount of augmented training samples by interpolating text in hidden space. The technique leverages advances in data augmentation to guess low-entropy labels for unlabeled data, making them as easy to use as labeled data.

**Stochastic Depth** aims to shrink the depth of a network during training, while
keeping it unchanged during testing. This is achieved by randomly dropping entire [ResBlocks](https://paperswithcode.com/method/residual-block) during training and bypassing their transformations through skip connections. 

Let $b\_{l} \in$ {$0, 1$} denote a Bernoulli random variable, which indicates whether the $l$th ResBlock is active ($b\_{l} = 1$) or inactive ($b\_{l} = 0$). Further, let us denote the “survival” probability of ResBlock $l$ as $p\_{l} = \text{Pr}\left(b\_{l} = 1\right)$. With this definition we can bypass the $l$th ResBlock by multiplying its function $f\_{l}$ with $b\_{l}$ and we extend the update rule to:

$$ H\_{l} = \text{ReLU}\left(b\_{l}f\_{l}\left(H\_{l-1}\right) + \text{id}\left(H\_{l-1}\right)\right) $$

If $b\_{l} = 1$, this reduces to the original [ResNet](https://paperswithcode.com/method/resnet) update and this ResBlock remains unchanged. If $b\_{l} = 0$, the ResBlock reduces to the identity function, $H\_{l} = \text{id}\left((H\_{l}−1\right)$.

Stochastic Depth

Deep Networks with Stochastic Depth

MixText

MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

**RetinaNet-RS** is an object detection model produced through a model scaling method based on changing the the input resolution and [ResNet](https://paperswithcode.com/method/resnet) backbone depth. For [RetinaNet](https://paperswithcode.com/method/retinanet), we scale up input resolution from 512 to 768 and the ResNet backbone depth from 50 to 152. As RetinaNet performs dense one-stage object detection, the authors find scaling up input resolution leads to large resolution feature maps hence more anchors to process. This results in a higher capacity dense prediction heads and expensive NMS. Scaling stops at input resolution 768 × 768 for RetinaNet.

Source	MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com