R-Mix (Random Mix-up) is a Mix-up family Data Augmentation method. It combines random Mix-up with Saliency-guided mix-up, producing a procedure that is fast and performant, while reserving good characteristics of Saliency-guided Mix-up such as low Expected Calibration Error and high Weakly-supervised Object Localization accuracy.

**Fraternal Dropout** is a regularization method for recurrent neural networks that trains two identical copies of an RNN (that share parameters) with different [dropout](https://paperswithcode.com/method/dropout) masks while minimizing the difference between their (pre-[softmax](https://paperswithcode.com/method/softmax)) predictions. This encourages the representations of RNNs to be invariant to dropout mask, thus being robust.

Fraternal Dropout

R-Mix

Expeditious Saliency-guided Mix-up through Random Gradient Thresholding

**NT-ASGD**, or **Non-monotonically Triggered ASGD**, is an averaged stochastic gradient descent technique. 

In regular ASGD, we take steps identical to [regular SGD](https://paperswithcode.com/method/sgd) but instead of returning the last iterate as the solution, we return $\frac{1}{\left(K-T+1\right)}\sum^{T}\_{i=T}w\_{i}$, where $K$ is the total number of iterations and $T < K$ is a user-specified averaging trigger.

NT-ASGD has a non-monotonic criterion that conservatively triggers the averaging when the validation metric fails to improve for multiple cycles. Given that the choice of triggering is irreversible, this conservatism ensures that the randomness of training does not play a major role in the decision.

Source	Expeditious Saliency-guided Mix-up through Random Gradient Thresholding
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com