Viet-Anh on Software Logo

What is: Label Smoothing?

Year1985
Data SourceCC BY-SA - https://paperswithcode.com

Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the likelihood of logp(yx)\log{p}\left(y\mid{x}\right) directly can be harmful. Assume for a small constant ϵ\epsilon, the training set label yy is correct with probability 1ϵ1-\epsilon and incorrect otherwise. Label Smoothing regularizes a model based on a softmax with kk output values by replacing the hard 00 and 11 classification targets with targets of ϵk1\frac{\epsilon}{k-1} and 1ϵ1-\epsilon respectively.

Source: Deep Learning, Goodfellow et al

Image Source: When Does Label Smoothing Help?