**Label Smoothing** is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the likelihood of $\log{p}\left(y\mid{x}\right)$ directly can be harmful. Assume for a small constant $\epsilon$, the training set label $y$ is correct with probability $1-\epsilon$ and incorrect otherwise. Label Smoothing regularizes a model based on a [softmax](https://paperswithcode.com/method/softmax) with $k$ output values by replacing the hard $0$ and $1$ classification targets with targets of $\frac{\epsilon}{k-1}$ and $1-\epsilon$ respectively.

Source: Deep Learning, Goodfellow et al

Image Source: [When Does Label Smoothing Help?](https://arxiv.org/abs/1906.02629)

**Temporal Distribution Characterization**, or **TDC**, is a module used in the [AdaRNN](https://paperswithcode.com/method/adarnn) architecture to characterize the distributional information in a time series.

Based on the principle of maximum entropy, maximizing the utilization of shared knowledge underlying a times series under temporal covariate shift can be done by finding periods which are most dissimilar to each other, which is also considered as the worst case of temporal covariate shift since the cross-period distributions are the most diverse. TDC achieves this goal for splitting the time-series by solving an optimization problem whose objective can be formulated as:

$$
\max \_{0<K \leq K\_{0}} \max \_{n\_{1}, \cdots, n\_{K}} \frac{1}{K} \sum_{1 \leq i \neq j \leq K} d\left(\mathcal{D}\_{i}, \mathcal{D}\_{j}\right) 
$$

$$
\text { s.t. } \forall i, \Delta_{1}<\left|\mathcal{D}\_{i}\right|<\Delta_{2} ; \sum_{i}\left|\mathcal{D}\_{i}\right|=n
$$

where $d$ is a distance metric, $\Delta\_{1}$ and $\Delta\_{2}$ are predefined parameters to avoid trivial solutions (e.g., very small values or very large values may fail to capture the distribution information), and $K\_{0}$ is the hyperparameter to avoid over-splitting. The metric $d(\cdot, \cdot)$ above can be any distance function, e.g., Euclidean or Editing distance, or some distribution-based distance / divergence, like MMD [14] and KL-divergence.

The learning goal of the optimization problem (1) is to maximize the averaged period-wise distribution distances by searching $K$ and the corresponding periods so that the distributions of each period are as diverse as possible and the learned prediction model has better a more generalization ability.

Temporal Distribution Characterization

AdaRNN: Adaptive Learning and Forecasting of Time Series

Label Smoothing

Karim Hammoudi, Adnane Cabani, Bouthaina Slika, Halim Benhabiles, Fadi Dornaika and Mahmoud Melkemi. SuperpixelGridCut, SuperpixelGridMean and SuperpixelGridMix Data Augmentation, arXiv:2204.08458, 2022. https://doi.org/10.48550/arxiv.2204.08458

Year	1985
Data Source	CC BY-SA - https://paperswithcode.com