**Auxiliary Batch Normalization** is a type of regularization used in adversarial training schemes. The idea is that adversarial examples should have a separate [batch normalization](https://paperswithcode.com/method/batch-normalization) components to the clean examples, as they have different underlying statistics.

Lightweight or mobile neural networks used for real-time computer vision tasks contain fewer parameters than normal
networks, which lead to a constrained performance. In this work, we proposed a novel activation function named Tanh Exponential
Activation Function (TanhExp) which can improve the performance for these networks on image classification task significantly.
The definition of TanhExp is $f(x) = x tanh(e^x)$. We demonstrate the simplicity, efficiency, and robustness of TanhExp on various
datasets and network models and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour
also remains stable even with noise added and dataset altered. We show that without increasing the size of the network, the
capacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters
added.

TanhExp

TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks

Auxiliary Batch Normalization

Adversarial Examples Improve Image Recognition

Dynamic Time Warping (DTW) [1] is one of well-known distance measures between a pairwise of time series. The main idea of DTW is to compute the distance from the matching of similar elements between time series. It uses the dynamic programming technique to find the optimal temporal matching between elements of two time series.

For instance, similarities in walking could be detected using DTW, even if one person was walking faster than the other, or if there were accelerations and decelerations during the course of an observation. DTW has been applied to temporal sequences of video, audio, and graphics data — indeed, any data that can be turned into a linear sequence can be analyzed with DTW. A well known application has been automatic speech recognition, to cope with different speaking speeds. Other applications include speaker recognition and online signature recognition. It can also be used in partial shape matching application.

In general, DTW is a method that calculates an optimal match between two given sequences (e.g. time series) with certain restriction and rules:

1. Every index from the first sequence must be matched with one or more indices from the other sequence, and vice versa
2. The first index from the first sequence must be matched with the first index from the other sequence (but it does not have to be its only match)
3. The last index from the first sequence must be matched with the last index from the other sequence (but it does not have to be its only match)
4. The mapping of the indices from the first sequence to indices from the other sequence must be monotonically increasing, and vice versa, i.e. if j>i  are indices from the first sequence, then there must not be two indices l>k in the other sequence, such that index i is matched with index l and index j is matched with index k, and vice versa.

[1] Sakoe, Hiroaki, and Seibi Chiba. "Dynamic programming algorithm optimization for spoken word recognition." IEEE transactions on acoustics, speech, and signal processing 26, no. 1 (1978): 43-49.

Source	Adversarial Examples Improve Image Recognition
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: Auxiliary Batch Normalization?

Viet-Anh on Software