**LIME**, or **Local Interpretable Model-Agnostic Explanations**, is an algorithm that can explain the predictions of any classifier or regressor in a faithful way, by approximating it locally with an interpretable model. It modifies a single data sample by tweaking the feature values and observes the resulting impact on the output. It performs the role of an "explainer" to explain predictions from each data sample. The output of LIME is a set of explanations representing the contribution of each feature to a prediction for a single sample, which is a form of local interpretability.

Interpretable models in LIME can be, for instance, [linear regression](https://paperswithcode.com/method/linear-regression) or decision trees, which are trained on small perturbations (e.g. adding noise, removing words, hiding parts of the image) of the original model to provide a good local approximation.

M. Bulacu, A. Brink, T. v. d. Zant and L. Schomaker, "Recognition of Handwritten Numerical Fields in a Large Single-Writer Historical Collection," 2009 10th International Conference on Document Analysis and Recognition, Barcelona, Spain, 2009, pp. 808-812, doi: 10.1109/ICDAR.2009.8.

Code: https://github.com/GrHound/imagemorph.c

In contrast with the EM algorithm (Baum-Welch) for HMMs, training the basic character recognizer for a segmentation-based handwriting recognition system is a tricky issue without a standard solution. Our approach was to collect a labeled base set of digit images segmented by hand and then to augment this data by generating synthetic examples using random geometric distortions. We were incited by the record performance in digit recognition reported in Simard et al. (2003) but developed our own algorithm for this purpose.
For every pixel (i,j) of the template image, a random displacement vector ($\Delta x,\Delta y$) is generated. The displacement field of the complete image is smoothed using a Gaussian convolution kernel with standard deviation $\sigma$. The field is finally rescaled to an average amplitude A. The new morphed image (i',j') is generated using the displacement field and bilinear interpolation i'=i+$\Delta$x,j'=j+$\Delta$y. This morphing process is controlled by two parameters: the smoothing radius r and the average pixel displacement D. Both parameters are measured in units of pixels.

An intuitive interpretation is to imagine that the characters are written on a rubber sheet and we apply non-uniform random local distortions, contracting one part, while maybe expanding another part of the character (see Fig. 5). This random elastic morphing is more general than affine transforms, providing a rich ensemble of shape variations. We applied it to our base set of labeled digits (~130 samples per class) to obtain a much expanded training dataset (from 1 up to 80 times). The expansion factor f controls the amount of synthetic data: for every base example, f - 1 additional morphed patterns are generated and used in training.

This is a cheap method relying on random numbers and basic computer graphics. In this way, a virtually infinite volume of training samples can be fabricated. This stratagem is very successful and does not increase the load at recognition time for parametric classifiers. Essentially, we tum the tables around and, instead of trying to recognize a character garbled in an unpredictable way by the writer in the instantaneous act of handwriting, we generate the deformations ourselves, while training a neural network to become immune to such distortions.

The accompanying image, a crop of an RGB page scan, containing the cursive handwritten word 'Zwolle'
was morphed a number of times, with parameters dist=1.5, radius=8.5
This distortion is sufficient to introduce a believable variance in the appearance.

imagemorph 1.5 8.5 < Zwolle.ppm > Zwolle-morphed.ppm

Netpbm image format is common in many CV tools. You can use ImageMagick's convert
or other tools to convert to/fro .ppm


Also see: 
P. Simard, D. Steinkraus, and J. Platt. Best practices for convolutional neural networks applied to visual document analysis. In Proc. of 7th ICDAR, pp 958-962, Edinburgh, Scotland, 2003.

imagemorph

LIME

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

**Cosine Annealing** is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before being increased rapidly again. The resetting of the learning rate acts like a simulated restart of the learning process and the re-use of good weights as the starting point of the restart is referred to as a "warm restart" in contrast to a "cold restart" where a new set of small random numbers may be used as a starting point.

$$\eta\_{t} = \eta\_{min}^{i} + \frac{1}{2}\left(\eta\_{max}^{i}-\eta\_{min}^{i}\right)\left(1+\cos\left(\frac{T\_{cur}}{T\_{i}}\pi\right)\right)
$$

Where where $\eta\_{min}^{i}$ and $ \eta\_{max}^{i}$ are ranges for the learning rate, and $T\_{cur}$ account for how many epochs have been performed since the last restart.

Text Source: [Jason Brownlee](https://machinelearningmastery.com/snapshot-ensemble-deep-learning-neural-network/)

Image Source: [Gao Huang](https://www.researchgate.net/figure/Training-loss-of-100-layer-DenseNet-on-CIFAR10-using-standard-learning-rate-orange-and-M_fig2_315765130)

Source	"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com