**Local Prior Matching** is a semi-supervised objective for speech recognition that distills knowledge from a strong prior (e.g. a language model) to provide learning signal to a discriminative model trained on unlabeled speech. The LPM objective minimizes the cross entropy between the local prior and the model distribution, and is minimized when $q\_{y\mid{x}} = \bar{p}\_{y\mid{x}}$. Intuitively, LPM encourages the ASR model to assign posterior probabilities proportional to the linguistic probabilities of the proposed hypotheses.

**Gumbel-Softmax** is a continuous distribution that has the property that it can be smoothly annealed into a categorical distribution, and whose parameter gradients can be easily computed via the reparameterization trick.

Gumbel Softmax

Categorical Reparameterization with Gumbel-Softmax

Semi-Supervised Speech Recognition via Local Prior Matching

Spatial & temporal attention combines the advantages of spatial attention and temporal attention as it adaptively selects both important regions and key frames. Some works compute temporal attention and spatial attention separately, while others produce joint spatio & temporal attention maps. Further works focusing on capturing pairwise relations.

Source	Semi-Supervised Speech Recognition via Local Prior Matching
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com