**FRILL** is a non-semantic speech embedding model trained via knowledge distillation that is fast enough to be run in real-time on a mobile device. The fastest model runs at 0.9 ms, which is 300x faster than TRILL and 25x faster than TRILL-distilled.

**Batch Nuclear-norm Maximization** is an approach for aiding classification in label insufficient situations. It involves maximizing the nuclear-norm of the batch output matrix. The nuclear-norm of a matrix is an upper bound of the Frobenius-norm of the matrix. Maximizing nuclear-norm ensures large Frobenius-norm of the batch matrix, which leads to increased discriminability. The nuclear-norm of the batch matrix is also a convex approximation of the matrix rank, which refers to the prediction diversity.

Batch Nuclear-norm Maximization

Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations

FRILL

FRILL: A Non-Semantic Speech Embedding for Mobile Devices

Like [DARTS](https://paperswithcode.com/method/darts), except subtract the max weight gradients.

Max-W Weighting:
\begin{equation}
output_i = (1 - max(w) + w_i) * op_i(input_i)
\label{eqn:max_w}
\end{equation}

Source	FRILL: A Non-Semantic Speech Embedding for Mobile Devices
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com