Viet-Anh on Software Logo

What is: Distance to Modelled Embedding?

SourceOut-of-Distribution Example Detection in Deep Neural Networks using Distance to Modelled Embedding
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

DIME, or Distance to Modelled Embedding, is a method for detecting out-of-distribution examples during prediction time. Given a trained neural network, the training data drawn from some high-dimensional distribution in data space XX is transformed into the model’s intermediate feature vector space Rp\mathbb{R}^{p}. The training set embedding is linearly approximated as a hyperplane. When we then receive new observations it is difficult to assess if observations are out-of-distribution directly in data space, so we transform them into the same intermediate feature space. Finally, the Distance-to-Modelled-Embedding (DIME) can be used to assess whether new observations fit into the expected embedding covariance structure.