Viet-Anh on Software Logo

What is: Adaptive Input Representations?

SourceAdaptive Input Representations for Neural Language Modeling
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Adaptive Input Embeddings extend the adaptive softmax to input word representations. The factorization assigns more capacity to frequent words and reduces the capacity for less frequent words with the benefit of reducing overfitting to rare words.