**Entropy Minimized Ensemble of Adapters**, or **EMEA**, is a method that optimizes the ensemble weights of the pretrained language adapters for each test sentence by minimizing the entropy of its predictions. The intuition behind the method is that a good [adapter](https://paperswithcode.com/method/adapter) weight $\alpha$ for a test input $x$ should make the model more confident in its prediction for $x$, that is, it should lead to lower model entropy over the input

Make-A-Scene is a text-to-image method that (i) enables a simple control mechanism complementary to text in the form of a scene, (ii) introduces elements that improve the tokenization process by employing domain-specific knowledge over key image regions (faces and salient objects), and (iii) adapts classifier-free guidance for the transformer use case.

Make-A-Scene

Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

EMEA

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties

**Weight Tying** improves the performance of language models by tying (sharing) the weights of the embedding and [softmax](https://paperswithcode.com/method/softmax) layers. This method also massively reduces the total number of parameters in the language models that it is applied to. 

Language models are typically comprised of an embedding layer, followed by a number of [Transformer](https://paperswithcode.com/method/transformer) or [LSTM](https://paperswithcode.com/method/lstm) layers, which are finally followed by a softmax layer. Embedding layers learn word representations, such that similar words (in meaning) are represented by vectors that are near each other (in cosine distance). [Press & Wolf, 2016] showed that the softmax matrix, in which every word also has a vector representation, also exhibits this property. This leads them to propose to share the softmax and embedding matrices, which is done today in nearly all language models.  

This method was independently introduced by [Press & Wolf, 2016](https://paperswithcode.com/paper/using-the-output-embedding-to-improve) and [Inan et al, 2016](https://paperswithcode.com/paper/tying-word-vectors-and-word-classifiers-a).

Additionally, the Press & Wolf paper proposes Three-way Weight Tying, a method for NMT models in which the embedding matrix for the source language, the embedding matrix for the target language, and the softmax matrix for the target language are all tied. That method has been adopted by the Attention Is All You Need model and many other neural machine translation models.

Source	Efficient Test Time Adapter Ensembling for Low-resource Language Varieties
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com