Viet-Anh on Software Logo

What is: Universal Transformer?

SourceUniversal Transformers
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

The Universal Transformer is a generalization of the Transformer architecture. Universal Transformers combine the parallelizability and global receptive field of feed-forward sequence models like the Transformer with the recurrent inductive bias of RNNs. They also utilise a dynamic per-position halting mechanism.