[Transformer](https://paperswithcode.com/method/transformer)-Decoder is a modification to Transformer-Encoder-Decoder for long sequences that drops the encoder
module, combines the input and output sequences into a single ”sentence” and is trained as a standard language model. It is used in [GPT](https://paperswithcode.com/method/gpt) and later revisions.

**Random Grayscale**  is an image data augmentation that converts an image to grayscale with probability $p$.

Random Grayscale

Generating Wikipedia by Summarizing Long Sequences

**Dilated Convolutions** are a type of [convolution](https://paperswithcode.com/method/convolution) that “inflate” the kernel by inserting holes between the kernel elements. An additional parameter $l$ (dilation rate) indicates how much the kernel is widened. There are usually $l-1$ spaces inserted between kernel elements. 

Note that concept has existed in past literature under different names, for instance the *algorithme a trous*,  an algorithm for wavelet decomposition (Holschneider et al., 1987; Shensa, 1992).

Source	Generating Wikipedia by Summarizing Long Sequences
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com