The **Truncation Trick** is a latent sampling procedure for generative adversarial networks, where we sample $z$ from a truncated normal (where values which fall outside a range are resampled to fall inside that range). 
The original implementation was in [Megapixel Size Image Creation with GAN](https://paperswithcode.com/paper/megapixel-size-image-creation-using).
In [BigGAN](http://paperswithcode.com/method/biggan), the authors find this provides a boost to the Inception Score and FID.

Nyströmformer replaces the self-attention in [BERT](https://paperswithcode.com/method/bert)-small and BERT-base using the proposed Nyström approximation. This reduces self-attention complexity to $O(n)$ and allows the [Transformer](https://paperswithcode.com/method/transformer) to support longer sequences.

Nyströmformer

Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

Truncation Trick

Megapixel Size Image Creation using Generative Adversarial Networks

Using cubic splines to improve factorization machine accuracy with numerical features

Source	Megapixel Size Image Creation using Generative Adversarial Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com