**Dorylus** is a distributed system for training graph neural networks which uses cheap CPU servers and Lambda threads. It scales to
large billion-edge graphs with low-cost cloud resources.

**Linformer** is a linear [Transformer](https://paperswithcode.com/method/transformer) that utilises a linear self-attention mechanism to tackle the self-attention bottleneck with [Transformer models](https://paperswithcode.com/methods/category/transformers). The original [scaled dot-product attention](https://paperswithcode.com/method/scaled) is decomposed into multiple smaller attentions through linear projections, such that the combination of these operations forms a low-rank factorization of the original attention.

Linformer

Linformer: Self-Attention with Linear Complexity

Dorylus

Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads

**MODERN**, or **Modulated Residual Network**, is an architecture for [visual question answering](https://paperswithcode.com/task/visual-question-answering) (VQA). It employs [conditional batch normalization](https://paperswithcode.com/method/conditional-batch-normalization) to allow a linguistic embedding from an [LSTM](https://paperswithcode.com/method/lstm) to modulate the [batch normalization](https://paperswithcode.com/method/batch-normalization) parameters of a [ResNet](https://paperswithcode.com/method/resnet). This enables the linguistic embedding to manipulate entire feature maps by scaling them up or down, negating them, or shutting them off, etc.

Source	Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com