**SpatialDropout** is a type of [dropout](https://paperswithcode.com/method/dropout) for convolutional networks. For a given [convolution](https://paperswithcode.com/method/convolution) feature tensor of size $n\_{\text{feats}}$×height×width, we perform only $n\_{\text{feats}}$ dropout
trials and extend the dropout value across the entire feature map. Therefore, adjacent pixels in the dropped-out feature
map are either all 0 (dropped-out) or all active as illustrated in the figure to the right.

**Chimera** is a pipeline model parallelism scheme which combines bidirectional pipelines for efficiently training large-scale models. The key idea of Chimera is to combine two pipelines in different directions (down and up pipelines). 

Denote $N$ as the number of micro-batches executed by each worker within a training iteration, and $D$ the number of pipeline stages (depth), and $P$ the number of workers.

The Figure shows an example with four pipeline stages (i.e. $D=4$). Here we assume there are $D$ micro-batches executed by each worker within a training iteration, namely $N=D$, which is the minimum to keep all the stages active. 

In the down pipeline, stage$\_{0}$∼stage$\_{3}$ are mapped to $P\_{0}∼P\_{3}$ linearly, while in the up pipeline the stages are mapped in a completely opposite order. The $N$ (assuming an even number) micro-batches are equally partitioned among the two pipelines. Each pipeline schedules $N/2$ micro-batches using 1F1B strategy, as shown in the left part of the Figure. Then, by merging these two pipelines together, we obtain the pipeline schedule of Chimera. Given an even number of stages $D$ (which can be easily satisfied in practice), it is guaranteed that there is no conflict (i.e., there is at most one micro-batch occupies the same time slot on each worker) during merging.

Chimera

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines

SpatialDropout

Efficient Object Localization Using Convolutional Networks

An infinite image generator which is based on a patch-wise, periodically equivariant generator.

Source	Efficient Object Localization Using Convolutional Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com