Spatial Broadcast Decoder is an architecture that aims to improve disentangling, reconstruction accuracy, and generalization to held-out regions in data space. It provides a particularly dramatic
benefit when applied to datasets with small objects.

Source: [Watters et al.](https://arxiv.org/pdf/1901.07017v2.pdf)

Image source: [Watters et al.](https://arxiv.org/pdf/1901.07017v2.pdf)

Some object detection datasets contain an overwhelming number of easy examples and a small number of hard examples. Automatic selection of these hard examples can make training more
effective and efficient. **OHEM**, or **Online Hard Example Mining**, is a bootstrapping technique that modifies [SGD](https://paperswithcode.com/method/sgd) to sample from examples in a non-uniform way depending on the current loss of each example under consideration. The method takes advantage of detection-specific problem structure in which each SGD mini-batch consists of only one or two images, but thousands of candidate examples. The candidate examples are subsampled according to a distribution
that favors diverse, high loss instances.

OHEM

Training Region-based Object Detectors with Online Hard Example Mining

Spatial Broadcast Decoder

Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs

**Topographic VAE** is a method for efficiently training deep generative models with topographically organized latent variables. The model learns sets of approximately equivariant features (i.e. "capsules") directly from sequences and achieves higher likelihood on correspondingly transforming test sequences. The combined color/rotation transformation in input space $\tau\_{g}$ becomes encoded as a $\mathrm{Roll}$ within the capsule dimension. The model is thus able decode unseen sequence elements by encoding a partial sequence and Rolling activations within the capsules. This resembles a commutative diagram.

Source	Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com