**Topographic VAE** is a method for efficiently training deep generative models with topographically organized latent variables. The model learns sets of approximately equivariant features (i.e. "capsules") directly from sequences and achieves higher likelihood on correspondingly transforming test sequences. The combined color/rotation transformation in input space $\tau\_{g}$ becomes encoded as a $\mathrm{Roll}$ within the capsule dimension. The model is thus able decode unseen sequence elements by encoding a partial sequence and Rolling activations within the capsules. This resembles a commutative diagram.

Spatial Broadcast Decoder is an architecture that aims to improve disentangling, reconstruction accuracy, and generalization to held-out regions in data space. It provides a particularly dramatic
benefit when applied to datasets with small objects.

Source: [Watters et al.](https://arxiv.org/pdf/1901.07017v2.pdf)

Image source: [Watters et al.](https://arxiv.org/pdf/1901.07017v2.pdf)

Spatial Broadcast Decoder

Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs

Topographic VAE

Topographic VAEs learn Equivariant Capsules

**V-trace** is an off-policy actor-critic reinforcement learning algorithm that helps tackle the lag between when actions are generated by the actors and when the learner estimates the gradient. Consider a trajectory $\left(x\_{t}, a\_{t}, r\_{t}\right)^{t=s+n}\_{t=s}$ generated by the actor following some policy $\mu$. We can define the $n$-steps V-trace target for $V\left(x\_{s}\right)$, our value approximation at state $x\_{s}$ as:

$$ v\_{s} = V\left(x\_{s}\right) + \sum^{s+n-1}\_{t=s}\gamma^{t-s}\left(\prod^{t-1}\_{i=s}c\_{i}\right)\delta\_{t}V $$

Where $\delta\_{t}V = \rho\_{t}\left(r\_{t} + \gamma{V}\left(x\_{t+1}\right) - V\left(x\_{t}\right)\right)$ is a temporal difference algorithm for $V$, and $\rho\_{t} = \text{min}\left(\bar{\rho}, \frac{\pi\left(a\_{t}\mid{x\_{t}}\right)}{\mu\left(a\_{t}\mid{x\_{t}}\right)}\right)$ and $c\_{i} = \text{min}\left(\bar{c}, \frac{\pi\left(a\_{t}\mid{x\_{t}}\right)}{\mu\left(a\_{t}\mid{x\_{t}}\right)}\right)$ are truncated importance sampling weights. We assume that the truncation levels are such that $\bar{\rho} \geq \bar{c}$.

Source	Topographic VAEs learn Equivariant Capsules
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com