**VOS** is a type of video object segmentation model consisting of two network components. The target appearance model consists of a light-weight module, which is learned during the inference stage using fast optimization techniques to predict a coarse but robust target segmentation. The segmentation model is exclusively trained offline, designed to process the coarse scores into high quality segmentation masks.

A **LAPGAN**, or **Laplacian Generative Adversarial Network**, is a type of generative adversarial network that has a [Laplacian pyramid](https://paperswithcode.com/method/laplacian-pyramid) representation. In the sampling procedure following training, we have a set of generative convnet models {$G\_{0}, \dots , G\_{K}$}, each of which captures the distribution of coefficients $h\_{k}$ for natural images at a different level of the Laplacian pyramid. Sampling an image is akin to a reconstruction procedure, except that the generative
models are used to produce the $h\_{k}$’s:

$$ \tilde{I}\_{k} = u\left(\tilde{I}\_{k+1}\right) + \tilde{h}\_{k} = u\left(\tilde{I}\_{k+1}\right) + G\_{k}\left(z\_{k}, u\left(\tilde{I}\_{k+1}\right)\right)$$

The recurrence starts by setting $\tilde{I}\_{K+1} = 0$ and using the model at the final level $G\_{K}$ to generate a residual image $\tilde{I}\_{K}$ using noise vector $z\_{K}$: $\tilde{I}\_{K} = G\_{K}\left(z\_{K}\right)$. Models at all levels except the final are conditional generative models that take an upsampled version of the current image $\tilde{I}\_{k+1}$ as a conditioning variable, in addition to the noise vector $z\_{k}$.

The generative models {$G\_{0}, \dots, G\_{K}$} are trained using the CGAN approach at each level of the pyramid. Specifically, we construct a Laplacian pyramid from each training image $I$. At each level we make a stochastic choice (with equal probability) to either (i) construct the coefficients $h\_{k}$ either using the standard Laplacian pyramid coefficient generation procedure or (ii) generate them using $G\_{k}:

$$ \tilde{h}\_{k} = G\_{k}\left(z\_{k}, u\left(I\_{k+1}\right)\right) $$

Here $G\_{k}$ is a convnet which uses a coarse scale version of the image $l\_{k} = u\left(I\_{k+1}\right)$ as an input, as well as noise vector $z\_{k}$. $D\_{k}$ takes as input $h\_{k}$ or $\tilde{h}\_{k}$, along with the low-pass image $l\_{k}$ (which is explicitly added to $h\_{k}$ or $\tilde{h}\_{k}$ before the first [convolution](https://paperswithcode.com/method/convolution) layer), and predicts if the image was real or
generated. At the final scale of the pyramid, the low frequency residual is sufficiently small that it
can be directly modeled with a standard [GAN](https://paperswithcode.com/method/gan): $\tilde{h}\_{K} = G\_{K}\left(z\_{K}\right)$ and $D\_{K}$ only has $h\_{K}$ or $\tilde{h}\_{K}$ as input.

Breaking the generation into successive refinements is the key idea. We give up any “global” notion of fidelity; an attempt is never made to train a network to discriminate between the output of a cascade and a real image and instead the focus is on making each step plausible.

LAPGAN

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

Learning Fast and Robust Target Models for Video Object Segmentation

**T-Fixup** is an [initialization](https://paperswithcode.com/methods/category/initialization) method for [Transformers](https://paperswithcode.com/methods/category/transformers) that aims to remove the need for [layer normalization](https://paperswithcode.com/method/layer-normalization) and [warmup](https://paperswithcode.com/method/linear-warmup). The initialization procedure is as follows:

- Apply [Xavier initialization](https://paperswithcode.com/method/xavier-initialization) for all parameters excluding input embeddings. Use Gaussian initialization $\mathcal{N}\left(0, d^{-\frac{1}{2}}\right)$ for input embeddings where $d$ is the embedding dimension.
- Scale $\mathbf{v}\_{d}$ and $\mathbf{w}\_{d}$ matrices in each decoder [attention block](https://paperswithcode.com/method/multi-head-attention), weight matrices in each decoder [MLP block](https://paperswithcode.com/method/position-wise-feed-forward-layer) and input embeddings $\mathbf{x}$ and $\mathbf{y}$ in encoder and decoder by $(9 N)^{-\frac{1}{4}}$
- Scale $\mathbf{v}\_{e}$ and $\mathbf{w}\_{e}$ matrices in each encoder [attention block](https://paperswithcode.com/method/multi-head-attention) and weight matrices in each encoder [MLP block](https://paperswithcode.com/method/position-wise-feed-forward-layer) by $0.67 N^{-\frac{1}{4}}$

Source	Learning Fast and Robust Target Models for Video Object Segmentation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com