The training contains two steps. The first step is to train data for creating the T2D and D2T models. Then, in the second step, use these
models to infer self-memory for self-training the D2T model.  Then, the training takes self-memory and new data for self-training the D2T model but not the T2D model.

**TGAN** is a type of generative adversarial network that is capable of learning representation from an unlabeled video dataset and producing a new video. The generator consists of two sub networks
called a temporal generator and an image generator. Specifically, the temporal generator first yields a set of latent variables, each of which corresponds to a latent variable for the image generator. Then, the image generator transforms these latent variables into a video which has the same number of frames as the variables. The model comprised of the temporal and image generators can not only enable to efficiently capture the time series, but also be easily extended to frame interpolation. The authors opt for a [WGAN](https://paperswithcode.com/method/wgan) as the basic [GAN](https://paperswithcode.com/method/gan) structure and objective, but use [singular value clipping](https://paperswithcode.com/method/singular-value-clipping) to enforce the Lipschitz constraint.

TGAN

Temporal Generative Adversarial Nets with Singular Value Clipping

self-mem + new data

Self-training from Self-memory in Data-to-text Generation

**Gated Transformer-XL**, or **GTrXL**, is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based architecture for reinforcement learning. It introduces architectural modifications that improve the stability and learning speed of the original Transformer and XL variant. Changes include:

- Placing the [layer normalization](https://paperswithcode.com/method/layer-normalization) on only the input stream of the submodules. A key benefit to this reordering is that it now enables an identity map from the input of the transformer at the first layer to the output of the transformer after the last layer. This is in contrast to the canonical transformer, where there are a series of layer normalization operations that non-linearly transform the state encoding.
- Replacing [residual connections](https://paperswithcode.com/method/residual-connection) with gating layers. The authors' experiments found that [GRUs](https://www.paperswithcode.com/method/gru) were the most effective form of gating.

Source	Self-training from Self-memory in Data-to-text Generation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com