**BigGAN** is a type of generative adversarial network that was designed for scaling generation to high-resolution, high-fidelity images. It includes a number of incremental changes and innovations. The baseline and incremental changes are:

- Using [SAGAN](https://paperswithcode.com/method/sagan) as a baseline with spectral norm. for G and D, and using [TTUR](https://paperswithcode.com/method/ttur).
- Using a Hinge Loss [GAN](https://paperswithcode.com/method/gan) objective
- Using class-[conditional batch normalization](https://paperswithcode.com/method/conditional-batch-normalization) to provide class information to G (but with linear projection not MLP.
- Using a [projection discriminator](https://paperswithcode.com/method/projection-discriminator) for D to provide class information to D.
- Evaluating with EWMA of G's weights, similar to ProGANs.

The innovations are:

- Increasing batch sizes, which has a big effect on the Inception Score of the model.
- Increasing the width in each layer leads to a further Inception Score improvement.
- Adding skip connections from the latent variable $z$ to further layers helps performance.
- A new variant of [Orthogonal Regularization](https://paperswithcode.com/method/orthogonal-regularization).

**Deep Voice 3 (DV3)** is a fully-convolutional attention-based neural text-to-speech system. The Deep Voice 3 architecture consists of three components:

- Encoder: A fully-convolutional encoder, which converts textual features to an internal
learned representation.

- Decoder: A fully-convolutional causal decoder, which decodes the learned representation
with a multi-hop convolutional attention mechanism into a low-dimensional audio representation (mel-scale spectrograms) in an autoregressive manner.

- Converter: A fully-convolutional post-processing network, which predicts final vocoder
parameters (depending on the vocoder choice) from the decoder hidden states. Unlike the
decoder, the converter is non-causal and can thus depend on future context information.

The overall objective function to be optimized is a linear combination of the losses from the decoder and the converter. The authors separate decoder and converter and apply multi-task training, because it makes attention learning easier in practice. To be specific, the loss for mel-spectrogram prediction guides training of the attention mechanism, because the attention is trained with the gradients from mel-spectrogram prediction besides vocoder parameter prediction.

Deep Voice 3

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

BigGAN

Large Scale GAN Training for High Fidelity Natural Image Synthesis

**ENet Dilated Bottleneck** is an image model block used in the [ENet](https://paperswithcode.com/method/enet) semantic segmentation architecture. It is the same as a regular [ENet Bottleneck](https://paperswithcode.com/method/enet-bottleneck) but employs dilated convolutions instead.

Source	Large Scale GAN Training for High Fidelity Natural Image Synthesis
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com