**WaveGAN** is a generative adversarial network for unsupervised synthesis of raw-waveform audio (as opposed to image-like spectrograms). 

The WaveGAN architecture is based off [DCGAN](https://paperswithcode.com/method/dcgan). The DCGAN generator uses the [transposed convolution](https://paperswithcode.com/method/transposed-convolution) operation to iteratively upsample low-resolution feature maps into a high-resolution image. WaveGAN modifies this transposed [convolution](https://paperswithcode.com/method/convolution) operation to widen its receptive field, using a longer one-dimensional filters of length 25 instead of two-dimensional filters of size 5x5, and upsampling by a factor of 4 instead of 2 at each layer. The discriminator is modified in a similar way, using length-25 filters in one dimension and increasing stride
from 2 to 4. These changes result in WaveGAN having the same number of parameters, numerical
operations, and output dimensionality as DCGAN. An additional layer is added afterwards to allow for more audio samples. Further changes include:

1. Flattening 2D convolutions into 1D (e.g. 5x5 2D conv becomes length-25 1D).
2. Increasing the stride factor for all convolutions (e.g. stride 2x2 becomes stride 4).
3. Removing [batch normalization](https://paperswithcode.com/method/batch-normalization) from the generator and discriminator.
4. Training using the [WGAN](https://paperswithcode.com/method/wgan)-GP strategy.

Network On Network (NON) is practical tabular data classification model based on deep neural network to provide accurate predictions. Various deep methods have been proposed and promising progress has been made. However, most of them use operations like neural network and factorization machines to fuse the embeddings of different features directly, and linearly combine the outputs of those operations to get the final prediction. As a result, the intra-field information and the non-linear interactions between those operations (e.g. neural network and factorization machines) are ignored. Intra-field information is the information that features inside each field belong to the same field. NON is proposed to take full advantage of intra-field information and non-linear interactions. It consists of three components: field-wise network at the bottom to capture the intra-field information, across field network in the middle to choose suitable operations data-drivenly, and operation fusion network on the top to fuse outputs of the chosen operations deeply

Network On Network for Tabular Data Classification in Real-world Applications

WaveGAN

Adversarial Audio Synthesis

**Scaled Exponential Linear Units**, or **SELUs**, are activation functions that induce self-normalizing properties.

The SELU activation function is given by 

$$f\left(x\right) = \lambda{x} \text{ if } x \geq{0}$$
$$f\left(x\right) = \lambda{\alpha\left(\exp\left(x\right) -1 \right)} \text{ if } x < 0 $$

with $\alpha \approx 1.6733$ and $\lambda \approx 1.0507$.

Source	Adversarial Audio Synthesis
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com