**Scaled Exponential Linear Units**, or **SELUs**, are activation functions that induce self-normalizing properties.

The SELU activation function is given by 

$$f\left(x\right) = \lambda{x} \text{ if } x \geq{0}$$
$$f\left(x\right) = \lambda{\alpha\left(\exp\left(x\right) -1 \right)} \text{ if } x < 0 $$

with $\alpha \approx 1.6733$ and $\lambda \approx 1.0507$.

**WaveGAN** is a generative adversarial network for unsupervised synthesis of raw-waveform audio (as opposed to image-like spectrograms). 

The WaveGAN architecture is based off [DCGAN](https://paperswithcode.com/method/dcgan). The DCGAN generator uses the [transposed convolution](https://paperswithcode.com/method/transposed-convolution) operation to iteratively upsample low-resolution feature maps into a high-resolution image. WaveGAN modifies this transposed [convolution](https://paperswithcode.com/method/convolution) operation to widen its receptive field, using a longer one-dimensional filters of length 25 instead of two-dimensional filters of size 5x5, and upsampling by a factor of 4 instead of 2 at each layer. The discriminator is modified in a similar way, using length-25 filters in one dimension and increasing stride
from 2 to 4. These changes result in WaveGAN having the same number of parameters, numerical
operations, and output dimensionality as DCGAN. An additional layer is added afterwards to allow for more audio samples. Further changes include:

1. Flattening 2D convolutions into 1D (e.g. 5x5 2D conv becomes length-25 1D).
2. Increasing the stride factor for all convolutions (e.g. stride 2x2 becomes stride 4).
3. Removing [batch normalization](https://paperswithcode.com/method/batch-normalization) from the generator and discriminator.
4. Training using the [WGAN](https://paperswithcode.com/method/wgan)-GP strategy.

WaveGAN

Adversarial Audio Synthesis

SELU

Self-Normalizing Neural Networks

Diffusion-convolutional neural networks (DCNN) is a model for graph-structured data. Through the introduction of a diffusion-convolution operation, diffusion-based representations can be learned from graph structured data and used as an effective basis for node classification.

Description and image from: [Diffusion-Convolutional Neural Networks](https://arxiv.org/pdf/1511.02136.pdf)

Source	Self-Normalizing Neural Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com

Viet-Anh on Software

What is: Scaled Exponential Linear Unit?

Viet-Anh on Software