The **WaveGrad UBlock** is used for upsampling in [WaveGrad](https://paperswithcode.com/method/wavegrad). Neural audio generation models often use large receptive field. Dilation factors of four convolutional layers are 1, 2, 1, 2 for the first two UBlocks and 1, 2, 4, 8 for the rest. Orthogonal initialization is used.

Please enter a description about the method here

iGCL

Graph Contrastive Learning with Implicit Augmentations

WaveGrad UBlock

WaveGrad: Estimating Gradients for Waveform Generation

**Batch Normalization** aims to reduce internal covariate shift, and in doing so aims to accelerate the training of deep neural nets. It accomplishes this via a normalization step that fixes the means and variances of layer inputs. Batch Normalization also has a beneficial effect on the gradient flow through the network, by reducing the dependence of gradients on the scale of the parameters or of their initial values. This allows for use of much higher learning rates without the risk of divergence. Furthermore, batch normalization regularizes the model and reduces the need for [Dropout](https://paperswithcode.com/method/dropout).

We apply a batch normalization layer as follows for a minibatch $\mathcal{B}$:

$$ \mu\_{\mathcal{B}} = \frac{1}{m}\sum^{m}\_{i=1}x\_{i} $$

$$ \sigma^{2}\_{\mathcal{B}} = \frac{1}{m}\sum^{m}\_{i=1}\left(x\_{i}-\mu\_{\mathcal{B}}\right)^{2} $$

$$ \hat{x}\_{i} = \frac{x\_{i} - \mu\_{\mathcal{B}}}{\sqrt{\sigma^{2}\_{\mathcal{B}}+\epsilon}} $$

$$ y\_{i} = \gamma\hat{x}\_{i} + \beta = \text{BN}\_{\gamma, \beta}\left(x\_{i}\right) $$

Where $\gamma$ and $\beta$ are learnable parameters.

Source	WaveGrad: Estimating Gradients for Waveform Generation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com