A **ParaNet Convolution Block** is a convolutional block that appears in the encoder and decoder of the [ParaNet](https://paperswithcode.com/method/paranet) text-to-speech architecture. It consists of a 1-D [convolution](https://paperswithcode.com/method/convolution) with a gated linear unit ([GLU](https://paperswithcode.com/method/glu)) and a [residual connection](https://paperswithcode.com/method/residual-connection). It is similar to the [DV3 Convolution Block](https://paperswithcode.com/method/dv3-convolution-block).

**NormFormer** is a type of [Pre-LN](https://paperswithcode.com/method/layer-normalization) transformer that adds three normalization operations to each layer: a Layer Norm after self attention, head-wise scaling of self-attention outputs, and a Layer Norm after the first [fully connected layer](https://paperswithcode.com/method/position-wise-feed-forward-layer). The modifications introduce a small number of additional learnable parameters, which provide a cost-effective way for each layer to change the magnitude of its features, and therefore the magnitude of the gradients to subsequent components.

NormFormer

NormFormer: Improved Transformer Pretraining with Extra Normalization

ParaNet Convolution Block

Non-Autoregressive Neural Text-to-Speech

**MODNet** is a light-weight matting objective decomposition network that can process portrait matting from a single input image in real time. The design of MODNet benefits from optimizing a series of correlated sub-objectives simultaneously via explicit constraints. To overcome the domain shift problem, MODNet introduces a self-supervised strategy based on subobjective consistency (SOC) and  a one-frame delay trick to smooth the results when applying MODNet to portrait video sequence.

Given an input image $I$, MODNet predicts human semantics $s\_{p}$, boundary details $d\_{p}$, and final alpha matte $\alpha\_{p}$ through three interdependent branches, $S, D$, and $F$, which are constrained by specific supervisions generated from the ground truth matte $\alpha\_{g}$. Since the decomposed sub-objectives are correlated and help strengthen each other, we can optimize MODNet end-to-end.

Source	Non-Autoregressive Neural Text-to-Speech
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com