**Batch-Channel Normalization**, or **BCN**, uses batch knowledge to prevent channel-normalized models from getting too close to "elimination singularities". Elimination singularities correspond to the points on the training trajectory where neurons become consistently deactivated. They cause degenerate manifolds in the loss landscape which will slow down training and harm model performances.

**DeepViT** is a type of [vision transformer](https://paperswithcode.com/method/vision-transformer) that replaces the self-attention layer within the [transformer](https://paperswithcode.com/method/transformer) block with a [Re-attention module](https://paperswithcode.com/method/re-attention-module) to address the issue of attention collapse and enables training deeper ViTs.

DeepViT

DeepViT: Towards Deeper Vision Transformer

BatchChannel Normalization

Rethinking Normalization and Elimination Singularity in Neural Networks

Extends [PatchGAN](https://paperswithcode.com/method/patchgan) discriminator for the task of layout2image generation. The discriminator is comprised of two processing streams: one for the RGB image and one for its semantics, which are fused together at the later stages of the discriminator.

Source	Rethinking Normalization and Elimination Singularity in Neural Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com