**Local Patch Interaction**, or **LPI**, is a module used for the [XCiT layer](https://paperswithcode.com/method/xcit-layer) to enable explicit communication across patches. LPI consists of two [depth-wise 3×3 convolutional layers](https://paperswithcode.com/method/depthwise-convolution) with [Batch Normalization](https://paperswithcode.com/method/batch-normalization) and [GELU](https://paperswithcode.com/method/gelu) non-linearity in between. Due to its depth-wise structure, the LPI block has a negligible overhead in terms of parameters, as well as a limited overhead in terms of throughput and memory usage during inference.

FIERCE is an entropic regularization on the **feature** space

FIERCE

Preserving Fine-Grain Feature Information in Classification via Entropic Regularization

Local Patch Interaction

XCiT: Cross-Covariance Image Transformers

A **Pyramidal Residual Unit** is a type of residual unit where the number of channels gradually increases as a function of the depth at which the layer occurs, which is similar to a pyramid structure of which the shape gradually widens from the top downwards. It was introduced as part of the [PyramidNet](https://paperswithcode.com/method/pyramidnet) architecture.

Source	XCiT: Cross-Covariance Image Transformers
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com