A **Bottleneck Residual Block** is a variant of the [residual block](https://paperswithcode.com/method/residual-block) that utilises 1x1 convolutions to create a bottleneck. The use of a bottleneck reduces the number of parameters and matrix multiplications. The idea is to make residual blocks as thin as possible to increase depth and have less parameters. They were introduced as part of the [ResNet](https://paperswithcode.com/method/resnet) architecture, and are used as part of deeper ResNets such as ResNet-50 and ResNet-101.

**SqueezeBERT** is an efficient architectural variant of [BERT](https://paperswithcode.com/method/bert) for natural language processing that uses [grouped convolutions](https://paperswithcode.com/method/grouped-convolution). It is much like BERT-base, but with positional feedforward connection layers implemented as convolutions, and grouped [convolution](https://paperswithcode.com/method/convolution) for many of the layers.

SqueezeBERT

SqueezeBERT: What can computer vision teach NLP about efficient neural networks?

Bottleneck Residual Block

Deep Residual Learning for Image Recognition

**LeViT Attention Block** is a module used for [attention](https://paperswithcode.com/methods/category/attention-mechanisms) in the [LeViT](https://paperswithcode.com/method/levit) architecture. Its main feature is providing positional information within each attention block, i.e. where we explicitly inject relative position information in the attention mechanism. This is achieved by adding an attention bias to the attention maps.

Source	Deep Residual Learning for Image Recognition
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com