**Residual Blocks** are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced as part of the [ResNet](https://paperswithcode.com/method/resnet) architecture.
 
Formally, denoting the desired underlying mapping as $\mathcal{H}({x})$, we let the stacked nonlinear layers fit another mapping of $\mathcal{F}({x}):=\mathcal{H}({x})-{x}$. The original mapping is recast into $\mathcal{F}({x})+{x}$. The $\mathcal{F}({x})$ acts like a residual, hence the name 'residual block'.

The intuition is that it is easier to optimize the residual mapping than to optimize the original, unreferenced mapping. To the extreme, if an identity mapping were optimal, it would be easier to push the residual to zero than to fit an identity mapping by a stack of nonlinear layers. Having skip connections allows the network to more easily learn identity-like mappings.

Note that in practice, [Bottleneck Residual Blocks](https://paperswithcode.com/method/bottleneck-residual-block) are used for deeper ResNets, such as ResNet-50 and ResNet-101, as these bottleneck blocks are less computationally intensive.

A **Ghost BottleNeck** is a skip connection block, similar to the basic [residual block](https://paperswithcode.com/method/residual-block) in [ResNet](https://paperswithcode.com/method/resnet) in which several convolutional layers and shortcuts are integrated, but stacks [Ghost Modules](https://paperswithcode.com/method/ghost-module) instead (two stacked Ghost modules). It was proposed as part of the [GhostNet](https://paperswithcode.com/method/ghostnet) CNN architecture.

The first Ghost module acts as an expansion layer increasing the number of channels. The ratio between the number of the output channels and that of the input is referred to as the *expansion ratio*. The second Ghost module reduces the number of channels to match the shortcut path. Then the shortcut is connected between the inputs and the outputs of these two Ghost modules. The [batch normalization](https://paperswithcode.com/method/batch-normalization) (BN) and [ReLU](https://paperswithcode.com/method/relu) nonlinearity are applied after each layer, except that ReLU is not used after the second Ghost module as suggested by [MobileNetV2](https://paperswithcode.com/method/mobilenetv2). The Ghost bottleneck described above is for stride=1. As for the case where stride=2, the shortcut path is implemented by a downsampling layer and a [depthwise convolution](https://paperswithcode.com/method/depthwise-convolution) with stride=2 is inserted between the two Ghost modules. In practice, the primary [convolution](https://paperswithcode.com/method/convolution) in Ghost module here is [pointwise convolution](https://paperswithcode.com/method/pointwise-convolution) for its efficiency.

Ghost Bottleneck

GhostNet: More Features from Cheap Operations

Residual Block

Deep Residual Learning for Image Recognition

A **TridentNet Block** is a feature extractor used in object detection models. Instead of feeding in multi-scale inputs like the image pyramid, in a [TridentNet](https://paperswithcode.com/method/tridentnet) block we adapt the backbone network for different scales. These blocks create multiple scale-specific feature maps. With the help of dilated convolutions, different branches of trident blocks have the same network structure and share the
same parameters yet have different receptive fields. Furthermore, to avoid training objects with extreme scales, a scale-aware training scheme is employed to make each branch specific to a given scale range matching its receptive field. Weight sharing is used to prevent overfitting.

Source	Deep Residual Learning for Image Recognition
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com