**Channel-wise Cross Fusion Transformer** is a module used in the [UCTransNet](https://paperswithcode.com/method/uctransnet) architecture for semantic segmentation. It fuses the multi-scale encoder features with the advantage of the long dependency modeling in the [Transformer](https://paperswithcode.com/method/transformer). The [CCT](https://paperswithcode.com/method/cct) module consists of three steps: multi-scale feature embedding, multi-head [channel-wise cross attention](https://paperswithcode.com/method/channel-wise-cross-attention) and Multi-Layer Perceptron (MLP).

**Channel Shuffle** is an operation to help information flow across feature channels in convolutional neural networks. It was used as part of the [ShuffleNet](https://paperswithcode.com/method/shufflenet) architecture. 

If we allow a group [convolution](https://paperswithcode.com/method/convolution) to obtain input data from different groups, the input and output channels will be fully related. Specifically, for the feature map generated from the previous group layer, we can first divide the channels in each group into several subgroups, then feed each group in the next layer with different subgroups. 

The above can be efficiently and elegantly implemented by a channel shuffle operation: suppose a convolutional layer with $g$ groups whose output has $g \times n$ channels; we first reshape the output channel dimension into $\left(g, n\right)$, transposing and then flattening it back as the input of next layer. Channel shuffle is also differentiable, which means it can be embedded into network structures for end-to-end training.

Channel Shuffle

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

Channel-wise Cross Fusion Transformer

UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer

**SegNet** is a semantic segmentation model. This core trainable segmentation architecture consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the
VGG16 network. The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature maps. Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to
perform non-linear upsampling.

Source	UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com