**Efficient Channel Attention** is an architectural unit based on [squeeze-and-excitation](https://paperswithcode.com/method/squeeze-and-excitation-block) blocks that reduces model complexity without dimensionality reduction. It was proposed as part of the [ECA-Net](https://paperswithcode.com/method/eca-net) CNN architecture. 

After channel-wise [global average pooling](https://paperswithcode.com/method/global-average-pooling) without dimensionality reduction, the ECA captures local cross-channel interaction by considering every channel and its $k$ neighbors. The ECA can be efficiently implemented by fast $1D$ [convolution](https://paperswithcode.com/method/convolution) of size $k$, where kernel size $k$ represents the coverage of local cross-channel interaction, i.e., how many neighbors participate in attention prediction of one channel.

**Unigram Segmentation** is a subword segmentation algorithm based on a unigram language model. It provides multiple segmentations with probabilities. The language model allows for emulating the noise generated during the segmentation of actual data.

The unigram language model makes an assumption that each subword occurs independently, and consequently, the probability of a subword sequence $\mathbf{x} = (x_1,\ldots,x_M)$ is
formulated as the product of the subword occurrence probabilities
$p(x_i)$:

$$
  P(\mathbf{x}) = \prod_{i=1}^{M} p(x_i), \\\\
  \forall i\,\, x_i \in \mathcal{V},\,\,\,
  \sum_{x \in \mathcal{V}} p(x) = 1, \nonumber
$$

where $\mathcal{V}$ is a pre-determined vocabulary.  The most probable
segmentation $\mathbf{x}^*$ for the input sentence $X$ is then given by:

$$
  \mathbf{x}^{*} = \text{argmax}_{\mathbf{x} \in \mathcal{S}(X)} P(\mathbf{x}),
$$

where $\mathcal{S}(X)$ is a set of segmentation candidates built from
the input sentence $X$.  $\mathbf{x}^*$ is obtained with the Viterbi
algorithm.

Unigram Segmentation

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Efficient Channel Attention

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

A **Linear Layer** is a projection $\mathbf{XW + b}$.

Source	ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com