**Pointer Networks** tackle problems where input and output data are sequential data, but can't be solved by seq2seq type models because discrete categories of output elements depend on the variable input size (and are not decided in advance).

A Pointer Network learns the conditional  probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. They solve the problem of variable size output dictionaries using [additive attention](https://paperswithcode.com/method/additive-attention). But instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, Pointer Networks use attention as a pointer to select a member of the input sequence as the output. 

Pointer-Nets can be used to learn approximate solutions to challenging geometric problems such as finding planar convex hulls, computing Delaunay triangulations, and the planar Travelling Salesman Problem.

**MaxUp** is an adversarial data augmentation technique for improving the generalization performance of machine learning models. The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data.  By doing so, we implicitly introduce a smoothness or robustness regularization against the random perturbations, and hence improve the generation performance.  For example, in the case of Gaussian perturbation, MaxUp is asymptotically equivalent to using the gradient norm of the loss as a penalty to encourage smoothness.

MaxUp

MaxUp: A Simple Way to Improve Generalization of Neural Network Training

Pointer Network

Pointer Networks

To aggregate global spatial information,
an SE block applies global pooling to the feature map.
However, it ignores pixel-wise spatial information,
which is important in dense prediction tasks.
Therefore, Roy et al. proposed
spatial and channel SE blocks (scSE). 
Like BAM, spatial SE blocks are used, complementing SE blocks, 
to provide spatial attention weights to focus on important regions.

Given the input feature map $X$, two parallel modules, spatial SE and channel SE, are applied to feature maps to encode spatial and channel information respectively. The channel SE module is an ordinary SE block, while the spatial SE module adopts $1\times 1$ convolution for spatial squeezing. The outputs from the two modules are fused. The overall process can be written as
\begin{align}
    s_c & = \sigma (W_{2} \delta (W_{1}\text{GAP}(X)))
\end{align}
\begin{align}
    X_\text{chn} & = s_c  X 
\end{align}
\begin{align}
    s_s &= \sigma(\text{Conv}^{1\times 1}(X))
\end{align}
\begin{align}
    X_\text{spa} & = s_s  X
\end{align}
\begin{align}
    Y &= f(X_\text{spa},X_\text{chn})  
\end{align}

where $f$ denotes the fusion function, which can be  maximum, addition, multiplication or concatenation. 

The proposed scSE block combines channel and spatial attention to
enhance features as well as 
capturing pixel-wise spatial information.
Segmentation tasks are greatly benefited as a result.
The integration of an scSE block in F-CNNs makes a consistent improvement in semantic segmentation at negligible extra cost.

Source	Pointer Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com