**Local Augmentation for Graph Neural Networks**, or **LA-GNN**, is a data augmentation technique that enhances node features by its local subgraph structures. Specifically, it learns the conditional distribution of the connected neighbors’ representations given the representation of the central node, which has an analogy with the [Skip-gram of word2vec](https://paperswithcode.com/method/skip-gram-word2vec) model that predicts the probability of the context given the central word. After augmenting the neighborhood, we concat the initial and the generated feature matrix as input for GNNs.

A **TridentNet Block** is a feature extractor used in object detection models. Instead of feeding in multi-scale inputs like the image pyramid, in a [TridentNet](https://paperswithcode.com/method/tridentnet) block we adapt the backbone network for different scales. These blocks create multiple scale-specific feature maps. With the help of dilated convolutions, different branches of trident blocks have the same network structure and share the
same parameters yet have different receptive fields. Furthermore, to avoid training objects with extreme scales, a scale-aware training scheme is employed to make each branch specific to a given scale range matching its receptive field. Weight sharing is used to prevent overfitting.

TridentNet Block

Scale-Aware Trident Networks for Object Detection

Local Augmentation

Local Augmentation for Graph Neural Networks

**Adaptive Masking** is a type of attention mechanism that allows a model to learn its own context size to attend over. For each head in [Multi-Head Attention](https://paperswithcode.com/method/multi-head-attention), a masking function is added to control for the span of the attention. A masking function is a non-increasing function that maps a
distance to a value in $\left[0, 1\right]$. Adaptive masking takes the following soft masking function $m\_{z}$ parametrized by a real value $z$ in $\left[0, S\right]$:

$$ m\_{z}\left(x\right) = \min\left[\max\left[\frac{1}{R}\left(R+z-x\right), 0\right], 1\right] $$

where $R$ is a hyper-parameter that controls its softness. The shape of this piecewise function as a function of the distance. This soft masking function is inspired by [Jernite et al. (2017)](https://arxiv.org/abs/1611.06188). The attention weights from are then computed on the masked span:

$$ a\_{tr} = \frac{m\_{z}\left(t-r\right)\exp\left(s\_{tr}\right)}{\sum^{t-1}\_{q=t-S}m\_{z}\left(t-q\right)\exp\left(s\_{tq}\right)}$$

A $\mathcal{l}\_{1}$ penalization is added on the parameters $z\_{i}$ for each attention head $i$ of the model to the loss function:

$$ L = - \log{P}\left(w\_{1}, \dots, w\_{T}\right) + \frac{\lambda}{M}\sum\_{i}z\_{i} $$

where $\lambda > 0$ is the regularization hyperparameter, and $M$ is the number of heads in each
layer. This formulation is differentiable in the parameters $z\_{i}$, and learnt jointly with the rest of the model.

Source	Local Augmentation for Graph Neural Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com