**Dot-Product Attention** is an attention mechanism where the alignment score function is calculated as: 

$$f_{att}\left(\textbf{h}_{i}, \textbf{s}\_{j}\right) = h\_{i}^{T}s\_{j}$$

It is equivalent to [multiplicative attention](https://paperswithcode.com/method/multiplicative-attention) (without a trainable weight matrix, assuming this is instead an identity matrix). Here $\textbf{h}$ refers to the hidden states for the encoder, and $\textbf{s}$ is the hidden states for the decoder. The function above is thus a type of alignment score function. 

Within a neural network, once we have the alignment scores, we calculate the final scores/weights using a [softmax](https://paperswithcode.com/method/softmax) function of these alignment scores (ensuring it sums to 1).

**node2vec** is a framework for learning graph embeddings for nodes in graphs. Node2vec maximizes a likelihood objective over mappings which preserve neighbourhood distances in higher dimensional spaces. From an algorithm design perspective, node2vec exploits the freedom to define neighbourhoods for nodes and provide an explanation for the effect of the choice of neighborhood on the learned representations. 

For each node, node2vec simulates biased random walks based on an efficient network-aware search strategy and the nodes appearing in the random walk define neighbourhoods. The search strategy accounts for the relative influence nodes exert in a network. It also generalizes prior work alluding to naive search strategies by providing flexibility in exploring neighborhoods.

node2vec

node2vec: Scalable Feature Learning for Networks

Dot-Product Attention

Effective Approaches to Attention-based Neural Machine Translation

FINCH is a parameter-free fast and scalable clustering algorithm. it stands out for its speed and clustering quality.

Source	Effective Approaches to Attention-based Neural Machine Translation
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com