**SHA-RNN**, or **Single Headed Attention RNN**, is a recurrent neural network, and language model when combined with an embedding input and [softmax](https://paperswithcode.com/method/softmax) classifier, based on a core [LSTM](https://paperswithcode.com/method/lstm) component and a [single-headed attention](https://paperswithcode.com/method/single-headed-attention) module. Other design choices include a Boom feedforward layer and the use of [layer normalization](https://paperswithcode.com/method/layer-normalization). The guiding principles of the author were to ensure simplicity in the architecture and to keep computational costs bounded (the model was originally trained with a single GPU).

A **Dual Path Network (DPN)** is a convolutional neural network which presents a new topology of connection paths internally. The intuition is that [ResNets](https://paperswithcode.com/method/resnet) enables feature re-usage while [DenseNet](https://paperswithcode.com/method/densenet) enables new feature exploration, and both are important for learning good representations. To enjoy the benefits from both path topologies, Dual Path Networks share common features while maintaining the flexibility to explore new features through dual path architectures. 

We formulate such a dual path architecture as follows:

$$x^{k} = \sum\limits\_{t=1}^{k-1} f\_t^{k}(h^t) \text{,}  $$

$$
y^{k} = \sum\limits\_{t=1}^{k-1} v\_t(h^t) = y^{k-1} + \phi^{k-1}(y^{k-1}) \text{,} \\\\
$$

$$
r^{k} = x^{k} + y^{k} \text{,} \\\\
$$

$$
h^k = g^k \left( r^{k} \right) \text{,}
$$

where $x^{k}$ and $y^{k}$ denote the extracted information at $k$-th step from individual path, $v_t(\cdot)$ is a feature learning function as $f_t^k(\cdot)$. The first equation refers to the densely connected path that enables exploring new features. The second equation refers to the residual path that enables common features re-usage. The third equation defines the dual path that integrates them and feeds them to the last transformation function in the last equation.

Dual Path Networks

SHA-RNN

Single Headed Attention RNN: Stop Thinking With Your Head

**Social-STGCNN** is a method for human trajectory prediction. Pedestrian trajectories are not only influenced by the pedestrian itself but also by interaction with surrounding objects.

Source	Single Headed Attention RNN: Stop Thinking With Your Head
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com