A GNN for dags, which injects their topological order as an inductive bias via asynchronous message passing.

**Linear Warmup With Cosine Annealing** is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule afterwards.

Linear Warmup With Cosine Annealing

DAGNN

Directed Acyclic Graph Neural Networks

**SERLU**, or **Scaled Exponentially-Regularized Linear Unit**, is a type of activation function. The new function introduces a bump-shaped function in the region of negative input. The bump-shaped function has approximately zero response to large negative input while being able to push the output of SERLU towards zero mean statistically.

$$ \text{SERLU}\left(x\right)) = \lambda\_{serlu}x \text{ if } x \geq 0 $$
$$ \text{SERLU}\left(x\right)) = \lambda\_{serlu}\alpha\_{serlu}xe^{x} \text{ if } x < 0 $$

where the two parameters $\lambda\_{serlu} > 0$ and $\alpha\_{serlu} > 0$ remain to be specified.

Source	Directed Acyclic Graph Neural Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com