**Linear Warmup** is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.

Image Credit: [Chengwei Zhang](https://www.dlology.com/about-me/)

**Variational Entanglement Detection** is a variational quantum algorithm which uses criteria based on positive maps as a bridge and works as follows. Given an unknown target bipartite quantum state, it firstly decomposes the chosen positive map into a linear combination of NISQ implementable quantum operations. Then, it variationally estimates the minimal eigenvalue of the final state, obtained by executing these quantum operations on the target state and averaging the output states. Deterministic and probabilistic methods are proposed to compute the average. At last, it asserts that the target state is entangled if the optimized minimal eigenvalue is negative. VLNE builds upon a linear decomposition of the transpose map into Pauli terms and the recently proposed trace distance estimation algorithm. It variationally estimates the well-known logarithmic negativity entanglement measure and could be applied to quantify entanglement on near-term quantum devices.

Variational Entanglement Detection

Detecting and quantifying entanglement on near-term quantum devices

Linear Warmup

A **Class Attention** layer, or **CA Layer**, is an [attention mechanism](https://paperswithcode.com/methods/category/attention-mechanisms-1) for [vision transformers](https://paperswithcode.com/methods/category/vision-transformer) used in [CaiT](https://paperswithcode.com/method/cait) that aims to extract information from a set of processed patches. It is identical to a [self-attention layer](https://paperswithcode.com/method/scaled), except that it relies on the attention between (i) the class embedding $x_{\text {class }}$ (initialized at CLS in the first CA) and (ii) itself plus the set of frozen patch embeddings $x_{\text {patches }} .$ 

Considering a network with $h$ heads and $p$ patches, and denoting by $d$ the embedding size, the multi-head class-attention is parameterized with several projection matrices, $W_{q}, W_{k}, W_{v}, W_{o} \in \mathbf{R}^{d \times d}$, and the corresponding biases $b_{q}, b_{k}, b_{v}, b_{o} \in \mathbf{R}^{d} .$ With this notation, the computation of the CA residual block proceeds as follows. We first augment the patch embeddings (in matrix form) as $z=\left[x_{\text {class }}, x_{\text {patches }}\right]$. We then perform the projections:

$$Q=W\_{q} x\_{\text {class }}+b\_{q}$$

$$K=W\_{k} z+b\_{k}$$

$$V=W\_{v} z+b\_{v}$$

The class-attention weights are given by

$$
A=\operatorname{Softmax}\left(Q . K^{T} / \sqrt{d / h}\right)
$$

where $Q . K^{T} \in \mathbf{R}^{h \times 1 \times p}$. This attention is involved in the weighted sum $A \times V$ to produce the residual output vector

$$
\operatorname{out}\_{\mathrm{CA}}=W\_{o} A V+b\_{o}
$$

which is in turn added to $x\_{\text {class }}$ for subsequent processing.

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com