A **Harmonic Block** is an image model component that utilizes [Discrete Cosine Transform](https://paperswithcode.com/method/discrete-cosine-transform) (DCT) filters. Convolutional neural networks (CNNs) learn filters in order to capture local correlation patterns in feature space. In contrast, DCT has preset spectral filters, which can be better for compressing information (due to the presence of redundancy in the spectral domain).

DCT has been successfully used for JPEG encoding to transform image blocks into spectral representations to capture the most information with a small number of coefficients. Harmonic blocks learn how to optimally combine spectral coefficients at every layer to produce a fixed size representation defined as a weighted sum of responses to DCT filters. The use of DCT filters allows to address the task of model compression.

Neighborhood Attention is a restricted self attention pattern in which each token's receptive field is limited to its nearest neighboring pixels. It was proposed in [Neighborhood Attention Transformer](https://paperswithcode.com/paper/neighborhood-attention-transformer) as an alternative to other local attention mechanisms used in Hierarchical Vision Transformers.

NA is in concept similar to [stand alone self attention (SASA)](https://paperswithcode.com/method/sasa), in that both can be implemented with a raster scan sliding window operation over the key value pair. However, NA would require a modification to handle corner pixels, which helps maintain a fixed receptive field size and an increased number of relative positions.

The primary challenge in experimenting with both NA and SASA has been computation. Simply extracting key values for each query is slow, takes up a large amount of memory, and is eventually intractable at scale. NA was therefore implemented through a new CUDA extension to PyTorch, [NATTEN](https://github.com/SHI-Labs/NATTEN).

Neighborhood Attention

Neighborhood Attention Transformer

Harmonic Block

Harmonic Convolutional Networks based on Discrete Cosine Transform

\emph{Canonical partition} $\mathcal{P}$ crops the index-restricted d-hop neighborhood around the center node from the target graph. $\mathcal{D}(G_t,v_i,v_c)$ means the shortest distance between $v_i$ and $v_c$ on $G_t$.
 \begin{equation}
        \mathcal{P}(G_t, v_c, d) = G_c, \operatorname{ s.t. } G_c \subseteq G_t, V_c = \{ v_i \in V_t|\mathcal{D}(G_t,v_i,v_c) \leq d , v_i \leq v_c\}
\end{equation}

The graph $G_c$ obtained by canonical partition is called the \emph{canonical neighborhood}. Canonical neighborhoods can correctly substitute the target graph in canonical count. The subgraph count of query in target equals the summation of the canonical count of query in canonical neighborhoods for all target nodes. Canonical neighborhoods are acquired with canonical partition $\mathcal{P}$, given any $d$ greater than the diameter of the query.
\begin{equation}
	\mathcal{C}(G_q,G_t) = \sum_{v_c \in V_t} \mathcal{C}_c(G_q, \mathcal{P}(G_t, v_c, d),v_c), 
	d \geq \max_{v_i, v_j \in V_q} \mathcal{D}(G_q, v_i, v_j)
\end{equation}

Source	Harmonic Convolutional Networks based on Discrete Cosine Transform
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com