**MATE** is a [Transformer](https://paperswithcode.com/method/transformer) architecture designed to model the structure of web tables. It uses sparse attention in a way that allows heads to efficiently attend to either rows or columns in a table. Each attention head reorders the tokens by either column or row index and then applies a windowed attention mechanism. Unlike traditional self-attention, Mate scales linearly in the sequence length.

A **trainable 3D interaction space** aims to captures the associations between the triplet components and helps model the recognition of multiple triplets in the same frame.

Source: [Nwoye et al.](https://arxiv.org/pdf/2007.05405v1.pdf)

Image source: [Nwoye et al.](https://arxiv.org/pdf/2007.05405v1.pdf)

3DIS

Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets

MATE

MATE: Multi-view Attention for Table Transformer Efficiency

**Primer** is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based architecture that improves upon the [Transformer](https://paperswithcode.com/method/transformer) architecture with two improvements found through [neural architecture search](https://paperswithcode.com/methods/category/neural-architecture-search): [squared RELU activations](https://paperswithcode.com/method/squared-relu) in the feedforward block, and [depthwise convolutions]() added to the attention multi-head projections: resulting in a new module called [Multi-DConv-Head-Attention](https://paperswithcode.com/method/multi-dconv-head-attention).

Source	MATE: Multi-view Attention for Table Transformer Efficiency
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com