**GHM-C** is a loss function designed to balance the gradient flow for anchor classification. The GHM first performs statistics on the number of examples with similar attributes w.r.t their gradient density and then attaches a harmonizing parameter to the gradient of each example according to the density. The modification of gradient can be equivalently implemented by reformulating the loss function. Embedding the GHM into the classification loss is denoted as GHM-C loss. Since the gradient density is a statistical variable depending on the examples distribution in a mini-batch, GHM-C is a dynamic loss that can adapt to the change of data distribution in each batch as well as to the updating of the model.

GHM-C

Gradient Harmonized Single-stage Detector

**MT-PET** is a multi-task version of [Pattern Exploiting Training](https://arxiv.org/abs/2001.07676) (PET) for exaggeration detection, which leverages knowledge from complementary cloze-style QA tasks to improve few-shot learning. It defines pairs of complementary pattern-verbalizer pairs for a main task and auxiliary task. These PVPs are then used to train PET on data from both tasks.

PET uses the masked language modeling objective of pretrained language models to transform a task into one or more cloze-style question answering tasks.  In the original PET implementation, PVPs are defined for a single target task. MT-PET extends this by allowing for auxiliary PVPs from related tasks, adding complementary cloze-style QA tasks during training. The motivation for the multi-task approach is two-fold: 1) complementary cloze-style tasks can potentially help the model to learn different aspects of the main task, i.e. the similar tasks of exaggeration detection and claim strength prediction; 2) data on related tasks can be utilized during training, which is important in situations where data for the main task is limited.

MT-PET

Semi-Supervised Exaggeration Detection of Health Science Press Releases

**Domain Adaptive Neighborhood Clustering via Entropy Optimization (DANCE)** is a self-supervised clustering method that harnesses the cluster structure of the target domain using self-supervision. This is done with a neighborhood clustering technique that self-supervises feature learning in the target. At the same time, useful source features and class boundaries are preserved and adapted with a partial domain alignment loss that the authors refer to as entropy separation loss. This loss allows the model to either match each target example with the source, or reject it as unknown.

DANCE

Universal Domain Adaptation through Self Supervision

**Inception-ResNet-v2-B** is an image model block for a 17 x 17 grid used in the [Inception-ResNet-v2](https://paperswithcode.com/method/inception-resnet-v2) architecture. It largely follows the idea of Inception modules - and grouped convolutions - but also includes residual connections.

Inception-ResNet-v2-B

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

**Target Policy Smoothing** is a regularization strategy for the value function in reinforcement learning. Deterministic policies can overfit to narrow peaks in the value estimate, making them highly susceptible to functional approximation error, increasing the variance of the target. To reduce this variance, target policy smoothing adds a small amount of random noise to the target policy and averages over mini-batches - approximating a [SARSA](https://paperswithcode.com/method/sarsa)-like expectation/integral.

The modified target update is:

$$ y = r + \gamma{Q}\_{\theta'}\left(s', \pi\_{\theta'}\left(s'\right) + \epsilon \right) $$

$$ \epsilon \sim \text{clip}\left(\mathcal{N}\left(0, \sigma\right), -c, c \right) $$

where the added noise is clipped to keep the target close to the original action. The outcome is an algorithm reminiscent of [Expected SARSA](https://paperswithcode.com/method/expected-sarsa), where the value estimate is instead learned off-policy and the noise added to the target policy is chosen independently of the exploration policy. The value estimate learned is with respect to a noisy policy defined by the parameter $\sigma$.

Target Policy Smoothing

Addressing Function Approximation Error in Actor-Critic Methods

Combines the channel attention of the widely known [spatial squeeze and channel excitation (SE)](https://paperswithcode.com/method/squeeze-and-excitation-block) block and the spatial attention of the [channel squeeze and spatial excitation (sSE)](https://paperswithcode.com/method/channel-squeeze-and-spatial-excitation#) block to build a spatial and channel attention mechanism for image segmentation tasks.

Concurrent Spatial and Channel Squeeze & Excitation

Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks

**Sparse Sinkhorn Attention** is an attention mechanism that reduces the memory complexity of the [dot-product attention mechanism](https://paperswithcode.com/method/scaled) and is capable of learning sparse attention outputs. It is based on the idea of differentiable sorting of internal representations within the self-attention module. SSA incorporates a meta sorting network that learns to rearrange and sort input sequences. Sinkhorn normalization is used to normalize the rows and columns of the sorting matrix. The actual SSA attention mechanism then acts on the block sorted sequences.

Sparse Sinkhorn Attention

**Hunger Games Search (**HGS**)** is a general-purpose population-based optimization technique with a simple structure, special stability features and very competitive performance to realize the solutions of both constrained and unconstrained problems more effectively. HGS is designed according to the hunger-driven activities and behavioural choice of animals. This dynamic, fitness-wise search method follows a simple concept of “Hunger” as the most crucial homeostatic motivation and reason for behaviours, decisions, and actions in the life of all animals to make the process of optimization more understandable and consistent for new users and decision-makers. The Hunger Games Search incorporates the concept of hunger into the feature process; in other words, an adaptive weight based on the concept of hunger is designed and employed to simulate the effect of hunger on each search step. It follows the computationally logical rules (games) utilized by almost all animals and these rival activities and games are often adaptive evolutionary by securing higher chances of survival and food acquisition. This method's main feature is its dynamic nature, simple structure, and high performance in terms of convergence and acceptable quality of solutions, proving to be more efficient than the current optimization methods. 

Implementation of the HGS algorithm is available at [https://aliasgharheidari.com/HGS.html](https://aliasgharheidari.com/HGS.html).

In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization. We propose OFA, a Task-Agnostic and Modality-Agnostic framework that supports Task Comprehensiveness. OFA unifies a diverse set of cross-modal and unimodal tasks, including image generation, visual grounding, image captioning, image classification, language modeling, etc., in a simple sequence-to-sequence learning framework. OFA follows the instruction-based learning in both pretraining and finetuning stages, requiring no extra task-specific layers for downstream tasks. In comparison with the recent state-of-the-art vision & language models that rely on extremely large cross-modal datasets, OFA is pretrained on only 20M publicly available image-text pairs. Despite its simplicity and relatively small-scale training data, OFA achieves new SOTAs in a series of cross-modal tasks while attaining highly competitive performances on uni-modal tasks. Our further analysis indicates that OFA can also effectively transfer to unseen tasks and unseen domains. Our code and models are publicly available at https://github.com/OFA-Sys/OFA.

OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

**PipeMare** is an asynchronous (bubble-free) pipeline parallel method for training large neural networks. It involves two main techniques: learning rate rescheduling and discrepancy correction.

PipeMare

PipeMare: Asynchronous Pipeline Parallel DNN Training

**DROID-SLAM** is a deep learning based SLAM system. It consists of recurrent iterative updates of camera pose and pixelwise depth through a Dense Bundle Adjustment layer. This layer leverages geometric constraints, improves accuracy and robustness, and enables a monocular system to handle stereo or RGB-D input without retraining. It builds a dense 3D map of the environment while simultaneously localizing the camera within the map.

DROID-SLAM

DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras

**ZoomNet** is a 2D human whole-body pose estimation technique. It aims to localize dense landmarks on the entire human body including face, hands, body, and feet. ZoomNet follows the top-down paradigm. Given a human bounding box of each person, ZoomNet first localizes the easy-to-detect body keypoints and estimates the rough position of hands and face. Then it zooms in to focus on the hand/face areas and predicts keypoints using features with higher resolution for accurate localization. Unlike previous approaches which usually assemble multiple networks, ZoomNet has a single network that is end-to-end trainable. It unifies five network heads including the human body pose estimator, hand and face detectors, and hand and face pose estimators into a single network with shared low-level features.

ZoomNet

Whole-Body Human Pose Estimation in the Wild

**Convolution-enhanced image Transformer** (**CeiT**) combines the advantages of CNNs in extracting low-level features, strengthening locality, and the advantages of Transformers in establishing long-range dependencies. Three modifications are made to the original Transformer: 1) instead of the straightforward tokenization from raw input images, we design an **Image-to-Tokens** (**I2T**) module that extracts patches from generated low-level features; 2) the feed-froward network in each encoder block is replaced with a **Locally-enhanced Feed-Forward** (**LeFF**) layer that promotes the correlation among neighbouring tokens in the spatial dimension; 3) a **Layer-wise Class token Attention** (**LCA**) is attached at the top of the Transformer that utilizes the multi-level representations.

CeiT

Incorporating Convolution Designs into Visual Transformers

**Conditional Relation Network**, or **CRN**, is a building block to construct more sophisticated structures for representation and reasoning over video. CRN takes as input an array of tensorial objects and a conditioning feature, and computes an array of encoded output objects. Model building becomes a simple exercise of replication, rearrangement and stacking of these reusable units for diverse modalities and contextual information. This design thus supports high-order relational and multi-step reasoning.

Hierarchical Conditional Relation Networks for Video Question Answering

**DU-GAN** is a [generative adversarial network](https://www.paperswithcode.com/methods/category/generative-adversarial-networks) for LDCT denoising in medical imaging. The generator produces denoised LDCT images, and two independent branches with [U-Net](https://paperswithcode.com/method/u-net) based discriminators perform at the image and gradient domains. The U-Net based discriminator provides both global structure and local per-pixel feedback to the generator. Furthermore, the image discriminator encourages the generator to produce photo-realistic CT images while the gradient discriminator is utilized for better edge and alleviating streak artifacts caused by photon starvation.

DU-GAN

DU-GAN: Generative Adversarial Networks with Dual-Domain U-Net Based Discriminators for Low-Dose CT Denoising

**LIMix**, or **Lifelong Infinite Mixture**, is a lifelong learning model which grows a mixture of models to adapt to an increasing number of tasks.  LIMix can automatically expand its network architectures or choose an appropriate component to adapt its parameters for learning a new task, while preserving its previously learnt information. Knowledge is incorporated by means of Dirichlet processes by using a gating mechanism which computes the dependence between the knowledge learnt previously and stored in each component, and a new set of data. Besides, a Student model is trained which can accumulate cross-domain representations over time and make quick inferences.

LIMix

Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process

**Embedded Dot Product Affinity** is a type of affinity or self-similarity function between two points $\mathbb{x\_{i}}$ and $\mathbb{x\_{j}}$ that uses a dot product function in an embedding space:

$$ f\left(\mathbb{x\_{i}}, \mathbb{x\_{j}}\right) = \theta\left(\mathbb{x\_{i}}\right)^{T}\phi\left(\mathbb{x\_{j}}\right) $$

Here $\theta\left(x\_{i}\right) = W\_{θ}x\_{i}$ and $\phi\left(x\_{j}\right) = W\_{φ}x\_{j}$ are two embeddings.

The main difference between the dot product and [embedded Gaussian affinity](https://paperswithcode.com/method/embedded-gaussian-affinity) functions is the presence of [softmax](https://paperswithcode.com/method/softmax), which plays the role of an activation function.

Embedded Dot Product Affinity

Non-local Neural Networks

**Sample Redistribution** is a [data augmentation](https://paperswithcode.com/methods/category/image-data-augmentation) technique for face detection which augments training samples based on the statistics of benchmark datasets via large-scale cropping. During training data augmentation, square patches are cropped from the original images with a random size from the set $[0.3,1.0]$ of the short edge of the original images. To generate more positive samples for stride 8, the random size range is enlarged from $[0.3,1.0]$ to $[0.3,2.0]$. When the crop box is beyond the original image, average RGB values fill the missing pixels.

The motivation is that for efficient [face detection](https://paperswithcode.com/task/face-detection) under a fixed VGA resolution (i.e. 640×480), most of the faces (78.93%) in [WIDER FACE](https://paperswithcode.com/dataset/wider-face-1) are smaller than 32×32 pixels, and thus they are predicted by shallow stages. To obtain more training samples for these shallow stages, Sample Redistribution (SR) is used.

AI Glossary

Gradient Harmonizing Mechanism C

MT-PET

Domain Adaptative Neighborhood Clustering via Entropy Optimization

Inception-ResNet-v2-B

Target Policy Smoothing

Concurrent Spatial and Channel Squeeze & Excitation (scSE)

Sparse Sinkhorn Attention

Hunger Games Search

OFA

PipeMare

DROID-SLAM

ZoomNet

Convolution-enhanced image Transformer

Conditional Relation Network

DU-GAN

Lifelong Infinite Mixture

Embedded Dot Product Affinity

Sample Redistribution