**TaxoExpan** is a self-supervised taxonomy expansion framework. It automatically generates a set of <query concept, anchor concept> pairs from the existing taxonomy as training data. Using such self-supervision data, TaxoExpan learns a model to predict whether a query concept is the direct hyponym of an anchor concept. TaxoExpan features: (1) a position-enhanced graph neural network that encodes the local structure of an anchor concept in the existing taxonomy, and (2) a noise-robust training objective that enables the learned model to be insensitive to the label noise in the self-supervision data.

**PipeDream-2BW** is an asynchronous pipeline parallel method that supports memory-efficient pipeline parallelism, a hybrid form of parallelism that combines data and model parallelism with input pipelining. PipeDream-2BW uses a novel pipelining and weight gradient coalescing strategy, combined with the double buffering of weights, to ensure high throughput, low memory footprint, and weight update semantics similar to data parallelism. In addition, PipeDream2BW automatically partitions the model over the available hardware resources, while respecting hardware constraints such as memory capacities of accelerators, and topologies and bandwidths of interconnects. PipeDream-2BW also determines when to employ existing memory-savings techniques, such as activation recomputation, that trade off extra computation for lower memory footprint.

The two main features are a double-buffered weight update (2BW) and flush mechanisms ensure high throughput. PipeDream-2BW
splits models into stages over multiple workers, and each stage is replicated an equal number of times (with data-parallel updates across replicas of the same stage).  Such parallel pipelines work well for models where each layer is repeated a fixed number of times (e.g., [transformer](https://paperswithcode.com/method/transformer) models).

PipeDream-2BW

Memory-Efficient Pipeline-Parallel DNN Training

TaxoExpan

TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network

**YOLOX** is a single-stage object detector that makes several modifications to [YOLOv3](https://paperswithcode.com/method/yolov3) with a  [DarkNet53](https://www.paperswithcode.com/method/darknet53) backbone. Specifically, YOLO’s head is replaced with a decoupled one. For each level of [FPN](https://paperswithcode.com/method/fpn) feature, we first adopt a 1 × 1 conv layer to reduce the feature channel to 256 and then add two parallel branches with two 3 × 3 conv layers each for classification and regression tasks respectively.

Additional changes include adding Mosaic and [MixUp](https://paperswithcode.com/method/mixup) into the augmentation strategies to boost YOLOX’s performance. The anchor mechanism is also removed so YOLOX is anchor-free. Lastly, SimOTA for label assignment  -- where label assignment is formulated as an optimal transport problem via a top-k strategy.

Source	TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com