**MobileBERT** is a type of inverted-bottleneck [BERT](https://paperswithcode.com/method/bert) that compresses and accelerates the popular BERT model. MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. To train MobileBERT, we first train a specially designed teacher model, an inverted-bottleneck incorporated BERT_LARGE model. Then, we conduct knowledge transfer from this teacher to MobileBERT. Like the original BERT, MobileBERT is task-agnostic, that is, it can be generically applied to various downstream NLP tasks via simple fine-tuning. It is trained by layer-to-layer imitating the inverted bottleneck BERT.

**PointASNL** is a non-local neural network for point clouds processing It consists of two general modules: adaptive sampling (AS) module and local-Nonlocal (L-NL) module. The AS module first re-weights the neighbors around the initial sampled points from farthest point sampling (FPS), and then adaptively adjusts the sampled points beyond the entire point cloud. The AS module can not only benefit the feature learning of point clouds, but also ease the biased effect of outliers. The L-NL module capture the neighbor and long-range dependencies of the sampled point, and enables the learning process to be insensitive to noise.

PointASNL

PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling

MobileBERT

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

A **Panoptic FPN** is an extension of an [FPN](https://paperswithcode.com/method/fpn) that can generate both instance and semantic segmentations via FPN. The approach starts with an FPN backbone and adds a branch for performing semantic segmentation in parallel with the existing region-based branch for instance segmentation. No changes are made to the FPN backbone when adding the dense-prediction branch, making it compatible with existing instance segmentation methods. 

The new semantic segmentation branch achieves its goal as follows. Starting from the deepest FPN level (at 1/32 scale), we perform three upsampling stages to yield a feature map at 1/4 scale, where each upsampling stage consists of 3×3 [convolution](https://paperswithcode.com/method/convolution), group norm, [ReLU](https://paperswithcode.com/method/relu), and 2× bilinear upsampling. This strategy is repeated for FPN scales 1/16, 1/8, and 1/4 (with progressively fewer upsampling stages). The result is a set of feature maps at the same 1/4 scale, which are then element-wise summed. A final 1×1 convolution, 4× bilinear upsampling, and [softmax](https://paperswithcode.com/method/softmax) are used to generate the per-pixel class labels at the original image resolution. In addition to stuff classes, this branch also outputs a special ‘other’ class for all pixels belonging to objects (to avoid predicting stuff classes for such pixels).

Source	MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com