**Semi-Supervised Knowledge Distillation** is a type of knowledge distillation for person re-identification that exploits weakly annotated data by assigning soft pseudo labels to YouTube-Human to improve models' generalization ability. SSKD first trains a student model (e.g. [ResNet](https://paperswithcode.com/method/resnet)-50) and a teacher model (e.g. ResNet-101) using labeled data from multi-source domain datasets. Then, SSKD develops an [auxiliary classifier](https://paperswithcode.com/method/auxiliary-classifier) to imitate the soft predictions of unlabeled data generated by the teacher model. Meanwhile, the student model is also supervised by hard labels and predicted soft labels by the teacher model for labeled data.

**RIFE**, or **Real-time Intermediate Flow Estimation** is an intermediate flow estimation algorithm for Video Frame Interpolation (VFI). Many recent flow-based VFI methods first estimate the bi-directional optical flows, then scale and reverse them to approximate intermediate flows, leading to artifacts on motion boundaries. RIFE uses a neural network named [IFNet](https://paperswithcode.com/method/ifnet) that can directly estimate the intermediate flows from coarse-to-fine with much better speed. It introduces a privileged distillation scheme for training intermediate flow model, which leads to a large performance improvement.

In RIFE training, given two input frames $I_{0}, I_{1}$, we directly feed them into the IFNet to approximate intermediate flows $F_{t \rightarrow 0}, F_{t \rightarrow 1}$ and the fusion map $M$. During training phase, a privileged teacher refines student's results to get $F_{t \rightarrow 0}^{T e a}, F_{t \rightarrow 1}^{T e a}$ and $M^{\text {Tea }}$ based on ground truth $I_{t}$. The student model and the teacher model are jointly trained from scratch using the reconstruction loss. The teacher's approximations are more accurate so that they can guide the student to learn.

RIFE

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

SSKD

Semi-Supervised Domain Generalizable Person Re-Identification

**Disentangled Attention Mechanism** is an attention mechanism used in the [DeBERTa](https://paperswithcode.com/method/deberta) architecture. Unlike [BERT](https://paperswithcode.com/method/bert) where each word in the input layer is represented using a vector which is the sum of its word (content) embedding and position embedding, each word in DeBERTa is represented using two vectors that encode its content and position, respectively, and the attention weights among words are computed using disentangled matrices based on their contents and relative positions, respectively. This is motivated by the observation that the attention weight of a word pair depends on not only their contents but their relative positions. For example, the dependency between the words “deep” and “learning” is much stronger when they occur next to each other than when they occur in different sentences.

Source	Semi-Supervised Domain Generalizable Person Re-Identification
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com