Viet-Anh on Software Logo

What is: RIFE?

SourceRIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

RIFE, or Real-time Intermediate Flow Estimation is an intermediate flow estimation algorithm for Video Frame Interpolation (VFI). Many recent flow-based VFI methods first estimate the bi-directional optical flows, then scale and reverse them to approximate intermediate flows, leading to artifacts on motion boundaries. RIFE uses a neural network named IFNet that can directly estimate the intermediate flows from coarse-to-fine with much better speed. It introduces a privileged distillation scheme for training intermediate flow model, which leads to a large performance improvement.

In RIFE training, given two input frames I0,I1I_{0}, I_{1}, we directly feed them into the IFNet to approximate intermediate flows Ft0,Ft1F_{t \rightarrow 0}, F_{t \rightarrow 1} and the fusion map MM. During training phase, a privileged teacher refines student's results to get Ft0Tea,Ft1TeaF_{t \rightarrow 0}^{T e a}, F_{t \rightarrow 1}^{T e a} and MTea M^{\text {Tea }} based on ground truth ItI_{t}. The student model and the teacher model are jointly trained from scratch using the reconstruction loss. The teacher's approximations are more accurate so that they can guide the student to learn.