Viet-Anh on Software Logo

What is: MoCo v2?

SourceImproved Baselines with Momentum Contrastive Learning
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

MoCo v2 is an improved version of the Momentum Contrast self-supervised learning algorithm. Motivated by the findings presented in the SimCLR paper, authors:

  • Replace the 1-layer fully connected layer with a 2-layer MLP head with ReLU for the unsupervised training stage.
  • Include blur augmentation.
  • Use cosine learning rate schedule.

These modifications enable MoCo to outperform the state-of-the-art SimCLR with a smaller batch size and fewer epochs.