Viet-Anh on Software Logo

What is: Data-efficient Image Transformer?

SourceTraining data-efficient image transformers & distillation through attention
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

A Data-Efficient Image Transformer is a type of Vision Transformer for image classification tasks. The model is trained using a teacher-student strategy specific to transformers. It relies on a distillation token ensuring that the student learns from the teacher through attention.