Vision Transformer applies pure transformers to image patches for classification

Vision Transformer (ViT) treats images as sequences of patches fed into a standard Transformer, achieving state-of-the-art accuracy on ImageNet and other benchmarks when pre-trained on large datasets, with less compute than comparable CNNs.

Based on the paper An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale by Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn and 8 others

Applying Transformers to vision requires handling large image inputs efficiently

Transformers have become the dominant architecture in natural language processing due to their scalability and ability to model long-range dependencies via self-attention.

In computer vision, convolutional neural networks (CNNs) remain dominant because images are large and applying full self-attention to all pixels is computationally prohibitive due to quadratic scaling with input size.

Previous attempts to use attention in vision either combine it with CNNs or restrict attention to local neighborhoods or specialized patterns, limiting scalability or requiring complex engineering.

You have read the opening. The rest is free with a Bezen account, and no card is needed.

Create a free account Log in

In the rest of this paper

How the method works, step by step, with a flow chart
Three figures from the paper, and how to read them
How it was tested, what was found, and where it falls short

Applying Transformers to vision requires handling large image inputs efficiently

Sign in to read the whole paper