Image GPT
OpenAI Blog
OpenAI trained a large transformer model on pixel sequences to generate images, demonstrating that the same architecture used for language can produce coherent image completions and samples. The model's best variant achieved image classification accuracy competitive with top convolutional networks in unsupervised learning benchmarks. This suggests transformers can serve as general-purpose models for both language and vision tasks without architecture-specific modifications.
Why it matters
We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting.
