OFA: Towards Building a One-For-All Model

GitHub Pages · 3 years ago · 9

Researchers proposed OFA, a unified multimodal pretrained model that combines vision and language understanding and generation tasks in a single Transformer-based architecture using instruction-based multitask pretraining. The largest OFA model (930M parameters) achieved comparable performance to the 80B-parameter Flamingo model on visual question answering and state-of-the-art results on image captioning and visual grounding tasks. The unified approach enables the model to perform compositional generalization on unseen tasks by changing only the instruction prompt, and the team released models ranging from 33M to 930M parameters with publicly reproducible training.