Unlocking the potential of vision language models on satellite imagery through fine-tuning

Mistral AI · 1 year ago · 25

Mistral fine-tuned its Pixtral-12B vision model on satellite imagery classification using Low-Rank Adaptation (LoRA), a technique that efficiently adapts model weights without retraining the entire network. Overall accuracy improved from 0.56 to 0.91, while hallucinations dropped from 5% to 0.1%, at a cost of under $10 using 8,000 training samples across 30 image categories. This demonstrates that domain-specific fine-tuning enables vision models to handle nuanced satellite imagery tasks where general-purpose models struggle with subtle visual distinctions.