Comparing the Performance of LLMs: A Deep Dive into Roberta, Llama 2, and Mistral for Disaster Tweets Analysis with Lora
Hugging Face Blog
Researchers compared three language models—RoBERTa, Llama 2, and Mistral 7B—fine-tuned with LoRA (Low-Rank Adaptation) for classifying disaster-related tweets. The study used 7,613 training tweets split into train (6,090), validation (1,523), and test (3,263) samples, with class weights of 1.16 for positive and 0.88 for negative examples to address imbalance. Fine-tuning was performed with LoRA to reduce trainable parameters while maintaining performance on the sequence classification task.
