Comparing the Performance of LLMs: A Deep Dive into Roberta, Llama 2, and Mistral for Disaster Tweets Analysis with Lora

Hugging Face · 2 years ago · 43

Researchers compared three language models—RoBERTa, Llama 2, and Mistral 7B—fine-tuned with LoRA (Low-Rank Adaptation) for classifying disaster-related tweets. The study used 7,613 training tweets split into train (6,090), validation (1,523), and test (3,263) samples, with class weights of 1.16 for positive and 0.88 for negative examples to address imbalance. Fine-tuning was performed with LoRA to reduce trainable parameters while maintaining performance on the sequence classification task.