Fine-Tuning Small Open-Source LLMs to Outperform Large Closed-Source Models by 60% on Specialized Tasks

Together AI · 1 year ago · 47

Parsed and Together AI demonstrated that a fine-tuned 27-billion-parameter Gemma model outperforms Claude Sonnet 4 on healthcare clinical scribing by 60% while requiring 10-100 times less compute. The fine-tuning used tens of thousands of task-specific examples paired with a multi-layered evaluation framework that decomposed clinical documentation into granular binary checks across four dimensions: clinical soundness, source fidelity, coverage, and style conformance. This approach enables healthcare companies and other specialized domains to replace expensive proprietary models with smaller, cheaper, and more transparent fine-tuned open-source alternatives optimized for their specific tasks.