Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3

MarkTechPost · 1 month ago · 48

The article walks through an end-to-end supervised fine-tuning pipeline that streams and parses the XYZ-Aquila-SFT tool-use dataset, converts tool schemas into Qwen3-compatible ChatML, and fine-tunes Qwen3-0.6B using PEFT/LoRA with assistant-only loss masking. It sets MAX_SEQ_LEN to 2048 and runs training for MAX_STEPS=30. Tool-call prediction is evaluated before and after training, and the transformed dataset plus corpus statistics are exported for follow-up experiments.