Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3
MarkTechPost Sana Hassan
A tutorial shows how to fine-tune Qwen3-0.6B on XYZ-Aquila-SFT for tool use. It keeps the model's reasoning text intact instead of letting the chat template eat it.
Based on reporting by MarkTechPost, Sana Hassan — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
MarkTechPost’s tutorial walks through a full supervised fine-tuning setup for tool-calling data, from the raw XYZ-Aquila-SFT rows all the way to a LoRA-tuned Qwen3-0.6B. The stack is familiar enough — Hugging Face Transformers, PyTorch, PEFT, Accelerate — but the interesting part is the data work, not the plumbing.
The first step is to stream a slice of the dataset, inspect the schema, and parse each trajectory into something usable. The code pulls out questions, answers, declared tool-call counts, message roles, embedded tool schemas, and the assistant’s `<think>` blocks and `<tool_call>` payloads. It also uses a nesting-safe JSON scanner, which is the sort of boring detail that saves a lot of pain when tool arguments themselves contain structured objects.
From there, the tutorial checks the corpus itself. It counts tool calls per trajectory, message depth, character length, tool frequency, and argument keys, then plots the distributions. One useful result is a consistency check: the parsed tool-call count is compared against the dataset’s own declared number of tool calls, so the pipeline can tell you whether its parser is actually matching the source data.
The formatting step is where the tutorial takes a clear position. Instead of relying on Qwen3’s built-in chat template, it renders ChatML manually so the assistant’s reasoning text does not disappear from all but the last turn. That matters because the point of this dataset is not just to predict a function name; it is to learn the full tool-use pattern, including the reasoning and observations wrapped around the call.
Training then uses a custom PyTorch dataset and collator, assistant-only loss masking, a max sequence length of 2048, and a simple truncate policy for long examples. The run settings are small — 30 training steps, gradient accumulation of 8, learning rate 1e-4, LoRA rank 16 — but the tutorial also evaluates tool-call prediction before and after training and exports both the transformed dataset and corpus stats for later experiments.
My take — AI-written commentary, not fact-checked reporting
This is the right kind of unglamorous AI work: less magical thinking, more making sure the training signal isn’t quietly wrecked by a template. Too many tool-calling demos celebrate the model while the dataset is doing half the job in a format nobody checked. Open models keep winning when people actually do the boring part properly.
Read more about this at: MarkTechPost
