How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications
Hugging Face
VocabX rebuilt the tokenizer vocabulary for a production essay-grading system on Qwen3-4B without changing the base model, training data, or training recipe. The output-layer vocabulary shrank from about 152,000 to about 65,000 rows. In the measured grader tests, average response length fell from 350 to 101 tokens and measured inference cost dropped 77% while bench score rose from 0.703 to 0.810.