How Specializing SLM Vocabulary Cut Measured Inference Cost by 77% With VocabX
How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications
- Who
- Dharma AI
- What
- previews results from the VocabX paper on cost-weighted vocabulary specialization
What happened
Dharma AI previews VocabX, a cost-weighted vocabulary adaptation method that cut measured inference cost by 77.0% in its Portuguese essay grader built on Qwen3-4B. With the same base model, data and training recipe, average responses shrank from 350 to 101 tokens and the bench score rose from 0.703 to 0.810. The results come from a paper still under review and may change before publication.
Why it matters
The grader experiment illustrates why vocabulary compression should preserve decision-bearing tokens, since separating grades from surrounding JSON improved both measured cost savings and assessment quality.
The story so far
Dharma AI previewed VocabX, a vocabulary adaptation method that it said reduced measured inference cost by 77.0% in a Portuguese essay grader built on Qwen3-4B. The results come from a paper still under review and may change before publication. VocabX specializes vocabulary around the cost of each token rather than its frequency. Dharma AI reported that, with the same base model, data and training recipe, average responses shortened from 350 to 101 tokens while the benchmark score rose from 0.703 to 0.810. Those measurements apply to its Portuguese essay grader.
Keep reading
Latest daily · Issue 28Mistral AI releases Voxtral 3 speech models and a voice agent SDKTrending now
- 1Mistral and NVIDIA Announce Industrial Engineering AI Collaboration and Open Benchmarks
- 2Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
- 3PHP releases security updates after OpenAI says its models helped researchers find critical flaws
- 4DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits
Tomorrow at 08:00, a briefing like this arrives in your inbox.
The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue · 2026-10-21