# Hugging Face says VocabX cut inference cost by 77%

- Published: 2026-10-21T17:35:36.000Z
- Updated: 2026-10-21T22:09:13.418Z
- Section: Development
- Topics: Research
- Page: https://aidailyjournal.com/en/n/2026/10/21/hugging-face-says-vocabx-cut-inference-cost-by-77/
- Sources: [Hugging Face](https://huggingface.co/blog/Dharma-AI/vocabx-vocabulary-specialization)

## Summary

Hugging Face says it cut the measured inference cost of an automated Portuguese essay grader by 77% without changing the Qwen3-4B base model, the data or the training recipe. The system produces JSON evaluations and used the model’s original tokenizer, with 151,669 entries, designed for many languages and types of text. The company says that vocabulary fragmented recurring phrases and made the model calculate scores for many entries rarely used in the task.

The VocabX method rebuilds the vocabulary around the cost of each token rather than only its frequency. Hugging Face says the reduction is especially relevant during generation, when small models make successive passes and are limited by memory bandwidth. The figures come from the current version of a paper still under review and may change before publication.

## Why it matters (editorial interpretation)

If the current paper’s results hold, rebuilding the vocabulary around token cost could reduce inference costs without changing the base model, data, or training recipe—an approach relevant to applications that generate structured outputs. The figures may still change before publication.

_Summary written by AI Daily Journal in its own words, based on the sources above. For details, read the original articles._