Microsoft and Hugging Face say ThinkingBox improved Qwen3.8-27B success

Microsoft and Hugging Face say ThinkingBox turns executable checks of a database’s final state into rewards for training agents. In an experiment using a fixed pool of tasks selected for difficulty relative to the model’s policy, fine-tuning Qwen3.8-27B for 50 GRPO steps raised average success from 51.70% to 60.78% and increased the number of tasks passing all 20 attempts from 38 to 107. The result came with a small decline in tasks solved at least once, from 453 to 446.

ThinkingBox-Training is now available on GitHub. In ThinkingBox-Bench, each of 507 tasks gives the agent a backend and tools before comparing the actual database changes with the expected outcome. In one example cited in the post, the fine-tuned model preserved booking and payment state in all 20 attempts on a task requiring manager approval for a room downgrade.