MIT researchers say GPT-6 Astra improved Qwen by editing 0.016% of its weights
🎧 Listen to this story
MIT CSAIL researchers say they used GPT-6 Astra to directly edit 0.016% of Qwen3.5-0.8B’s weights, without retraining it, to correct certain mathematical reasoning loops. According to Xataka, Qwen’s performance on MATH-500 rose from 7.9% to 48.1%. MIT called the approach Weight-Space Programming.
The method could offer a cheaper way to improve smaller models, but the work is still preliminary and has not been peer-reviewed. According to Xataka, it produced almost no gain on Google’s Gemma 3 270M, which was less affected by those loops—suggesting that the technique depends on having something to remove. The researchers also point to the future possibility of models applying improvements to themselves, although that remains distant.
Why it matters · editorial interpretation
The result suggests that editing a tiny fraction of a model’s weights could be an alternative to retraining for correcting specific failures in smaller models, but the evidence is still preliminary and not peer-reviewed. The limited gain on Gemma indicates that usefulness depends on the type of problem present in the model.
Sources
Research