Researchers say predictor improved agents that modify their own software
🎧 Listen to this story
Researchers from the University of Virginia, Microsoft Research and Tsinghua University developed the Improvement Dynamics Model (IDM), a component that tries to predict whether changes made by AI agents could worsen their performance. When it detects that risk, the system allows an additional review before the final evaluation. The method was incorporated into HyperAgents, which records previous versions to guide new changes to the agents’ code and instructions.
In an experiment with 166 programming tasks not used during the improvement or selection process, the initial agent succeeded in approximately 16% of cases. Selected HyperAgents versions without the predictor reached a median close to 22%, while the prediction method reached almost 30%. The results, based on five runs per configuration, were presented in a study published on arXiv and reflect the conditions of that experiment.
Why it matters · editorial interpretation
The approach suggests a way to reduce regressions when agents modify their own code or instructions by adding a review step before final evaluation. The observed advantage, however, is limited to the experiment’s conditions, its 166 tasks, and five runs per configuration.
Sources
ResearchAgentsCode