Researchers explore “reverse game theory” to control AI agents

AI safety researchers are studying “reverse game theory,” also known as mechanism design, to create rules that encourage desired behavior. According to MIT Technology Review, a preprint by MIT economist Stephen Morris and colleagues from Yale and Columbia proposes rewarding a model for predicting another model’s answer, making it harder for the first to pretend to be less capable; other ideas include setting agents against each other or pairing models with opposite biases and rewarding matching outputs.

Elad Hazan, a computer science professor at Princeton, proposes a system inspired by social norms, the justice system, and economic incentives: agents would alternate between solving problems and auditing others’ work. Solvers caught cheating would be punished, as would auditors that failed to detect cheating. The approach starts from the possibility that the internal workings of frontier models may never be fully understood.