OpenAI's Noam Brown on multi-agent systems: 10,000 agents deserve at most 10% of the credit for the Millennium Prize breakthrough

Summary

Noam Brown, an OpenAI researcher and core author of o1, discussed his current focus, multi-agent systems, in a recent podcast. He said OpenAI previously used 10,000 agents over 88 hours and 130 billion tokens to solve a Millennium Prize mathematics problem. Brown was explicit that the result came from having a general, powerful model, and that he would not even attribute 10% of the credit to multi-agent methods, which he feels have received disproportionate praise because they are eye-catching. Multi-agent systems simply scale test-time compute in parallel rather than serially, at some efficiency loss since agents do not share full context. His testing reached only 16 agents: four agents solve a problem about twice as fast for roughly double the cost, and 16 still improve performance with slightly lower efficiency. Speedups are somewhat sublinear and domain-dependent, and ablation experiments at 1,000-agent scale would be unaffordable.