Marcus on AI: Checking the math behind OpenAI's latest headline

Summary

OpenAI scored a big result yesterday, with respect to an Erdos problem: Clearly impressive. But as with so much else, it should be viewed with skepticism. In an email to me this morning, Cal Newport made a number of good points that he said I could share, summarizing both what was found and some limitations: OpenAI used a new reasoning model (not yet released) to help identify a counterexample that disproved a conjecture from discrete geometry first proposed by Paul Erdos 80 years ago. The model was tuned for so-called chain-of-thought reasoning where the model endlessly “thinks out loud” about whatever it is trying to solve, an approach that lets you approximate something like memory and dynamic computation using LLMs, which are otherwise static and feed-forward. Professional mathematicians identified the counterexample from within a long transcript of the model’s reasoning, and then extracted the key parts and rewrote it as a more succinct proof in a more standard style. How was the model able to solve something that human mathematicians had failed to do? In a companion article released by OpenAI, the mathematician Thomas Bloom, who reviewed the full model output, identified the factors that came together to make this counterexample ripe for LLM-aided discovery. He noted that though the conjecture is old, those who have worked on it have largely shared Erdos’s original belief that it was true and therefore focused on trying to solve it. What the LLM-based tool did instead was to systematically apply and extend existing techniques in search of evidence that the conjecture was false. Here’s Bloom: “[the AI’s] success here echoes previous achievements: it often produces the most surprising results by persevering down the paths that a human may have dismissed as not worth their time to explore, combining superhuman levels of patience with familiarity with a vast array of technical machinery.” A few observations of my own: (1) Non-mathematicians might not be familiar with the degree to which LLM-technology has been combined with existing computer-aided math tools in recent years to seek new math results through the systematic and patient exploration of techniques and corners of problem spaces that are too exhausting to interest most human mathematicians. The real technical headline of the new OpenAI result, therefore, is that chain-of-thought reasoning was abl…