Conjecture Machines: AI agents and the new validation bottleneck in science

Summary

Conjecture Machines: AI agents and the new validation bottleneck in science Over the past year, agents have transformed what it means to code. Scientific research may be next — from proposing novel hypotheses, to designing experiments, to discovering algorithms that improve on the best that humans have designed. This raises urgent questions for policymakers and science funders, the biggest being how to validate the coming wave of AI-generated ideas. New approaches are also needed to ensure that scientists can access agents, that datasets are agent-ready, and that peer review is kept afloat. We sat down with 10 researchers and engineers from Google DeepMind to try to figure out what comes next. Microbiologist José Penadés and his team at Imperial College London took most of a decade to work out how a family of superbugs spreads antibiotic resistance. The result was unpublished, known only inside his lab. Then, in 2024, he described the problem to Co-Scientist, an AI agent from Google DeepMind. Within two days, Co-Scientist returned five potential explanations, ranked in priority. Number 1 was the same hypothesis Penadés’s team had spent so long working to prove: that some superbugs acquire tails from viruses and use them as “keys” to jump between host species. Stunned, his first thought was that his computer had been compromised, so he emailed Google to check. Confirmed: no peeks taken. If he could have gone back in time, that insight would have saved his team years. An AI agent such as Co-Scientist is a large language model (LLM)- based system given a goal and the tools to pursue it. Unlike a query-answering chatbot, an agent can plan how to achieve the goal you give it, breaking it down into steps, running multiple subagents and processes in parallel, and detecting and correcting errors as it goes. It can also engage with the wider world, for example by calling online databases and tools, or writing code to operate robots. The arrival of AI agents is timely. Researchers face a rapidly growing “burden of knowledge”, with more to learn before they can meaningfully contribute. The questions that count in drug discovery, climate modelling, materials design, and biology are becoming too complex and interdisciplinary for human teams to tackle at pace. Science depends in part on a social infrastructure that has gradually evolved: labs, teams, institutions, peer r…