The hard parts of AI-assisted science

Summary

AI can already assist with a surprising amount of scientific work: search literature, analyze datasets, write code, and generate and test hypotheses. The harder problem is building systems scientists can steer as a research project unfolds, especially when new evidence changes which hypotheses to pursue, experiments to run, or questions to investigate next. That challenge was the focus of an August 27 event at Ai2 marking an expansion of our work with the Paul G. Allen Research Center at Providence Swedish Cancer Institute. We’ve written separately about the cancer research that grew out of that collaboration. At the event, that work served as a starting point for a broader look at what today’s scientific AI still lacks. The conversation included: Kelly Paulson, MD, PhD, Lead, Center for Immuno-Oncology, Paul G. Allen Research Center & Medical Director, Melanoma and Cutaneous Oncology, Providence Swedish Cancer Institute Bodhisattwa Prasad Majumder, Senior Research Scientist, Ai2 Sasha Stanton, MD, PhD, Medical Breast Oncologist & Lead, Cancer Immunoprevention Laboratory, Earle A. Chiles Research Institute, Providence Cancer Institute Kyle Travaglini, Assistant Investigator, Allen Institute for Brain Health Accelerator Abraham Flaxman, PhD, Professor of Health Metrics Sciences and Global Health, University of Washington & Institute for Health Metrics and Evaluation Stephen Salerno, PhD, Assistant Professor, WashU Bursky School of Public Health Hoifung Poon, PhD, Chief AI Officer, Recursion Across the presentations and panel, five challenges kept resurfacing. 1. Leaving room for the human element Majumder described scientific taste as the judgment that helps a researcher distinguish an interesting result from one that’s trivial, implausible, already understood, or unlikely to lead anywhere. Today’s AI-for-science systems don’t reliably make those distinctions on their own. The goal isn’t necessarily to transfer that judgment wholesale to an AI system. The challenge is giving researchers better ways to bring their expertise, priorities, and changing understanding of a problem into the system so AI can assist without deciding on its own which directions are worth pursuing. Ai2’s work with Providence researchers made that distinction clear—AutoDiscovery could surface hypotheses that looked surprising statistically but made little biological or clinical sense wi…