Laboratory Differences May Mislead Scientific AI Models, Researchers Warn

Summary

UNIVERSITY PARK, Pa. — Artificial intelligence could dramatically accelerate the search for technologies that transform carbon dioxide into useful fuels, but a new multi-laboratory study shows that even the most advanced model can be undermined by something as simple as inconsistent stirring. Researchers from four U.S. laboratories have demonstrated that experimental variability can substantially alter measurements of catalyst performance, creating a serious challenge for scientists who want to use real-world laboratory data to train artificial intelligence and machine-learning systems. The work, led by scientists at SLAC National Accelerator Laboratory with contributions from Penn State, Stanford University and the University of California, Santa Barbara, was published in Nature Catalysis. The study focused on a rhodium-based catalyst used in carbon dioxide hydrogenation, a chemical process in which carbon dioxide reacts with hydrogen to produce carbon-containing compounds. In the experiments, the catalyst was designed to promote the formation of carbon monoxide, an important intermediate that can be converted into synthetic fuels and other industrial chemicals. Methane was also produced as an undesirable side product. By comparing results from laboratories that followed a shared experimental protocol, the researchers hoped to measure how reproducible catalyst testing would be when performed by independent teams. The experiment was structured as a “round-robin” study, in which the same material and testing instructions are distributed among multiple laboratories so that differences in performance can be traced to procedures, equipment or operating conditions rather than to the catalyst itself. The motivation is closely tied to the growing use of artificial intelligence in chemistry and chemical engineering. A well-trained model can examine relationships among catalyst composition, temperature, pressure, reaction time and product yield, then predict which combinations are most likely to work. Instead of testing thousands of possibilities experimentally, researchers could use the model to identify a smaller set of promising candidates and validate those predictions in the laboratory. Such an approach could reduce the time, cost and energy required to develop catalysts for carbon capture, fuel production and industrial chemical manufacturing. However, machine…