A multidisciplinary team of UMBC researchers was recently awarded $3.8 million from the Defense Advanced Research Projects Agency (DARPA) to develop new computational methods for assessing the feasibility of scientific claims. The project is motivated by the speed and volume of new developments in science and the need for tools to help evaluate the soundness of new claims.
“Some scientific claims are peer reviewed by independent experts, but others, such as from company press releases, or papers posted to pre-prints sites, are not,” says Frank Ferraro, an associate professor of computer science and lead researcher on the award. And even when science is peer-reviewed, there can still be errors, Ferraro notes. For example, recent research highlighting the danger of flame retardant chemicals leaching from black plastic into food was widely reported on, before it was discovered that an error in the paper led to a significant overstating of the exposure risk.
An AI science assistant
The idea of an AI assistant specifically designed to tackle complex research questions has been catching on recently. OpenAI, the creators of ChatGPT, recently debuted their “DeepResearch” tool, which the company says can use reasoning to synthesize large amounts of online information and complete multi-step research tasks.
Ferraro says the tool he and his colleagues hope to develop should stand-out from other approaches by employing a strongly iterative process, necessary to really take apart and analyze claims. The tool will break down a scientific claim into constituent sub-claims, and apply a wide range of evidence and reasoning approaches to assess the feasibility of each one.
The team, which includes UMBC faculty Tejas Gokhale, computer science and electrical engineering, and Tyler Josephson, chemical, biochemical, and environmental engineering, as well as colleagues from Stony Brook University, the University of Texas at Austin, and the University of Cambridge, will develop and test the tool on three leading areas of scientific research: materials science, AI, and quantum computing.
“It’s a little meta that the AI will be analyzing AI,” Ferraro says. More than 20 UMBC undergrads, grad students, and post-docs will take part in the project. “They’ll get opportunities to network with peers on a highly relevant topic,” Ferraro says. “They’ll be helping guide the national conversation on these issues.”