Researchers criticize how OpenAI chooses to release its new scientific results

US artificial intelligence company OpenAI announced breakthroughs in mathematics, computer science and physics on Monday, along with what it describes as a new approach to sharing them with the scientific community. Researchers who spoke to The Verge take the results seriously, but are frustrated that the initial scrutiny is being left to experts chosen by the company.

Three results, three release strategies

OpenAI says each result has a different release plan, following advice from outside "shepherds." For the mathematics result, it has published the AI-generated output, leaving researchers free to organize their own efforts to explain it. For the computer science finding, it will assemble an external group of researchers to explain the result. The physics result has not been released at all, and the company says it will defer to outside experts on how to release the materials.

The mathematics result purports to resolve the Kakeya conjecture in all dimensions. Roughly, the problem asks how little space is needed to rotate a needle in every direction. A proof of the three-dimensional version was among the achievements for which NYU mathematician Hong Wang received a Fields Medal this year. Brendan Hassett, a mathematics professor at Brown University, said the claim "looks well supported," though he has not yet checked the Lean formalization, a computational way of verifying results.

The physics result concerns quantum chromodynamics, which describes the strong nuclear force, and its link to Yang-Mills theory. That theory underlies one of the seven Millennium Prize problems, although OpenAI's work does not address the unresolved "mass gap." The computer science result claims to disprove the exponential time hypothesis, which is closely related to the P versus NP problem.

Criticism over restricted access

Constantin Kogler of the Institute for Advanced Study welcomed the smaller batch of three results compared with the hundreds released earlier, but said it makes no sense for only a few distinguished scientists to look first. Andreas Karch, a physics professor at the University of Texas at Austin, called the hand-picked group "a disaster." The Advisory Group on Mathematics and Artificial Intelligence (AGMAI) had advised AI companies a day earlier not to adopt such a practice, and later urged OpenAI to publish its physics outputs without delay.

Stanford mathematician Jared Duker Lichtman suggested that no approach will satisfy everyone. Xiaoyu He, a professor at the Georgia Institute of Technology, said the deeper issue is that fundamental results are being solved so quickly without human contribution, and that the papers are hard to read and verify.