OpenAI’s Math: Fast Generation, Slow Verification

OpenAI's math release has been public since October 6. The first bill for checking it has already arrived.

Open github.com/openai/math and the first thing you notice is what's missing: no benchmark harness or one-command way to reproduce the evaluation.

Instead, you find mathematics: manuscripts sorted into 372 “families,” a Lean library, ten summaries of a model's reasoning, and a README saying an unreleased internal OpenAI model produced the collection.

The release launched on October 6 with 722 manuscripts. As I write on October 9, it holds 719. We'll get to the three that left.

OpenAI says the average result used the equivalent of roughly three hours of ChatGPT Pro thinking compute. The README doesn't say whether a “result” means a manuscript or a family, so I won't multiply that out. For scale, give each paper one week of a specialist's attention—a stingy allowance for serious research mathematics—and reviewing 719 papers would take about fourteen years.

Writing got cheap and checking didn't. That gap is what this repository is really about, and within a day of release it started to show.

Three papers gone in a day

The README promises that corrections will be recorded as new versions, with old versions kept. On October 7, the repo's history.md showed what that looks like.

A sign error in “Algebraicity of Weil classes on split abelian eightfolds” broke a cancellation argument and a construction that two other papers relied on. All three were withdrawn: that one, a paper on Kuga–Satake correspondences for K3 surfaces, and “The rational Hodge conjecture for products of K3 surfaces.” That's the drop from 722 to 719.

The same entry lists fourteen more manuscripts revised, with proof repairs, corrected statements and clearer hypotheses. Another thirteen were updated to cite the revised editions. Withdrawn papers carry notices explaining the gap and linking to archived manuscripts.

Give OpenAI credit here. The correction process worked quickly and as advertised. The changelog doesn't say who found the error or how, which I'd like to know.

The cascade matters: the README says some outputs build on earlier results, and one flawed paper took two dependents down with it. The structure that makes the collection look large also makes it brittle.

There's a related question around the Hodge conjecture result for CM abelian varieties, which the README flags as an exception to the standard procedure. Weil classes are a classic hard case in this area, as I understand it. That overlap is a reason to look closely, not evidence that anything is wrong. The changelog doesn't establish whether the separate CM-abelian-varieties result is affected.

How this got staged

• September 8: OpenAI announces what it calls a solution to a Navier–Stokes problem, separate from this repository.

• September 11: 25 Fields Medalists publish “A Severe Misalignment of AI in Mathematics,” objecting to open problems being used as AI benchmarks.

• September 21: OpenAI announces an independent advisory group at the Institute for Advanced Study, with nine mathematicians including Timothy Gowers and Edward Witten (The Next Web). The group can advise and comment publicly, but it has no decision-making power.

• September 29: the group publishes release guidelines, informed by more than 600 replies. The survey concerned a specific situation: OpenAI announcing many results without giving details.

• October 6: the repository goes public. The advisory group says the release begins, rather than completes, the work of human understanding, and leaves it to the mathematical community to judge compliance.

• October 8: TechCrunch reports that the release deviated from the guidelines.

So this is partly a math release and partly a governance experiment. The guidelines give us a checklist to hold it against.

A partial scorecard

I read the advisory group's recommendations directly. They set technical norms for releasing results not yet understood, call for funding human understanding, and encourage broad model access.

This is my reading, not an official verdict. Items marked “not checked” require reading the manuscripts themselves.

OpenAI has supplied many technical artifacts at least partly, including Comparator challenge files, copyright headers, and formalization.yaml. But it missed the group's opening request, hasn't named the model, and has no neutral repository yet. Its call for broad access to publicly available models also remains unfulfilled: the model behind this collection isn't public.

The row I'd stare at longest is the failure count. Four thousand problems in and 372 families out looks like a hit rate of about one in eleven, but it isn't a reliable success rate. The README doesn't explain how problems map to families, how the significance filter works, or how many comparable problems failed. Some outputs build on earlier results, so the mapping is unlikely to be one to one. The published papers are survivors of a selection process whose rules outsiders can't see—and the unreleased model means they can't reproduce how the results were found.

Lean: what it fixes, and what it moves

Lean is the part of this pipeline where checking can be cheap. It mechanically checks a proof of the exact statement encoded in its formal environment. Whether that statement faithfully captures the claim in the paper is a separate question.

The repo uses Comparator for the last mile. Its documentation describes a “challenge” file containing the theorem statement. The checker confirms that the solution proves it, uses only permitted axioms, and replays through Lean's kernel. The formalization.yaml file lists Comparator configurations for the formalized main results.

But the check is meaningful only if the challenge statement is correct; Lean can't catch human error in how it's written or presented. The cost shifts from refereeing an eighty-page proof to deciding whether the challenge statement is the theorem the paper claims.

That's a good trade when the statement is short. In an unrelated paper using Comparator, the authors say a human can audit a thirty-line challenge file without reading the proof. It's a worse trade when the library lacks definitions a research-level result needs. Whoever formalizes the result has to introduce them, and each new definition is another place a subtle mistake can hide. My rule of thumb—not an official one—is to check the length of the challenge file and how many new definitions it introduces.

There's fresh evidence that this failure mode is real. On October 6, three mathematicians at King's College London and Cambridge posted “Navier–Stokes lost in translation”. They argue that the Lean version of OpenAI's announced Navier–Stokes proof doesn't correspond to the written one. TechCrunch reports at least two discrepancies.

That is a different artifact, not evidence about the 300 formalized results in this repository. And the paper's authors don't claim the written proof is wrong. Still, it shows what to look for: a green checkmark can sit on top of the wrong theorem.

One final caveat on the numbers: OpenAI counts 300 formalized top-line results, while lean/formalization.yaml describes its scope as “partial progress.” An outside count on October 6 found 162 papers in the file with a formalized main result (FourWeekMBA). I can't tell how that figure relates to OpenAI's 300.

The claims you'll read first

I'm working from titles here, and a title isn't evidence. The catalogue includes “An Upper Bound of 9/4 for the Matrix Multiplication Exponent”—while the best published bound I know of is around 2.37—and “Thompson's group F is nonamenable,” a question open for decades. Other titles claim the Euclidean plane is not five-colorable and give counterexamples to two Kaplansky conjectures. I haven't read these papers.

Then there's the quasi-Riemann hypothesis: roughly, that nontrivial zeta zeros stay a fixed distance from the line Re(s) = 1. That's much weaker than the Riemann hypothesis, which places them all at 1/2. To my knowledge, it was open.

The README mentions a zero-free region for Re(s) > 11/12 and says that write-up was edited by a human for readability. Yet the catalogue lists “The Quasi-Riemann Hypothesis: A Zero-Free Half-Plane Re(s)>7/8,” dated September 30. I can't tell how the two relate: it could be a stronger later result, a revision, or something else.

An outside reading of the contents file also identifies a family claiming that the irrationality exponent of π is exactly 2. The best published upper bound I know of is about 7.1, from Zeilberger and Zudilin (2020). Two is what almost every real number has—and what every irrational algebraic number has by Roth's theorem—but a title alone doesn't establish this claim.

The README names the zeta zero-free region and Hodge result as exceptions to the standard procedure. It says they “include” those examples, so there may be others, but doesn't explain what made them different. Those exceptions deserve scrutiny precisely because the standard pipeline doesn't explain them.

The cost of “new”

Correctness is only one cost. Someone also has to establish that each result is actually new.

In October 2025, OpenAI staff said GPT-5 had solved ten Erdős problems. The maintainer of the Erdős problems site pushed back: “open” on the list meant he personally didn't know of a paper, and the model had turned up existing literature.

Someone has to do that literature search for every one of these families.

What to watch

The first history.md entry is already the most informative thing in the repository. More than the counts or headlines, it shows how the process behaves under pressure.

I'd track four questions:

• What gets corrected next? Was the error in a proof, the Lean statement, or the claim to be new?

• Do errors cluster? The changelog doesn't say whether the three withdrawn papers had Lean formalizations.

• Does the model get named and released? Without that, outsiders can't test how the results were found.

• Does the collection move? The README says community hosting is being explored.

Harvard's Melanie Wood, who sits on the advisory group, told TechCrunch after the release that nobody understands these results at the moment they come out—and that now the work begins.

Seven hundred and nineteen papers is an announcement. How the next corrections get handled will be the science.

Method note, as of October 9, 2026. I read the README and history.md, the visible portion of lean/formalization.yaml, the September 29 guidelines, TechCrunch's October 8 report, and the abstract of the Navier–Stokes formalization paper. I did not read the manuscripts, overview.pdf, reasoning summaries, or Lean code. I couldn't render CONTENTS.md, so descriptions of individual families come from outside reporting. I know OpenAI's announcement through press quotations. “Not checked” items depend on material I haven't reviewed. Check the live README before publication; the repository is changing quickly.