We Checked All 209,985 Citations in ACL 2026

The companion essay on this site argues that the citation ledger is fine while the citation currency is dying. "The ledger is fine" was doing a lot of unexamined work in that sentence. This note is the receipt: we audited every reference in the ACL 2026 proceedings, all 4,459 papers, all 209,985 references, for whether the cited work exists and, where a citation backs a specific claim, whether the cited paper actually supports it. The full report is at tuto.fim.ai, the pipeline is open source (github.com/tao-hpu/tuto, Apache-2.0), and the citation-level judgments ship as an anonymized CC BY dataset in the same repository. Disclosure up front: Tuto is a product of fim.ai, my company; this note is the summary I would want as a reader, not neutral coverage.

Three findings, in the order they surprised me. (Updated September 2026: the second finding originally reported a single claim-support rate; two re-runs did not reproduce it, and the note now reports all three.)

Fabricated references are a rounding error. The funnel went from 209,985 references to 28,569 not found in any local snapshot, to 18,750 survivors of fuzzy rescue, to 12 suspicious leads after LLM triage, to exactly 2 confirmed nonexistent references after human verification. That is 0.001%. Both confirmed cases had the same unsettling shape: real authors, real topic, plausible venue, no such paper. But the scare narrative about LLM-hallucinated bibliographies flooding top venues does not survive contact with the full corpus. At the existence level, the ledger holds.

The claim-support rate did not replicate. These are real papers cited for claims they never make: the citation resolves, the DOI works, and the sentence it props up is not in the cited work. No existence check can catch them, because nothing is missing from the ledger. In the audited run over 100 papers, 16% of papers carried at least one confirmed case, and 0.95% of claim citations (20 of 2,110) were confirmed defects, where "confirmed" means the flag survived an adversarial second review instructed to refute it. Then we ran the same code, at the same commit, on two further independent draws of 100 papers each. They returned 5.66% and 6.12%: consistent with each other, pooling to 5.90% [5.12, 6.80] over 3,033 claim citations, and 7.9 standard errors away from the audited run. A single run of this pipeline does not tell you the rate.

The instrument is the finding. The gap sits in the support-judgment layer. The first-pass judge's model was read from an environment variable that was never logged, so the run that produced 0.95% cannot be attributed to a known model. The first-pass precision from that run (about 15%, 20 of 130 flags) does not hold across re-runs either: in the later draws the arbiter confirmed 61 to 67% of flags. The paper therefore reports all three runs and takes the non-replication, not any single rate, as its principal result. The practical rule: pin the judge model and log it with every judgment. We now do. Any citation-integrity tool that publishes a rate without showing it repeats should be read as reporting one draw, which is the "measure the claim, not the mechanism" lesson from my memory post-mortem, applied to my own detector.

One design constraint shaped everything downstream: the public report contains aggregate statistics and paraphrased, anonymized cases only. No paper is named, no author is identifiable, and the released dataset strips every verbatim string, because in a fully indexed literature any quoted sentence is a search key. Accountability infrastructure that starts by naming names would be a worse version of the false-positive problem with a person attached.

The report is the canonical version and carries all three runs. The existence census above is unaffected by the replication problem.