The AI tools writing your test suites did not learn software engineering. They learned the internet, a particular and ageing snapshot of it, filtered, compressed, and now partly dissolved. That distinction carries consequences, because the web they trained on is vanishing faster than anyone anticipated, and the test suites they produce carry the epistemological weight of a library whose shelves are quietly catching fire. According to Pew Research Center’s 2024 analysis, 25% of all webpages that existed at any point between 2013 and 2023 are now inaccessible. A quarter of the general web from that decade is gone. Whether the code-specific slice of it decayed at the same rate is not something anyone has measured, and there are reasons to think it did not: Stack Overflow does not delete answers, and popular package documentation is mirrored heavily. But the tutorials, the blog posts, the migration guides and the annotated stack traces that surround that durable core are ordinary webpages, subject to ordinary decay, and they are a large part of what these models read. Engineers who trust AI-generated tests are trusting a model that learned, in substantial part, from things that are no longer there to verify. That sounds like it should be the whole story, and for a while I thought it was. It is not. The most direct test of that mechanism has been run, and it came back negative. What follows is partly an argument and partly a retraction of one.
A Library Eating Its Own Citations
Link rot is not an abstraction. Pew Research Center’s 2024 analysis found that 54% of Wikipedia pages contain at least one broken reference link, and 11% of all Wikipedia reference links now point to nothing. The encyclopaedia that code-generating models treat as a quality signal for factual grounding has already surrendered a tenth of its cited evidence base to entropy. The situation deteriorates further in journalism: 23% of news webpages carry at least one broken link; 21% of government webpages do the same. Eighteen per cent of tweets vanish within three months of posting. This creates a temporal trap for any model trained on web crawls. Yu, Liu and Xiong, introducing Craw4LLM (arXiv:2502.13347, 2025), note that existing pipelines discard over 90% of raw crawled data as low quality, citing DataComp-LM, FineWeb and TXT360. Their own contribution is a crawler that scores pages by pretraining influence rather than graph connectivity; it measures crawling efficiency, and it makes no claim about the age profile of what survives filtering. That claim would be the interesting one, and as far as I can find, no published study has measured it. Whether surviving pretraining data skews older is an open question, and I am naming it as a gap rather than filling it. What is not in question is the consequence of a fixed training cutoff, whatever the age profile of the data behind it. A model cannot distinguish a package that existed in 2021 from one that exists today, cannot flag an API whose endpoint has moved, and cannot know that a security advisory has superseded the example it memorised. It produces confident output from partial evidence, because its training rewarded confidence and it has no mechanism for checking whether the evidence is still there.
Phantom Packages, Baked-In Biases
Package hallucination is the most measurable symptom. A 2024 arXiv study (arXiv:2406.10279) spanning 576,000 code samples generated by 16 LLMs found that 19.7% of all software package names recommended were hallucinated, non-existent names that no engineer could install. Open-source models fared worse, reaching 21.7%. Even GPT-4 Turbo, the strongest closed-source model tested in that study, hallucinated 3.59% of packages in Python, the lowest rate of any model in the study. Those figures describe models tested in January 2024 and should be read with that date attached. A 2026 replication of the same methodology (Churilov, arXiv:2605.17062) ran five frontier models released between October 2025 and March 2026 across 199,845 paired Python and JavaScript prompts and measured rates between 4.62% and 6.10%, compressing the original 5.2-to-21.7 spread by roughly an order of magnitude. The floor has not moved: no 2026 model beat GPT-4 Turbo’s 3.59% from two years earlier. The ceiling has fallen sharply. Any argument that leans on a 21.7% open-source rate is arguing about a cohort that has been superseded. What survives the replication is narrower and more useful: 127 package names that all five frontier models invent identically, which is a registrable attack surface of 127 rather than 205,474, and a far more tractable thing to put on an allowlist. Those numbers are not confined to a controlled benchmark; they describe the routine experience of developers who ask a code-generating assistant to scaffold a test and receive imports referencing libraries that were deprecated, renamed, or never real.
A separate experiment in the same paper looked at whether these errors repeat. The authors took 500 prompts that had already produced a hallucination and re-ran each ten times across four of the sixteen models. The result splits sharply in two directions: 43% of hallucinated names came back identically on all ten runs, and 39% never came back at all. The persistent 43% is the half that matters for security, because a name that recurs is a name an attacker can enumerate and register. The 39% is the half that matters for interpretation, because it means roughly two in five of these errors really are one-off noise.
The errors are not random statistical noise. They are repeatable, and they are specific to individual models. What they are not, on the evidence, is an artefact of the vanishing web. Spracklen and colleagues tested that hypothesis directly: they compiled 12,871 packages that existed on PyPI between 2020 and 2022 and have since been removed, then checked them against every hallucination their study produced. Only 133 appeared, 0.17% of the corpus. The authors expected a sizeable share and reported that the finding contradicted their hypothesis. The same study found that 81% of unique hallucinated names came from exactly one of the sixteen models tested, including across models within the same family. A shared pool of dead links would predict convergent errors. The data shows the opposite. Whatever generates these names lives in the individual model’s generative process, and that is a harder problem than a stale corpus, not an easier one.
A developer who runs the generated test, watches it fail at the import line, and manually resolves the dependency has paid a tax that appears nowhere on any productivity dashboard. Multiply that correction across every team, every sprint, every CI pipeline running test generation at scale, and the phrase “hallucination debt” stops sounding metaphorical. It becomes a line item nobody has yet budgeted for, accruing silently beneath a layer of green status badges.
What would settle this. If hallucinated package names correlated with packages removed from PyPI or npm after a model’s training cutoff, the link-rot mechanism would have direct support. That test has been run and came back negative. A sharper test is still open: hold model and prompt constant and compare hallucination rates on API surfaces with heavy documentation churn against surfaces with stable, widely mirrored documentation. Until someone runs it, treat the argument here as a hypothesis about a mechanism rather than a measured finding. My confidence that link rot materially drives package hallucination specifically is about 25%. My confidence that it drives stale API and version errors, which is a different failure mode with a different evidence base, is about 60%.
Speed Without Correctness
QA consequences are showing up in survey data, with the caveat that survey data is what we have. DeviQA, a QA outsourcing vendor, polled 300 QA engineers, SDETs and test leads through its own network in 2026; 52% reported that bug volume has increased since their development teams began using AI, and only 2% reported a decrease. The report is candid about its limits, and so should anyone citing it be: this is a non-probability sample of 300, it measures perceived rather than measured change in defect volume, and the organisation publishing it sells QA capacity. Those caveats do not make the number worthless. They make it a signal to corroborate rather than a measurement to quote. Developers simultaneously report higher output, with teams using AI merging 98% more pull requests than teams that do not, and individual developers completing 21% more tasks, per LinearB’s 2026 benchmarks report covering 8.1 million pull requests across 4,800 organisations. The productivity gain and the quality degradation are two faces of the same acceleration. AI is generating more code than human reviewers can adequately interrogate, and the tests it appends are not catching what the code is introducing. LinearB’s data adds a further layer: AI-authored pull requests are accepted at a rate of 32.7%, against 84.4% for human-authored PRs, and they wait 4.6 times longer for review pickup. Reviewers, whether consciously or not, are applying extra scepticism to AI output. The irony is that AI-generated tests rarely attract equivalent scrutiny. They run, they return a green result, and passing tests are culturally read as evidence of correctness regardless of what they actually assert. A test that verifies the output of a hallucinated dependency, or mocks an API endpoint whose contract shifted after the model’s training cutoff, provides the appearance of a safety net while offering none of its substance. It checks that the code does what the model believed it should do, which is an entirely different question from whether the code does what the system requires.
Security as the Bill Coming Due
Veracode’s GenAI Security Report 2025, spanning more than 100 LLMs across 80 coding tasks, found that 45% of AI-generated code samples contained at least one security vulnerability, and the rate held flat across successive model generations. Scaling the model did not reduce vulnerability density. That flatness is worth sitting with, because it does not match what happened to package hallucination over the same period. There, capability gains did move the number: the worst-case rate fell by roughly seventy per cent between the 2024 and 2026 cohorts, from 21.7% to 6.10%. Vulnerability density did not move at all. Two failure modes, one responding to model scale and one not, is a finding about the failure modes rather than about the models. Insecure patterns are well-formed, plausible, and abundant in training data in a way that non-existent package names are not, and a larger model reproduces them more fluently rather than less. That is a hypothesis, not a measurement, and I am flagging it as one. GitClear’s analysis of 623 million changed lines between 2023 and 2026, cited by DevIQA, found that copy-pasted code blocks rose 81% over that period, while refactored code fell from 21% to 3.8% of all changed lines. That is the structural signature of AI-assisted development at scale: suggestions arrive in bulk, get accepted as written, and accumulate without being interrogated. Tests written to cover those blocks without challenging their provenance are not safety nets. They are theatre.
The debt is real and the mechanism is not the one the title implies. Teams will not discover the gap through a failing build, because a hallucinated dependency that has been squatted resolves cleanly, installs cleanly, and passes the suite written to cover it. They will discover it through a breach, or through a quarterly defect trend they cannot explain because nobody recorded which code was AI-authored. As the chart below shows, web content decay across Wikipedia, news, and general domains mirrors the package hallucination rates of open-source models, highlighting a quality gap that QA teams are beginning to measure. The four controls that follow are cheap. The third one, the authorship field on the pull request, is the only way anyone, including me, will find out whether this article was pointing at the right thing.
What to Do on Monday
Four controls, each with a way to tell whether it worked.
Gate on resolution, before you gate on tests. Run dependency resolution as a discrete CI step ahead of the suite, so an invented import fails at the pipeline rather than on someone’s laptop at 16:40. Measure the count of build failures attributed to unresolvable packages per week. Expect it to spike, then decline. If it never spikes, the gate is not wired in.
Pin, hash, and scan the lockfile rather than the manifest. A hallucinated name that has been squatted resolves cleanly and installs cleanly. Hash pinning is what distinguishes the package you reviewed from the package you received. Measure the percentage of repositories with hash-pinned lockfiles and SCA running against them.
Mark AI-authored change at the pull request. You cannot measure the defect rate of AI-generated tests without a field that says which tests are AI-generated. This is a policy choice, not an engineering problem. Measure the percentage of PRs carrying authorship metadata, then segment escaped-defect rate by that field after one quarter. If the two segments look the same, this article was wrong and you will have the data to say so.
Block the known 127. The frontier-cohort replication published the set of package names that all five tested models invent identically: 109 on PyPI, 18 on npm. That is a finite, enumerable list, which makes it the rare supply-chain control you can implement in an afternoon. Deny them at the proxy or the package manager configuration, and refresh the list when the next replication lands. Measure whether any of the 127 was ever requested in your estate. If one was, you have just found a developer who accepted a hallucinated import, and that is worth a conversation rather than a ticket.
References
- Pew Research Center (2024). When Online Content Disappears. https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/
- Yu, S., Liu, Z., and Xiong, C. (2025). Craw4LLM: Efficient Web Crawling for LLM Pretraining. arXiv:2502.13347. https://arxiv.org/abs/2502.13347
- Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B., and Jadliwala, M. (2025). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. USENIX Security Symposium 2025. arXiv:2406.10279. https://arxiv.org/abs/2406.10279
- Churilov, A. (2026). The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort. arXiv:2605.17062. https://arxiv.org/abs/2605.17062
- DevIQA (2026). State of AI-Generated Code 2026: The QA and Testing Gap (citing LinearB 2026, Veracode GenAI Security Report 2025, GitClear 2026). https://www.deviqa.com/blog/state-of-ai-generated-code-2026-the-qa-and-testing-gap/
Discussion