The Risk of AI: Fabricated Sources
Incidents at Ars Technica and The Economic Times highlight the growing risk for publishers experimenting with AI-assisted reporting.
Condé Nast-owned Ars Technica recently retracted an article after editors discovered that several quotations attributed to a named source had been generated using a Claude Code-based AI tool rather than taken from an interview.
Ars quickly removed the feature with its Editor-in-Chief, Ken Fisher, apologising to both its readers and to Scott Shambaugh, who had been falsely quoted.
“This is a serious failure of our standards. Direct quotations must always reflect what a source actually said. That this happened at Ars is especially distressing.
“We have covered the risks of overreliance on AI tools for years, and our written policy reflects those concerns. In this case, fabricated quotations were published in a manner inconsistent with that policy.” Ken Fisher, Editor-in-Chief, Ars Technica
The incident highlights a growing challenge for news organisations experimenting with AI.
While publishers increasingly use AI tools to support editing, summarising and early drafting, Large Language Models (LLMs) cannot provide the raw material journalism depends on: verifiable sources and quotations.
When the Source Never Exists
The problem arises when AI-generated material goes beyond assistance and begins to shape the reporting itself. The Ars Technica case demonstrates how easily AI can blur this boundary.
Indeed, LLMs can hallucinate authoritative text even when the underlying statements have no factual basis. For editors reviewing a draft, AI-generated language can appear indistinguishable from legitimate reporting despite the absence of any interview or verifiable source.
These problems are not new. In early 2023, technology publication CNET corrected several AI-generated finance explainers after readers identified factual errors. Later that year, Sports Illustrated removed product review articles that appeared under author profiles which could not be linked to identifiable journalists.
In each case the problem was down to the editorial systems failing to detect errors prior to publication.
When the Mistake Spreads
A second incident shows how quickly these errors can extend far beyond the original article itself.
An AI-assisted story in The Economic Times about a fake viral video depicting Donald Trump and Barack Obama being arrested included a quotation attributed to disinformation researcher Nina Jankowicz. The quote was fabricated.
Once published, the article was indexed by search engines, copied by aggregation services and stored in systems used by AI tools to retrieve information. Even worse, versions of the fabricated quotation later began appearing in large language model responses. What began as an error in a single article effectively became a reference point across the internet.
The dynamic reflects how AI interacts with the open web and how easy it becomes for LLMs to hallucinate. Language models rely heavily on publicly available text when producing answers, which means inaccurate or synthetic information published online can become part of the material used to generate future outputs. Error is heaped upon error.
Why Hallucinations Persist
The reliability problem reflects how AI itself works.
LLMs generate text by predicting word sequences based on patterns in training data. Their objective is to produce language that sounds coherent and plausible rather than language that has been independently verified.
Because these systems optimise for plausibility rather than evidence, they can produce statements that read like reporting even when no source exists. If a prompt implies that an expert might comment on a topic, the model may generate a quotation that sounds credible even when the individual has never made that statement.
Some research suggests the problem may become more serious as models grow more capable. Tests of newer reasoning models have found higher hallucination rates than earlier systems, with some models producing incorrect information in a significant share of responses.
Jerry Tworek, one of the researchers behind OpenAI’s reasoning models, argues that the limitation runs deeper than isolated hallucinations. In a recent interview on the Unsupervised Learning podcast, he described the inability of current systems to learn effectively from their own mistakes.
“The biggest limitation of the models today is that if they fail, you get kind of hopeless pretty quickly. There isn’t a very good mechanism for a model to update its beliefs and its internal knowledge based on failure.” Jerry Tworek, VP of Research, OpenAI
In other words, the systems can produce answers but struggle to adapt when those answers are wrong. “Unless we get models that can work themselves through difficulties and get unstuck on solving a problem, I don’t think I would call it AGI,” Tworek said.
Human reasoning works differently. When people encounter an error, they test assumptions, revise their understanding and probe the problem until a solution emerges. As Tworek says, “Intelligence works at the problem and probes it until it solves it.”
Drawing the Editorial Boundary
For news organisations integrating AI into editorial workflows, the distinction is becoming clearer.
The real question is not whether newsrooms should use AI, but where it belongs and where it does not.
In practice, AI works best when applied to low-risk tasks such as background research, restructuring copy, generating headline variants, building SEO metadata or reformatting material that has already been verified.
The risks increase sharply when the technology is inserted into live reporting, political coverage or any workflow where editorial judgement matters more than speed.
The recent incidents at Ars Technica and The Economic Times demonstrate the difference. And as these errors show, once fabricated information enters the news ecosystem it can circulate through search engines, aggregators and AI systems long after the original mistake has been corrected.