AI prepared the report, Deloitte wrote the refund cheque

Deloitte has agreed to return part of a $440,000 payment to the federal government after acknowledging that generative artificial intelligence was used to produce sections of a report later found to contain multiple factual and citation errors.

The report, commissioned by the Department of Employment and Workplace Relations (DEWR) in December 2024, was meant to review the government’s targeted compliance framework and its IT system for monitoring jobseekers’ mutual obligations.

When it was first released in July, the 100-page document pointed to “system defects” and a lack of legislative traceability, describing the welfare compliance IT setup as “driven by punitive assumptions of non-compliance.”

However, the report was pulled from the DEWR website and republished on Friday after media outlets, including The Australian Financial Review, revealed numerous incorrect or fabricated citations — including fake academic papers and a nonexistent court case.

Dr Christopher Rudge, a University of Sydney academic who first raised the alarm, said the report displayed classic signs of AI “hallucination.”

“Instead of fixing one fake reference, they’ve added several more,” Rudge said. “That shows the claims weren’t based on any single, verifiable source.”

In the revised version, Deloitte added a disclosure acknowledging the use of generative AI in the report’s preparation — specifically citing Microsoft’s Azure OpenAI GPT-4o tool, hosted within DEWR’s secure system.

The department confirmed Deloitte would forgo its final payment under the contract. “Some footnotes and references were incorrect,” a DEWR spokesperson said, “but the review’s findings and recommendations remain unchanged.”

Deloitte, for its part, said the issue had been “resolved directly with the client” and insisted that the “substantive content” of the report was unaffected.

Still, the episode has reignited concerns about the growing use of AI in government-commissioned consulting work.

Labor senator Deborah O’Neill, who has been vocal on the integrity of large consulting firms, accused Deloitte of relying too heavily on algorithms instead of experts.

“Deloitte has a human intelligence problem,” she said. “A partial refund feels like apartial apology. Maybe it’s time clients just pay for a ChatGPT subscription instead.”

The AFR’s investigation found that among the incorrect citations were fabricated references to University of Sydney and Lund University professors, as well as a nonexistent “Amato v Commonwealth” court ruling in a robodebt case, which Deloitte later corrected in the amended report.

While critics say the episode exposes the risks of unverified AI use, Rudge noted that the overall conclusions of the report remain consistent with other evidence of systemic flaws in welfare compliance systems.