Deloitte will refund Australian government for AI hallucination-filled report (arstechnica.com)

Deloitte Australia has agreed to offer the federal government a partial refund after its AU$440,000 Targeted Compliance Framework Assurance Review—an audit of the technical framework used to automate welfare penalties—was found to contain multiple AI-hallucinated quotes and citations to nonexistent research. The errors were flagged after publication when academics noticed fabricated references, including papers erroneously attributed to University of Sydney professor Lisa Burton Crawford. Deloitte and the Department of Employment and Workplace Relations quietly issued an updated 273‑page report “to address a small number of corrections,” and only on page 58 disclosed that a “generative AI large language model (Azure OpenAI GPT‑4o) based tool chain” was used in the technical workstream to help map code state to business requirements and compliance needs. The episode underscores practical risks of embedding LLMs in high‑stakes government analysis: hallucinations can introduce false provenance and mislead decision‑makers unless outputs are rigorously validated. Technically, Deloitte’s admission that GPT‑4o was used as part of a toolchain—rather than as a mere drafting aid—raises questions about data lineage, prompt engineering, and post‑processing safeguards when LLMs are used to interpret code against regulatory requirements. For the AI/ML community this is a cautionary case on the need for provenance, human-in-the-loop verification, contractually mandated disclosure of model use, and stronger audit practices when generative models feed into official reporting.