AI Hallucination Cases

(Not an hallucination per se, but worth adding to the database anyway.)

As reported here.

The Claimant used AI to generate case citations in his pleadings; Tribunal found 46 inaccurate or misleading citations (9 wholly fictitious, 37 misrepresentations of real cases) and concluded the conduct was reckless and unreasonable, justifying a costs order.

Folllowing a show cause order, Counsel admitted to having used the tool together with Google Search, and explained that, although he was aware of the issues with AI models like ChatGPT, he said he did not expect this tool to fall into the same issues.

The Court found Attorney James A. Johnson used Ghostwriter Legal to draft a motion that contained multiple fabricated case citations, misstated/false quotations attributed to authorities, and cited precedent that had been reversed by the Supreme Court. The Court found the conduct tantamount to bad faith and imposed sanctions under its inherent authority. Sanctions include an order to file, not under seal, this order "in any case in any court wherein he appears as counsel fortwelve (12) months after the date of this order."

The court noted prior fabricated citations in plaintiff's earlier briefing (for which counsel had already been sanctioned). In the current filing the court found no fabricated citations but identified inaccurate legal contentions—e.g., a rule statement claiming withholding ready-to-produce material while seeking extra time is sanctionable under Fed. R. Civ. P. 37(b)—which the court found unsupported and incorrect. The court suspected plaintiff used AI again, but simply removed the citations. The court admonished counsel to review AI-generated work and comply with Rule 11 but did not impose additional sanctions here.

The court admonished the pro se plaintiff, expressing concern he may be relying on artificial intelligence to draft filings and cite cases without confirming accuracy, and directed him to review Fed. R. Civ. P. 11; no specific fabricated citations or false quotations were identified in the opinion.

(Monetary sanction decided in later determination, available here.)

Claimant cited several inapplicable regulations to support reimbursement. When directed to supply the texts, claimant admitted he had used artificial intelligence to create his submission and withdrew reliance on the cited regulations except for JTR 053710. The Board denied the claim.

Earlier tentative order is here.

Original Show Cause Order is here.

The grounds of appeal contained at least two non-existent authorities. The judge concluded the false citations likely arose from unchecked generative-AI drafting and directed the solicitor to show cause why conduct should not be referred to the SRA.

Assessing Officer relied on three judicial decisions that the court found to be non-existent. The court observed such citations appear to have been fetched (implicitly) from AI and admonished that quasi-judicial officers must verify AI-generated results before relying on them; held Assessment Order violated principles of natural justice and remanded matter.

Show Cause Order is available here.

Court denied OSC/motion on procedural safe-harbor grounds but found defendants submitted miscited, non-existent, and inapposite authorities (and noted risk of AI-generated fake citations). Defendants accepted responsibility but no sanctions imposed.

"When criminal defense attorneys fail to comply with their ethical obligations, their conduct undermines the integrity of the judicial system. It also damages their credibility and potentially impugns the validity of the arguments they make on behalf of their clients, calling into question their competency and ability to ensure defendants are provided a meaningful opportunity to be heard. Thus, criminal defense attorneys must make every effort to confirm that the legal citations they supply exist and accurately reflect the law for which they are cited. That did not happen here."

Counsel admitted he did not verify citations. The court imposed a $1,000 sanction payable personally by counsel, ordered 1 hour CLE on generative AI, and referred him to the Disciplinary Committee.

The appellant cited two reported decisions which the Board could not locate and concluded likely do not exist and may have been generated with AI; the Board excluded those authorities from evidence and cautioned the appellant about nondisclosure of AI per the Board's Code of Conduct.

Monsieur Laprade filed a contestation containing multiple citations to non-existent authorities generated with the assistance of artificial intelligence. The Court found these to be fabricated (so-called "hallucinated") citations, constituting a manquement important to the conduct of the proceeding under art. 342 C.p.c., and imposed a 5,000$ sanction.

"This case adds yet another unfortunate chapter to the story of artificial intelligence misuse in the legal profession. Here, Defendants' counsel not only included an AI-hallucinated citation and quotations in the summary judgment brief that led to the filing of this motion for sanctions, but also included multiple new AIhallucinated citations and quotations in Defendants' brief opposing this motion. In other words, counsel relied upon unvetted AI—in his telling, via inadequately supervised colleagues—to defend his use of unvetted AI."

Story available here.

The Adjudication Officer found the complainant's submissions contained multiple inaccurate and non‑existent legal citations. The Respondent had flagged AI‑generated drafting and numerous phantom or misquoted determinations; the Officer concluded the complainant failed to establish a prima facie case and that the submissions contained egregious and misleading citations.

The appellate opinion and editor's note identify numerous incorrect or non-existent case citations in the appellant's filings. The court treated those citations as unreliable, found several to be fictitious or unlocatable, and declined to credit them in resolving the appeals.

Case dismissed for "good cause" in light of motions to dismiss expert. Expert had confessed using ChatGPT.

The Court observed that several authorities cited in the Kogers' pro se filings do not exist and appeared to be fabricated (noting possible use of AI), warned of Rule 9011 implications, and treated the filings as part of an abusive litigation strategy warranting dismissal and a one-year filing bar.

Plaintiff's counsel filed a Notice of Errata admitting that erroneous citation and quotation errors in multiple briefs resulted from the attorney's use of generative AI. The Court refused to consider the amended briefs/errata for purposes of its decision and treated any propositions supported solely by the incorrect or made-up citations as unsupported. The Court declined to impose sanctions but warned the attorney that relying on AI without proper oversight can be sanctionable under the New Jersey Rules of Professional Conduct.