AI Hallucination Cases

"This case adds yet another unfortunate chapter to the story of artificial intelligence misuse in the legal profession. Here, Defendants' counsel not only included an AI-hallucinated citation and quotations in the summary judgment brief that led to the filing of this motion for sanctions, but also included multiple new AIhallucinated citations and quotations in Defendants' brief opposing this motion. In other words, counsel relied upon unvetted AI—in his telling, via inadequately supervised colleagues—to defend his use of unvetted AI."

"Were Advani a lawyer, the Court would consider imposing sanctions on her. But in view of the fact that she is not a lawyer and of the dismissal of this case, the Court declines to pursue the matter further and merely warns Advani that presentation of false citations, quotations, and holdings in the future may indeed result in the imposition of sanctions."

The district court independently found numerous inaccurate quotations and citations in Plaintiff's Opposition—misquoted language attributed to binding Third Circuit authority, Westlaw citations and dates that were incorrect or non-existent, and miscited pincites. The court concluded the pattern suggested the brief may have been prepared using generative AI without adequate verification and ordered counsel to show cause under Rule 11 and ethical rules. The court preserved the inaccuracies in the official opinion but removed links to invalid citations.

AI Use

Alim Al-Hamim, appearing pro se (self-represented), used a generative AI tool to prepare his opening brief appealing the dismissal of his claims against his landlords. He had also submitted a document with fabricated citations in the lower court.

Hallucination Details

The appellate brief contained eight fictitious case citations alongside legitimate ones. The court could not locate the cases and issued an order to show cause.

Ruling/Sanction

Al-Hamim admitted relying on AI, confirmed the citations were hallucinations, stated he failed to inspect the brief, apologized, and accepted responsibility. The court affirmed the dismissal of his claims on the merits. While finding his submission violated Colorado Appellate Rules (C.A.R. 28(a)(7)(B)), the court exercised its discretion and declined to impose sanctions.

Key Judicial Reasoning

Factors against sanctions included Al-Hamim's pro se status, his contrition, lack of prior appellate violations, the absence of published Colorado precedent on sanctions for this issue, and the fact that opposing counsel did not raise the issue or request sanctions. However, the court issued a clear and strong warning to "the bar, and self-represented litigants" that future filings containing AI-generated hallucinations "may result in sanctions". The court emphasized the need for diligence, regardless of representation status.

Amazon alleged that Allen's undisclosed use of AI produced non-existent case citations and hallucinated quotations; the court declined to sanction, warned Allen and required future compliance with local AI-disclosure rule.

In an earlier decision, the court had already warned the plaintiff against "any further legal misrepresentations in future communications".

Defendant Holly Nelson, appearing pro se, filed a motion to dismiss that included a fabricated case citation, Finch v. Smith, which does not exist. The court inferred that Nelson used AI to draft the motion without verifying the accuracy of the citations. The court issued an order to show cause, requiring Nelson to justify why her filing does not violate Rule 11, or alternatively, to withdraw her motion. If she fails to do so, the court intends to strike her motion entirely.

The underlying motion was later withdrawn, with the result that the tentative ruling was not adopted.

AI Use

The plaintiff, proceeding pro se, submitted filings citing multiple nonexistent cases. The court noted patterns typical of ChatGPT hallucinations, referencing studies and prior cases involving AI errors, though the plaintiff did not admit using AI.

Hallucination Details

Several fake citations identified, including invented federal cases and misquoted Supreme Court opinions. Defendants flagged these to the court, and the court independently confirmed they were fictitious.

Ruling/Sanction

No sanctions imposed at this stage, citing special solicitude for pro se litigants. However, the court issued a formal warning: further false citations would lead to sanctions without additional leniency.

Key Judicial Reasoning

The court emphasized that even pro se parties must comply with procedural and substantive law, including truthfulness in court filings. Cited Mata v. Avianca and Park v. Kim as established examples where AI-generated hallucinations resulted in sanctions for attorneys, underscoring the seriousness of the misconduct.

"In her declaration, Attorney Cervantes described the erroneous citation as her own inadvertent error and further implied that Westlaw’s tool may have “glitched” by erroneously producing the citation. [Dkt. 31 at ¶ 4]. Specifically, she indicates that the copy citation tool mayhave inadvertently copied the wrong citation information.

Yet, that explanation seems highly improbable. The citation was incorrect in every respect and none of the sub-parts of the citation match each other, including the case docket number, reporter, pincite, court information, and date information. The likelihood of every component of a citation being simultaneously wrong due to a software malfunction in transcribing from a “correct” citation is, in the Court’s view, statistically improbable (particularly given Westlaw’s representations as to the accuracy of its tools and the apparent resources Westlaw has devoted to promoting the reputation of this tool). If one set of numbers had been transposed, or if the date were wrong, such transcription errors might be explicable. However, the fact that the party names, docket number, Westlaw citation, date, and court all failed to match, resulting in a mashup citation, cannot credibly be attributed to a software error, let alone a mistranscription or copying mistake. Rather,this Frankensteinian legal citation, stitched together from mismatched party names, docket numbers, dates, and courts, bears the hallmarks of an AI-generated hallucination, as documented in numerous published opinions and reports."

The Board found over 70% of citations in Huffman's reply brief inaccurate, including fabricated cases, misattributed reporter citations, cases that did not support cited propositions, and incorrect or non-existent transcript/Rule 4 citations. Counsel admitted using AI to generate portions of the brief. The Board treated the motion as one for Rule 11-type sanctions and struck the reply brief; it emphasized attorneys' duty to verify AI-generated content.

AI Use

Counsel admitted using a “new legal research medium”, appears to be a generative AI system or platform capable of generating fictitious case law. Counsel did not deny using AI, but claimed the system may have been corrupted or unreliable. The amended filing removed the false authorities.

Hallucination Details

The court did not identify the specific fake cases but confirmed that “citations to non-existent cases” were included in Defendants’ original brief. Counsel’s subsequent filing corrected the record but did not explain how the citations passed into the brief in the first place.

Ruling/Sanction

Judge William Griesbach denied the motion for summary judgment on the merits, but addressed the citation misconduct separately. He cited Rule 11 and Park v. Kim (91 F.4th 610, 615 (2d Cir. 2024)) to underline the duty to verify. No formal sanctions were imposed, but counsel was explicitly warned that further use of non-existent authorities would not be tolerated.

Key Judicial Reasoning

The court emphasized that even if the submission of false citations was not malicious, it was still a serious breach of Rule 11 obligations. Legal contentions must be “warranted by existing law,” and attorneys are expected to read and confirm cited cases. The failure to do so, even if caused by AI use, is unacceptable. The court accepted counsel’s corrective effort but insisted that future violations would be sanctionable.

The court found multiple exhibits (videos, photographs, messaging screenshots, and metadata) to be fabricated or materially altered using generative AI. The court deemed Plaintiffs' explanations not credible, declined criminal referral, declined monetary sanctions, and imposed a terminating sanction under Cal. Civ. Proc. Code § 128.7(b).

Order to Show Cause is here.