AI Hallucination Cases

"When criminal defense attorneys fail to comply with their ethical obligations, their conduct undermines the integrity of the judicial system. It also damages their credibility and potentially impugns the validity of the arguments they make on behalf of their clients, calling into question their competency and ability to ensure defendants are provided a meaningful opportunity to be heard. Thus, criminal defense attorneys must make every effort to confirm that the legal citations they supply exist and accurately reflect the law for which they are cited. That did not happen here."

The appellate opinion and editor's note identify numerous incorrect or non-existent case citations in the appellant's filings. The court treated those citations as unreliable, found several to be fictitious or unlocatable, and declined to credit them in resolving the appeals.

The appellant (self-represented) cited two nonexistent cases in her appellate brief. The respondent flagged the fictitious citations and requested sanctions. The court found the citations fictitious, discussed sanction authority and AI-generated filings, but declined to impose sanctions because the request was procedurally inappropriate, the appellant corrected filings promptly, and the legal propositions were, in fact, supported by existing authority.

Order to show cause is here here.

"In total, appellant's opening brief contains 23 case quotations, 21 of which are fabrications. Appellant's reply brief contains many more fabricated quotations. And, both briefs are peppered with inaccurate citations that do not support the propositions for which they are cited.

[...]

We conclude by noting that "hallucination" is a particularly apt word to describe the darker consequences of AI. AI hallucinates facts and law to an attorney, who takes them as real and repeats them to a court. This court detected (and rejected) these particular hallucinations. But there are many instances-hopefully not in a judicial setting-where hallucinations are circulated, believed, and become "fact" and "law" in some minds. We all must guard against those instances."

The court found multiple exhibits (videos, photographs, messaging screenshots, and metadata) to be fabricated or materially altered using generative AI. The court deemed Plaintiffs' explanations not credible, declined criminal referral, declined monetary sanctions, and imposed a terminating sanction under Cal. Civ. Proc. Code § 128.7(b).

"In her declaration, Attorney Cervantes described the erroneous citation as her own inadvertent error and further implied that Westlaw’s tool may have “glitched” by erroneously producing the citation. [Dkt. 31 at ¶ 4]. Specifically, she indicates that the copy citation tool mayhave inadvertently copied the wrong citation information.

Yet, that explanation seems highly improbable. The citation was incorrect in every respect and none of the sub-parts of the citation match each other, including the case docket number, reporter, pincite, court information, and date information. The likelihood of every component of a citation being simultaneously wrong due to a software malfunction in transcribing from a “correct” citation is, in the Court’s view, statistically improbable (particularly given Westlaw’s representations as to the accuracy of its tools and the apparent resources Westlaw has devoted to promoting the reputation of this tool). If one set of numbers had been transposed, or if the date were wrong, such transcription errors might be explicable. However, the fact that the party names, docket number, Westlaw citation, date, and court all failed to match, resulting in a mashup citation, cannot credibly be attributed to a software error, let alone a mistranscription or copying mistake. Rather,this Frankensteinian legal citation, stitched together from mismatched party names, docket numbers, dates, and courts, bears the hallmarks of an AI-generated hallucination, as documented in numerous published opinions and reports."

"Chandra’s self-represented status does not permit him to submit information blindly. Likeevery other person who appears before the court, he has an obligation to confirm that arguments andcase law submitted to the court are supported by existing law, and a failure to do so is sanctionable."

Penalty decided in an order from the next day.

Counsel's explanation of what happened can be found here.

Key Judicial Reasoning

Magistrate Judge Sheri Pym found the motion legally deficient on multiple grounds. In addition, she emphasized that counsel must not rely on fake or unverified authority. She cited Mata, Park, Gauthier, and others as cautionary examples of courts imposing sanctions for AI-generated hallucinations. The court reaffirmed that the use of AI does not lessen the duty to verify the existence and relevance of cited law.

AI Use

Plaintiff’s counsel admitted using generative AI to draft a motion to remand without independently verifying the legal citations or the factual accuracy of quoted complaint allegations.

Hallucination Details

Cited a fabricated case (details of the specific case name not listed in the ruling). Included fabricated quotations from the complaint, suggesting nonexistent factual allegations.

Ruling/Sanction

The Court imposed a $2,500 sanction payable by December 30, 2024. Counsel was also required to notify the California State Bar of the sanction and file proof of notification and payment. The Court recognized mitigating factors (health issues, post-hoc corrective measures) but stressed the seriousness of the violations.

Key Judicial Reasoning

Rule 11 requires attorneys to conduct a reasonable inquiry into both facts and law. Use of AI does not diminish this duty. Subjective good faith is irrelevant: violations occur even without intent to deceive. AI-generated filings must be reviewed with the same rigor as traditional submissions.

While the Court initially organised show cause proceedings leading to potential sanctions, the case was eventually settled. Nevertheless, the Court stated that it "intends to report Ms. Rasmussen’s use of mis-cited and nonexistent cases in the demurrer to the State Bar", unless she objected to "this tentative ruling".