AI Hallucination Cases

"In total, appellant's opening brief contains 23 case quotations, 21 of which are fabrications. Appellant's reply brief contains many more fabricated quotations. And, both briefs are peppered with inaccurate citations that do not support the propositions for which they are cited.

[...]

We conclude by noting that "hallucination" is a particularly apt word to describe the darker consequences of AI. AI hallucinates facts and law to an attorney, who takes them as real and repeats them to a court. This court detected (and rejected) these particular hallucinations. But there are many instances-hopefully not in a judicial setting-where hallucinations are circulated, believed, and become "fact" and "law" in some minds. We all must guard against those instances."

The court found nearly every citation in counsel's motion to be incomplete, inaccurate, or fabricated, describing the references as typical of AI hallucinations and ordering counsel to show cause why sanctions should not be imposed.

The court granted Plaintiff's motion to strike portions of the Opening Brief after Defendant raised concerns that the brief included a non-existent quotation attributed to an existing case, a mischaracterization of an existing case, a citation to a non-existent case, and a miscitation of a case that did not address the asserted issue. The court noted counsel had been sanctioned in a separate case for citation-related deficiencies consistent with AI-generated hallucinations. The stricken portions were removed and the ALJ decision was affirmed.

"In her declaration, Attorney Cervantes described the erroneous citation as her own inadvertent error and further implied that Westlaw’s tool may have “glitched” by erroneously producing the citation. [Dkt. 31 at ¶ 4]. Specifically, she indicates that the copy citation tool mayhave inadvertently copied the wrong citation information.

Yet, that explanation seems highly improbable. The citation was incorrect in every respect and none of the sub-parts of the citation match each other, including the case docket number, reporter, pincite, court information, and date information. The likelihood of every component of a citation being simultaneously wrong due to a software malfunction in transcribing from a “correct” citation is, in the Court’s view, statistically improbable (particularly given Westlaw’s representations as to the accuracy of its tools and the apparent resources Westlaw has devoted to promoting the reputation of this tool). If one set of numbers had been transposed, or if the date were wrong, such transcription errors might be explicable. However, the fact that the party names, docket number, Westlaw citation, date, and court all failed to match, resulting in a mashup citation, cannot credibly be attributed to a software error, let alone a mistranscription or copying mistake. Rather,this Frankensteinian legal citation, stitched together from mismatched party names, docket numbers, dates, and courts, bears the hallmarks of an AI-generated hallucination, as documented in numerous published opinions and reports."

Docket available here.

Counsel's Amended Motion cited a non-existent case (In re Berry Good, LLC) generated by AI. Court found the citation fabricated, counsel admitted AI use and negligence, implemented safeguards, and court declined sanctions while transmitting matter to disciplinary office. The court's reflections are worth quoting in full.

"When examined, paragraph 15 is the result of a flawed AI search that suggested an incorrect generalization falsely supported by a specific but non-existent case. Under any scenario, a suspect argument buttressed with a case from whole cloth violates the expressed and tacit norms governing the practice of law as it intersects with the courts. Simply, you cannot make stuff up to convince a court to do something.

In simpler times, that would end the inquiry with draconian consequences. The times, however, are no longer simple; thus, the inquiry does not end here.

AI interposes additional considerations that defy easy categorization or scrutiny when examining the traditional norms of practice and advocacy. Threshold, two fields of inquiry present in any AI debate. The first is whether exposition artificially generated by a computer is acceptable in what has contextually been a forum for human critical analysis, thought, and advocacy. That is not the instance here. Rather, it is the second; that is, when AI sacrifices accuracy to satisfy the consumer and that inaccuracy is advocated before the court.

[...]

The only acceptable conclusion is that citing a made-up case is a false representation to the court that violates the expressed norms of practice, is actionable, and should bear consequences. The answer, however, to the second question —whether an artificial component alters the calculus—is more difficult.

Here, our traditional norms are challenged. In context, the traditional advocacy process begins with black letter law in the form of statutes, codes, and regulations. Then, there are judicial opinions and orders interpreting black letter law in the context of justiciable issues based on discrete facts. Lawyers cite cases for their binding or persuasive authority and have done so ever since someone started committing judicial decisions to stone, parchment, or paper. The ability to find and cite cases has evolved over time from laborious research in courthouses, to compendiums, indices, digests, reporters, advance sheets, key numbers, and now the internet.

The internet changed this dynamic in two ways. First, it streamlined, refined, and improved the research process. Users could select fields, topics, and key numbers to find and locate cases. Eventually, more specific tools, such as key words or phrases, enhanced the process. Second, and more pertinent to the present issue, AI is creating a process where instead of just finding authority, the computer also does the supplicant's thinking and analysis for them. The first innovation continues to evolve; the second is new, likewise continues to evolve, but constitutes an entirely new dimension, rather than a continuum, in legal practice. Finding has evolved into finding and thinking.

Travel back to research, the first changed dynamic. From stone to the internet, how lawyers found and used case law took many forms. Some would carefully read every case they found; some would read every page of select cases they thought pertinent; others might read the digest entry and then read only that section of the case; others might read only the digest entry; others barely read anything at all and cited anything that looked like it might support their position. Each was a personal decision by the lawyer for which he or she is accountable under the traditional norms of practice both officially—through codes of conduct and Rule 9011—and unofficially per the norms and expectations of courts.

AI compresses this historical dynamic. The research engine is now the search and thinking engine. AI finds case law and tells the subscriber that it supports their position. All the personalities described above—and known oh-so-well to us all—have become one.

Except, however, AI is flawed. In its infancy—albeit one that may be in college soon—it is immature and does not always provide accurate information or correct analysis; it may even make things up.

A tool that potentially supplants your advocacy by doing the finding and thinking for you is enticing. 3 But in the end, it is only a tool. Despite our near total and unfortunate faith in the internet, it does not relieve the attorney of his or her responsibility to make sure that the information and analysis are correct.

Reliance on AI can, however, mitigate intent. Here, Counsel is responsible for their negligence and misplaced reliance on AI, but there is no indication—and this court does not believe —that they purposely misled the court. They did not decide to make up a case. Misplaced reliance or negligence does not always equal intentional misrepresentation."

In her latter response to the Court, Counsel apologised for her use of a "a non-firm approved general AI service".

Magistrate Judge found Plaintiff's Reply contained multiple citations that do not exist, quotes not found in the cited authorities, and material mischaracterizations of cases; court concluded counsel likely used generative AI and failed to verify outputs and recommended the District Court consider Rule 11 sanctions.

Counsel filed a motion containing case citations that did not exist; counsel admitted the citations were generated by Microsoft CoPilot and not independently verified. The court found a Rule 9011 violation, declined to impose monetary sanctions because of procedural limits and dismissal, and ordered CLE focused on AI ethics.

" When asked, Dwaard (his lawyer) stated that all this was caused by a problem converting a Word file to PDF. This statement raises questions. The court leaves these questions and the question of whether this violated Article 21 of the Dutch Code of Civil Procedure unanswered, because, on balance, Dwaard did not benefit from the incorrect representation of the facts in the statement of defense. " (Google Translate)

Although the tribunal did not address it as a case of hallucinations or misuse of artificial intelligence, the details make clear that this was very likely at issue.

While the decision is not public, further details have been reported by, uh, me:

🔗 Damien Charlotin, Niger’s proposal to disqualify Fernando Mantilla-Serrano from uranium mining arbitration is rejected; challenge procedure is marred by citations and authorities that couldn’t be borne out when scrutinized by co-arbitrators (Investment Arbitration Reporter, 28 August 2025).

"Counsel should take seriously its obligation to provide the Court with an accurate description of the law. See, e.g., United States v. Hayes, 763 F. Supp. 3d 1054 (E.D. Cal. 2025) (levying $1,500 in monetary sanctions against counsel personally for fictitious cases and quotations that the court suspected were produced using artificial intelligence),reconsideration denied, No. 2:24-CR-0280-DJC, 2025 WL 1067323 (E.D. Cal. Apr. 9, 2025); Grant v. City of Long Beach, 96 F.4th 1255 (9th Cir. 2024) (striking an appellant's brief and dismissing an appeal for materially misrepresenting or fabricating case citations). After New Materials freely accused opposing counsel of misstating the law, New Materials’ submission of non-existent quotes is troubling (Dkt. 30 at 5 (“Chenco's argument for remand collapses under the weight of its own misreading of the law”); id.at 7 (“Chenco fundamentally misrepresents the applicable removal standard”); Dkt. 34 at 1 (“The proposed sur-reply ... is necessary to address new legal misstatements ....”); id. at 2-3 (“Chenco's failure to address this standard ... misstates controlling law and warrants correction.”)). Accordingly, the Court reminds counsel of their duties to act according to the Idaho Rules of Professional Conduct."

The court issued an Order to Show Cause, and the plaintiff's Counsel acknowledged responsibility but did not explicitly admit to using AI. The court determined that the counsel violated Rule 11(b)(2) by failing to verify the accuracy of the citations and imposed several sanctions:

• "The pro hac vice status of Counsel shall be revoked and Counsel will be removed from this case;
• Plaintiff’s Opening Brief shall be stricken;
• Counsel will be ordered to promptly serve a copy of this Order on Plaintiff, who will in turn be afforded time to engage new counsel or proceed as a self-represented litigant;
• Counsel will be ordered to write a letter to the three Judges to whom she attributed fictitious cases, [...], notifying them of her use of fake cases with their respective names attached;
• Counsel will be ordered to transmit a copy of this Order to every Judge who presides over any case in which Counsel is attorney of record; and
• The Clerk of Court’s Office will be directed to serve a copy of this Order on the Washington State Bar Association, of which Counsel is a member. If Counsel is a member of any other state’s bar, she shall serve a copy of this Order on that state’s bar office."

The court identified issues with the use of artificial intelligence in preparing written submissions. The submissions contained fabricated citations and fictitious quotes, which were initially filed as joint submissions by the defense and prosecution. Upon discovery, the defense counsel took responsibility, citing the use of AI without proper verification. The court allowed revised submissions to be filed, emphasizing the importance of accuracy in legal documents and the responsible use of AI. No professional sanctions or monetary penalties were imposed, but the court reiterated the need for adherence to guidelines on AI use in litigation.

Underlying judgment (in which the hallucination had been observed) is here.

"The appearance of hallucinated citations in briefs generated from AI is no longer in its nascent stage. Regrettably, the number and regularity with which courts have been faced with hallucinations in court filings continues to rise both in this country and abroad. See Damien Charlotin, AI Hallucination Cases, (Aug. 6, 2025)https://www.damiencharlotin.com/hallucinations/ (database tracking legal decisions “in caseswhere generative AI produced hallucinated content,” evidencing 255 cases to date) (hereinafter “Charlotin Database”).

[...]

By far, the majority of courts impose sanctions upon the offending lawyer for this sort of conduct and warnings or reprimands have been meted out in cases typically involving pro se litigants. See Charlotin Database, supra. However, there are circumstances where, in the Court’s discretion, monetary sanctions have not been imposed notwithstanding the violation of Rule 11."

The court imposed a monetary sanction of $1,000 on the appellate counsel and ordered a report to be sent to the Illinois Attorney Registration and Disciplinary Commission (ARDC).

The appellate court dismissed the appeals due to lack of jurisdiction and ordered respondent's appellate Counsel to pay $1,000 in monetary sanctions for violating Illinois Supreme Court Rule 375 by citing multiple cases that did not stand for the propositions of law for which they were cited. The court also ordered a copy of the decision to be sent to the Illinois Attorney Registration and Disciplinary Commission.

Counsel had previously admitted to using AI to prepare briefs in another case (in Re Boy) without thoroughly reviewing the work-product, leading to similar issues. The court found it reasonable to conclude AI was used in this case as well, although not explicitly admitted by Counsel.

"37. The Court does not accept that there is a difference between the oral and written submissions. In fact, had there not been a request by the Plaintiff to respond, the Court would have ruled on the oral submission and such ruling could have significantly relied on the submissions of Defence Counsel. The implications are severe and serious when the Court cannot accept Counsel's assertion to be truthful and cases to be real. The risk of harm to the integrity of the judicial process is real and could bring the system into disrepute.

[...]

40. The Court does not accept Ms. Taylor's submission that the fictitious cases were verified before layover and that they were only used in speaking points and were "not intended to form the official record." I find there is no distinction between the speaking points, oral submissions and written submission. They are all submissions advanced by Counsel intended for the Court to rely on them in the process of decision making. The purpose of which was to guide a judgment in your client's favour. The attempt to draw such a distinction is one without merit and the Court rejects same without more."

" According to father, “Welfare and Institutions Code § 350(b) states, ‘Except where otherwise provided by this code, the procedures for the trials or hearings shall be in accordance with that prescribed by law for civil cases.’ ” Section 350, subdivision (b) does not so provide, and we have been unable to verify the legal source for this quotation.3 With respect to Code of Civil Procedure, section 1013, subdivision (a), the statute addresses service by mail, facsimile and electronic service, etc., and provides, in relevant part, for an additional 10 calendar days “if either the place of mailing or the place of address is outside the State of California but within the United States.” "

Footnote 3 then reads:

" Counsel is reminded that conduct of California attorneys is governed by the State Bar Rules of Professional Conduct. In particular, rule 3.3(a)(1) places a duty on counsel not to “knowingly make a false statement of fact or law to a tribunal...,” as does Business and Professions Code section 6068, subdivision (d). This duty is violated when counsel knowingly presents statutory authority to the court that does not exist. A person's “knowledge” under this rule may be inferred from the circumstances. (Rules Prof. Conduct, rule 1.0.1(f).) Moreover, the inclusion of nonexistent legal authority may also be considered a “matter [that is] not reasonably material to the appeal's determination.” (Cal. Rules of Court, rule 8.276(a)(2).) Failure to adhere to these standards of practice and rules may result in sanctions which themselves may be mandatorily reportable to the California State Bar. (Bus. & Prof. Code, § 6068, subd. (o)(3).) "

After being ordered to show cause, Counsel submitted a brief that compounded the issue.

"The Court was dismayed to see that some of the discussion/analysis provided by Mr. Chinweze appeared to be from artificial intelligence generated case summaries which was indicated by the inclusion several times of the statement that "Some case metadata and case summaries were written with the help of AI." Another indication that the paragraphs of discussion/analysis were copied from another source or were generated by AI is that several paragraphs have superscript for footnotes, but no footnotes are found on any page of the Appendix."

The court then cited this database to highlight the growing problem of hallucinations in legal cases.

Counsel was ordered to pay 1,000 USD to the Lawyers Fund for Client Protection, and he was referred to the bar authorities.

The court also encouraged, but did not require, Plaintiffs' counsel to complete a CLE course on LLMs and legal ethics.

The court imposed a fine of 1 salário-mínimo for bad faith litigation and notified the OAB for potential professional sanctions.

In their Response, Counsel confessed to the use of AI tools in their Response to the OSC.

(As recounted by Above the Law, the law firm involved quickly deleted a recent post they made about using AI.)

In the Order, the judge prefaced her findings by noting that "Even in cases like this one, where lawyers who cite AI hallucinations accept responsibility and apologize profusely, much damage is done. The opposing party expends resources identifying and exposing the fabrication; the court spends time reviewing materials, holding hearings, deliberating about sanctions, and explaining its ruling; the substance of the case is delayed; and public confidence about the trustworthiness of legal proceedings may be diminished."

The court further reasoned that "At the threshold, the court rejects the invitation to consider that actual authorities stand for the proposition that the bogus authorities were offered to support. That is a stroke of pure luck for these lawyers, and one that did not remediate the waste and harm their misconduct wrought. Further, any sanctions discount on this basis would amplify the siren call of unverified AI for lawyers who are already confident in their legal conclusion. This court will have no part of that."

It added that: "Likewise, the court rejects the invitation to consider that the involved lawyers and firm have been deeply embarrassed in media reports. For many very good reasons, courts traditionally have not relied on the media to do the difficult work of professional discipline, and this court is not about to start."

Counsel admitted his law clerk used AI tools and failed to verify the citations. The court imposed a $2,500 monetary sanction, required completion of CLE in ethics/legal technology, and ordered service of the sanction order to the plaintiff.

Counsel was sanctioned for citing eight nonexistent cases in briefs filed on behalf of his client, in an appeal concerning the termination of parental rights. The court found that Counsel violated Illinois Supreme Court Rule 375 by submitting fictitious case citations generated by AI without verification. As a result, he was ordered to disgorge $6,925.62 received for his work on the appeal and pay an additional $1,000 in monetary sanctions. The court also directed that a copy of the opinion be sent to the Illinois Attorney Registration and Disciplinary Commission.