AI Hallucination Cases

Counsel filed a motion containing case citations that did not exist; counsel admitted the citations were generated by Microsoft CoPilot and not independently verified. The court found a Rule 9011 violation, declined to impose monetary sanctions because of procedural limits and dismissal, and ordered CLE focused on AI ethics.

" When asked, Dwaard (his lawyer) stated that all this was caused by a problem converting a Word file to PDF. This statement raises questions. The court leaves these questions and the question of whether this violated Article 21 of the Dutch Code of Civil Procedure unanswered, because, on balance, Dwaard did not benefit from the incorrect representation of the facts in the statement of defense. " (Google Translate)

Although the tribunal did not address it as a case of hallucinations or misuse of artificial intelligence, the details make clear that this was very likely at issue.

While the decision is not public, further details have been reported by, uh, me:

🔗 Damien Charlotin, Niger’s proposal to disqualify Fernando Mantilla-Serrano from uranium mining arbitration is rejected; challenge procedure is marred by citations and authorities that couldn’t be borne out when scrutinized by co-arbitrators (Investment Arbitration Reporter, 28 August 2025).

"Counsel should take seriously its obligation to provide the Court with an accurate description of the law. See, e.g., United States v. Hayes, 763 F. Supp. 3d 1054 (E.D. Cal. 2025) (levying $1,500 in monetary sanctions against counsel personally for fictitious cases and quotations that the court suspected were produced using artificial intelligence),reconsideration denied, No. 2:24-CR-0280-DJC, 2025 WL 1067323 (E.D. Cal. Apr. 9, 2025); Grant v. City of Long Beach, 96 F.4th 1255 (9th Cir. 2024) (striking an appellant's brief and dismissing an appeal for materially misrepresenting or fabricating case citations). After New Materials freely accused opposing counsel of misstating the law, New Materials’ submission of non-existent quotes is troubling (Dkt. 30 at 5 (“Chenco's argument for remand collapses under the weight of its own misreading of the law”); id.at 7 (“Chenco fundamentally misrepresents the applicable removal standard”); Dkt. 34 at 1 (“The proposed sur-reply ... is necessary to address new legal misstatements ....”); id. at 2-3 (“Chenco's failure to address this standard ... misstates controlling law and warrants correction.”)). Accordingly, the Court reminds counsel of their duties to act according to the Idaho Rules of Professional Conduct."

The court identified issues with the use of artificial intelligence in preparing written submissions. The submissions contained fabricated citations and fictitious quotes, which were initially filed as joint submissions by the defense and prosecution. Upon discovery, the defense counsel took responsibility, citing the use of AI without proper verification. The court allowed revised submissions to be filed, emphasizing the importance of accuracy in legal documents and the responsible use of AI. No professional sanctions or monetary penalties were imposed, but the court reiterated the need for adherence to guidelines on AI use in litigation.

Underlying judgment (in which the hallucination had been observed) is here.

"The appearance of hallucinated citations in briefs generated from AI is no longer in its nascent stage. Regrettably, the number and regularity with which courts have been faced with hallucinations in court filings continues to rise both in this country and abroad. See Damien Charlotin, AI Hallucination Cases, (Aug. 6, 2025)https://www.damiencharlotin.com/hallucinations/ (database tracking legal decisions “in caseswhere generative AI produced hallucinated content,” evidencing 255 cases to date) (hereinafter “Charlotin Database”).

[...]

By far, the majority of courts impose sanctions upon the offending lawyer for this sort of conduct and warnings or reprimands have been meted out in cases typically involving pro se litigants. See Charlotin Database, supra. However, there are circumstances where, in the Court’s discretion, monetary sanctions have not been imposed notwithstanding the violation of Rule 11."

The court imposed a monetary sanction of $1,000 on the appellate counsel and ordered a report to be sent to the Illinois Attorney Registration and Disciplinary Commission (ARDC).

The appellate court found Husband's counsel made multiple misleading or incorrect citations and attributed quotations to cases that do not contain them. The court noted some citations used incorrect reporters that pointed to unrelated decisions, concluded the cited authorities did not support counsel's propositions (and in some instances contradicted them), forwarded the decision to the State Bar for possible ethical violations, and awarded costs to the prevailing party.

The appellate court dismissed the appeals due to lack of jurisdiction and ordered respondent's appellate Counsel to pay $1,000 in monetary sanctions for violating Illinois Supreme Court Rule 375 by citing multiple cases that did not stand for the propositions of law for which they were cited. The court also ordered a copy of the decision to be sent to the Illinois Attorney Registration and Disciplinary Commission.

Counsel had previously admitted to using AI to prepare briefs in another case (in Re Boy) without thoroughly reviewing the work-product, leading to similar issues. The court found it reasonable to conclude AI was used in this case as well, although not explicitly admitted by Counsel.

"37. The Court does not accept that there is a difference between the oral and written submissions. In fact, had there not been a request by the Plaintiff to respond, the Court would have ruled on the oral submission and such ruling could have significantly relied on the submissions of Defence Counsel. The implications are severe and serious when the Court cannot accept Counsel's assertion to be truthful and cases to be real. The risk of harm to the integrity of the judicial process is real and could bring the system into disrepute.

[...]

40. The Court does not accept Ms. Taylor's submission that the fictitious cases were verified before layover and that they were only used in speaking points and were "not intended to form the official record." I find there is no distinction between the speaking points, oral submissions and written submission. They are all submissions advanced by Counsel intended for the Court to rely on them in the process of decision making. The purpose of which was to guide a judgment in your client's favour. The attempt to draw such a distinction is one without merit and the Court rejects same without more."

" According to father, “Welfare and Institutions Code § 350(b) states, ‘Except where otherwise provided by this code, the procedures for the trials or hearings shall be in accordance with that prescribed by law for civil cases.’ ” Section 350, subdivision (b) does not so provide, and we have been unable to verify the legal source for this quotation.3 With respect to Code of Civil Procedure, section 1013, subdivision (a), the statute addresses service by mail, facsimile and electronic service, etc., and provides, in relevant part, for an additional 10 calendar days “if either the place of mailing or the place of address is outside the State of California but within the United States.” "

Footnote 3 then reads:

" Counsel is reminded that conduct of California attorneys is governed by the State Bar Rules of Professional Conduct. In particular, rule 3.3(a)(1) places a duty on counsel not to “knowingly make a false statement of fact or law to a tribunal...,” as does Business and Professions Code section 6068, subdivision (d). This duty is violated when counsel knowingly presents statutory authority to the court that does not exist. A person's “knowledge” under this rule may be inferred from the circumstances. (Rules Prof. Conduct, rule 1.0.1(f).) Moreover, the inclusion of nonexistent legal authority may also be considered a “matter [that is] not reasonably material to the appeal's determination.” (Cal. Rules of Court, rule 8.276(a)(2).) Failure to adhere to these standards of practice and rules may result in sanctions which themselves may be mandatorily reportable to the California State Bar. (Bus. & Prof. Code, § 6068, subd. (o)(3).) "

The court found multiple filings and exhibits from the plaintiffs to be fabricated or misleading (including a forged email and a fabricated news-image) and that plaintiffs repeatedly misrepresented legal authority (e.g., falsely attributing authorship of McGirt). The court relied on these findings in its Rule 11/§1927/inherent-power sanctions analysis, granted sanctions, dismissed the amended complaint with prejudice, and set a hearing to determine reasonable attorneys' fees.

After being ordered to show cause, Counsel submitted a brief that compounded the issue.

"The Court was dismayed to see that some of the discussion/analysis provided by Mr. Chinweze appeared to be from artificial intelligence generated case summaries which was indicated by the inclusion several times of the statement that "Some case metadata and case summaries were written with the help of AI." Another indication that the paragraphs of discussion/analysis were copied from another source or were generated by AI is that several paragraphs have superscript for footnotes, but no footnotes are found on any page of the Appendix."

The court then cited this database to highlight the growing problem of hallucinations in legal cases.

Counsel was ordered to pay 1,000 USD to the Lawyers Fund for Client Protection, and he was referred to the bar authorities.

The court also encouraged, but did not require, Plaintiffs' counsel to complete a CLE course on LLMs and legal ethics.

The court imposed a fine of 1 salário-mínimo for bad faith litigation and notified the OAB for potential professional sanctions.

In their Response, Counsel confessed to the use of AI tools in their Response to the OSC.

(As recounted by Above the Law, the law firm involved quickly deleted a recent post they made about using AI.)

In the Order, the judge prefaced her findings by noting that "Even in cases like this one, where lawyers who cite AI hallucinations accept responsibility and apologize profusely, much damage is done. The opposing party expends resources identifying and exposing the fabrication; the court spends time reviewing materials, holding hearings, deliberating about sanctions, and explaining its ruling; the substance of the case is delayed; and public confidence about the trustworthiness of legal proceedings may be diminished."

The court further reasoned that "At the threshold, the court rejects the invitation to consider that actual authorities stand for the proposition that the bogus authorities were offered to support. That is a stroke of pure luck for these lawyers, and one that did not remediate the waste and harm their misconduct wrought. Further, any sanctions discount on this basis would amplify the siren call of unverified AI for lawyers who are already confident in their legal conclusion. This court will have no part of that."

It added that: "Likewise, the court rejects the invitation to consider that the involved lawyers and firm have been deeply embarrassed in media reports. For many very good reasons, courts traditionally have not relied on the media to do the difficult work of professional discipline, and this court is not about to start."

Counsel admitted his law clerk used AI tools and failed to verify the citations. The court imposed a $2,500 monetary sanction, required completion of CLE in ethics/legal technology, and ordered service of the sanction order to the plaintiff.

Counsel was sanctioned for citing eight nonexistent cases in briefs filed on behalf of his client, in an appeal concerning the termination of parental rights. The court found that Counsel violated Illinois Supreme Court Rule 375 by submitting fictitious case citations generated by AI without verification. As a result, he was ordered to disgorge $6,925.62 received for his work on the appeal and pay an additional $1,000 in monetary sanctions. The court also directed that a copy of the opinion be sent to the Illinois Attorney Registration and Disciplinary Commission.

"The first reason I issue sanctions stems from [Counsel]'s claim of ignorance—he asserts he didn't know the use of AI in general and ChatGPT in particular could result in citations to fake cases. Mr. Nield disputes the court's statement in Wadsworth v. Walmart Inc. (D. Wyo. 2025) that it is "well-known in the legal community that AI resources generate fake cases." Indeed, [Counsel] aggressively chides that assertion, positing that "in making that statement, the Wadsworth court cited no study, law school journal article, survey of attorneys, or any source to support this blanket conclusion."

I find [Counsel]'s position troubling. At this point, to be blunt, any lawyer unaware that using generative AI platforms to do legal research is playing with fire is living in a cloud."

[...]

"If anything, [Counsel]’s alleged lack of knowledge of ChatGPT’s shortcomings leads me to do what courts have been doing with increasing frequency: announce loudly and clearly (so that everyone hears and understands) that lawyers blindly relying on generative AI and citing fake cases are violating Bankruptcy Rule 9011 and will be sanctioned"

Counsel submitted a response to an Order to Show Cause that included a false quote attributed to none other than Mata v. Avianca, a case about hallucinations.

In May, the court asked Counsel to show cause why they should not be sanctioned for filing briefs with hallucinations - especially since they continued filing hallucinated submissions after being warned about it.

In their Answer, Counsel revealed that "specific citations and quotes in question were inadvertently derived from internal draft text prepared using generative AI research tools designed to expedite legal research and brief drafting".

In the Order, the court noted that Counsel "was not candid to the Court when confronted about his use of AI, stating that some of these documents were “prepared under time constraints,” when he had nearly two more weeks before the deadline to submit his responses." The judge was also unimpressed by Counsel's attempt to shift the blame to a paralegal.

Finally, in the fee dispute order, the court cited this database to point out that its approach to award costs and fees was appropriate, especially given the egregiousness of the claimant's conduct in this case. The judge also took into account "the significant, if indirect, monetary losses that may arise from nonmonetary sanctions in other cases, such as loss of business and loss of reputation, or the monetary loss borne by a client when a motion or even an entire case is adversely decided due to counsel’s misuse of AI."

"Before getting into the nitty gritty of the issue above, as indicated when making reference to authorities cited by counsels for the respondent to this court, it behoves to say out of all cases referred, were non-existing and did not support the preposition. I have to emphasize that all lawyers have duties to the court, to their clients and to the administration of justice, duty to faithfully represent the law to the court, duty not to fabricate case precedents and not to mis-cite cases for propositions that they do not support and duty to use technology, conduct legal research, and prepare court documents competently. [...]

Without being to repetitive, advocates have a duty to ensure they do not mislead the court, whether through their own actions or failures to act, or by enabling or participating in any misleading conduct by their clients. The proper functioning of the justice system relies on the court’s ability to trust the honesty and integrity of legal practitioners, as well as their professional responsibility to present only arguments that are legitimately supported."

Plaintiff's Counsel cited nine non-existent cases in a response to a motion to dismiss, which were generated using AI software. The court found this to be a violation of Rule 11, as the citations were not checked for accuracy. Counsel admitted the error, apologized, and explained the circumstances, including staff transitions and the use of AI. The court imposed a $1000 sanction, required Mr. Braddy to attend a CLE course on AI ethics, and ordered reimbursement of Defendant's attorney fees and costs.

The submitted document was also an unedited draft containing instructional comments and placeholder text (e.g., "It is recommended to consult with a lawyer...") characteristic of an AI-generated template. The court held:

"The defendants' counsel rightly argued that referring to non-existent citations is particularly burdensome, because it causes a waste of time in checking all the references. Therefore, in light of both the severity of these imaginary citations and the contempt shown by the document as a whole, it is necessary to award exemplary costs in favor of the defendants, as well as costs to the state treasury. " (translation by Gemini 2.5 Pro.)

Counsel confessed having Claude Sonnet 4 to draft a legal submission, which included fabricated quotations from legal authorities. Counsel said he was pressed by time.

After holding that there is "no reason to distinguish between the submission of fabricated cases and the submission of fabricated quotations from real cases. In both postures, the attorney seeks to persuade the Court using legal authority that does not exist", the court held that Mr. Desmarais had violated Rule 11 of the Federal Rules of Civil Procedure by failing to verify the accuracy of the AI-generated content. Counsel was found to have acted in subjective bad faith, as he was aware of the potential for AI to hallucinate legal citations and failed to take corrective action even after the government pointed out the errors.

The court imposed a $1,000 monetary sanction and required Counsel to complete a CLE course on the ethical use of AI in legal practice and notify his client of the issue.

Counsel tried to blame the software he used, as well as an intern; court was unimpressed

This case was described and reviewed by Dean A. Blohm here.

Prior Order to Show Cause available here.

After reviewing - and dismissing - the factual allegations made by Counsel, and noting that they had submitted errata in parallel cases (dealing with other fabricated citations), the court swiftly concluded that they "have violated Rule 11 because they were not reasonable in certifying that the claims, defenses, and other legal contentions contained in Defendants’ Opposition to Motion in Limine [Doc. 283] were warranted by existing law or by a nonfrivolous argument for extending, modifying, or reversing existing law or for establishing new law."

Both counsel were sanctioned with a 3,000 USD fine, payable to the court.