AI Hallucination Cases

""101    The use of AI technology in the Courts has been the subject of judicial observations, particularly regarding legal practitioners who are subject to professional and ethical obligations and responsibilities. However, as Bell CJ observed in May v Costaras [2025] NSWCA 178 at [15], with whom Payne JA and McHugh JA agreed, in the context of considering the use of AI in the preparation of submissions, that “(a)ll litigants are under a duty not to misled the court or their opponent.” The reliance on unverified materials produced by generative AI does have the potential to misled the Court.

102    Although the termed used in relation to erroneously generated references by AI is “hallucinations”, this is a term which seeks to legitimise the use of AI. More properly, such erroneously generated references are simply fabricated, fictional, false, fake and as such could be misleading.

103    All persons appearing before the Court have a duty to verify that the case law and legislation referred to and relied on, is accurate and that such materials actually exist. The references in Ayinde at [85] and [86] and in Costaras at [14]-[15], to matters involving litigants who are acting in person who rely on AI generated material clearly supports the position that all are required to verify the submissions made to the Court. There are many publicly available legal research websites, some which are accessible without a fee. Further, and without attempting to be exhaustive, the Queensland Supreme Court library is open and available to the public.

104    The use of generative AI to prepare submissions that may include fake authorities will nearly always introduce added costs, complexity and add to the burden of other parties and to the Court: Costaras at [16] and [49]. I gratefully adopt the observation from Ayinde at [9] that “(t)here are serious implications for the administration of justice and public confidence in the justice system if artificial intelligence is misused.” The Court in Ayinde observed from [10] to [31] the existing guidance, regulatory duties of the profession, referrals and the Court’s powers. Matters which are within the Court’s own domain include in the most serious of cases contempt of Court. The observations in Ayinde at [26]-[28] regarding contempt of court are not limited to legal practitioners.

105    As the reasons above demonstrate, the circumstances of this case involved many fake authorities, fabricated quotes and false propositions. It is unhelpful for the Court to be referred to, and for parties to rely on, such matters."

"The case at bar epitomizes the concern. Inordinate judicial resources were expended on reviewing cases cited by Plaintiff that did not exist. No doubt Plaintiff’s opponent in this litigation was forced to expend similar energies. Here, too, as noted, certain cases Plaintiff cited in support of her arguments stood for the opposite result from that which Plaintiff stated in her briefs. This kind of activity not only wastes precious and limited judicial resources, but it also drives up the cost of litigation unnecessarily for those who must defend against or seek to prosecute claims on behalf of paying clients, given the underpinnings of the American Rule that attaches to most civil litigation in this country."

The debtor, Sonja Helvig DeRosa-Grund, engaged in substantial abuse of the Chapter 13 process by filing numerous frivolous motions and making false allegations against opposing counsel. The debtor repeatedly cited non-existent case law and fabricated quotes from existing cases, despite being warned about this behavior. The court dismissed the case with prejudice for one year, terminated the debtor's ECF filing privileges, and imposed additional protective measures to prevent future abuse. The court found that the debtor violated Bankruptcy Rule 9011(b) by making arguments based on non-existent case law and misquoting cases.

The court identified issues with the use of artificial intelligence in preparing written submissions. The submissions contained fabricated citations and fictitious quotes, which were initially filed as joint submissions by the defense and prosecution. Upon discovery, the defense counsel took responsibility, citing the use of AI without proper verification. The court allowed revised submissions to be filed, emphasizing the importance of accuracy in legal documents and the responsible use of AI. No professional sanctions or monetary penalties were imposed, but the court reiterated the need for adherence to guidelines on AI use in litigation.

The court imposed sanctions under Rule 11, requiring Mr. Owoc to complete 10 hours of community service and to certify the accuracy of legal citations in future filings if AI is used. No monetary penalty was imposed.

"Second, the protester's explanation--that it was “manual mismatches in secondary summaries” that caused the citation errors (Protester's Resp., Aug. 8, 2025, at 1)--does not meaningfully explain the number of citation errors in the protester's filings. Indeed, Oready's patently erroneous citations are far removed from mere typographical or scrivener's errors, and instead, bear the hallmarks of the use of a large-language model or other artificial intelligence (AI) without adequate verification that the generated results were accurate. "

" We have the authority to sanction Wife under Florida Rule of Appellate Procedure 9.410(a) for failure to comply with Florida Rule of Appellate Procedure 9.210(c). See Gutierrez v. Gutierrez, 399 So. 3d 1185, 1188 (Fla. 3d DCA 2024). However, we decline to do so. Our decision is tempered by the fact that Wife is defending a judgment on appeal in an unrepresented capacity, Husband has not sought the imposition of sanctions, and Wife has not brought any other meritless or frivolous filings in this Court. See id. at 1187–88; Al-Hamim, 564 P.3d at 1125–26. Instead, we admonish Wife for her counterfeit brief and warn her that the Court will not regard similar infractions as mildly in the future. "

Underlying judgment (in which the hallucination had been observed) is here.

"The appearance of hallucinated citations in briefs generated from AI is no longer in its nascent stage. Regrettably, the number and regularity with which courts have been faced with hallucinations in court filings continues to rise both in this country and abroad. See Damien Charlotin, AI Hallucination Cases, (Aug. 6, 2025)https://www.damiencharlotin.com/hallucinations/ (database tracking legal decisions “in caseswhere generative AI produced hallucinated content,” evidencing 255 cases to date) (hereinafter “Charlotin Database”).

[...]

By far, the majority of courts impose sanctions upon the offending lawyer for this sort of conduct and warnings or reprimands have been meted out in cases typically involving pro se litigants. See Charlotin Database, supra. However, there are circumstances where, in the Court’s discretion, monetary sanctions have not been imposed notwithstanding the violation of Rule 11."

Melvin Graham-Jackson, a pro se litigant, filed an appeal with false legal citations in his briefs, citing non-existent legal authorities and unrelated cases. The Court of Appeals identified this as a violation of Wisconsin's appellate rules. Despite being notified of these issues in the respondents' brief, Graham-Jackson continued to use false citations in his reply brief. The court imposed two sanctions: a $500 monetary penalty payable to the defendants to offset their legal expenses, and a requirement for Graham-Jackson to submit an affidavit in any future appeals certifying the accuracy and relevance of his legal citations.

The court imposed a monetary sanction of $1,000 on the appellate counsel and ordered a report to be sent to the Illinois Attorney Registration and Disciplinary Commission (ARDC).

The appellate court found Husband's counsel made multiple misleading or incorrect citations and attributed quotations to cases that do not contain them. The court noted some citations used incorrect reporters that pointed to unrelated decisions, concluded the cited authorities did not support counsel's propositions (and in some instances contradicted them), forwarded the decision to the State Bar for possible ethical violations, and awarded costs to the prevailing party.

The appellate court dismissed the appeals due to lack of jurisdiction and ordered respondent's appellate Counsel to pay $1,000 in monetary sanctions for violating Illinois Supreme Court Rule 375 by citing multiple cases that did not stand for the propositions of law for which they were cited. The court also ordered a copy of the decision to be sent to the Illinois Attorney Registration and Disciplinary Commission.

Counsel had previously admitted to using AI to prepare briefs in another case (in Re Boy) without thoroughly reviewing the work-product, leading to similar issues. The court found it reasonable to conclude AI was used in this case as well, although not explicitly admitted by Counsel.

The motion containing the fabrications had been withdrawn after the litigant was confronted, but the sanction was imposed nonetheless.

"37. The Court does not accept that there is a difference between the oral and written submissions. In fact, had there not been a request by the Plaintiff to respond, the Court would have ruled on the oral submission and such ruling could have significantly relied on the submissions of Defence Counsel. The implications are severe and serious when the Court cannot accept Counsel's assertion to be truthful and cases to be real. The risk of harm to the integrity of the judicial process is real and could bring the system into disrepute.

[...]

40. The Court does not accept Ms. Taylor's submission that the fictitious cases were verified before layover and that they were only used in speaking points and were "not intended to form the official record." I find there is no distinction between the speaking points, oral submissions and written submission. They are all submissions advanced by Counsel intended for the Court to rely on them in the process of decision making. The purpose of which was to guide a judgment in your client's favour. The attempt to draw such a distinction is one without merit and the Court rejects same without more."

"Were Advani a lawyer, the Court would consider imposing sanctions on her. But in view of the fact that she is not a lawyer and of the dismissal of this case, the Court declines to pursue the matter further and merely warns Advani that presentation of false citations, quotations, and holdings in the future may indeed result in the imposition of sanctions."

" According to father, “Welfare and Institutions Code § 350(b) states, ‘Except where otherwise provided by this code, the procedures for the trials or hearings shall be in accordance with that prescribed by law for civil cases.’ ” Section 350, subdivision (b) does not so provide, and we have been unable to verify the legal source for this quotation.3 With respect to Code of Civil Procedure, section 1013, subdivision (a), the statute addresses service by mail, facsimile and electronic service, etc., and provides, in relevant part, for an additional 10 calendar days “if either the place of mailing or the place of address is outside the State of California but within the United States.” "

Footnote 3 then reads:

" Counsel is reminded that conduct of California attorneys is governed by the State Bar Rules of Professional Conduct. In particular, rule 3.3(a)(1) places a duty on counsel not to “knowingly make a false statement of fact or law to a tribunal...,” as does Business and Professions Code section 6068, subdivision (d). This duty is violated when counsel knowingly presents statutory authority to the court that does not exist. A person's “knowledge” under this rule may be inferred from the circumstances. (Rules Prof. Conduct, rule 1.0.1(f).) Moreover, the inclusion of nonexistent legal authority may also be considered a “matter [that is] not reasonably material to the appeal's determination.” (Cal. Rules of Court, rule 8.276(a)(2).) Failure to adhere to these standards of practice and rules may result in sanctions which themselves may be mandatorily reportable to the California State Bar. (Bus. & Prof. Code, § 6068, subd. (o)(3).) "