AI Hallucination Cases

The Respondent, a self-represented litigant, used generative AI tools for drafting and preliminary research, leading to the citation of a non-existent case, 'Fontaine v Canada, 2004 FC 1777', in his Motion Record. The Court found this to be a fabricated citation, and the (allegedly) intended citation pointed to an irrelevant case.

The court further pointed out that the Respondent had already been caught fabricating citations in a previous proceeding. Despite acknowledging use of AI, the respondent had also failed to provide the declaration on this point required by the AI Practice Direction. The Court ordered the removal of the Motion Record from the file. Costs of $500 CAD were awarded to the Applicant.

In the case AQ v. BW, the applicant AQ claimed damages for the non-consensual sharing of an intimate image by the respondent BW. Both parties were self-represented. The tribunal found that BW shared an intimate image of AQ without consent, violating the Intimate Images Protection Act (IIPA). BW attempted to defend their actions by citing a fabricated version of CRTA section 92, which was identified as a hallucination likely generated by artificial intelligence. Judge held:

"16. I have considered my obligation to give sufficient reasons. I do not consider that obligation to include responding to arguments concocted by artificial intelligence that have no basis in law. I accept that artificial intelligence can be a useful tool to help people find the right language to present their arguments, if used properly. However, people who blindly use artificial intelligence often end up bombarding the CRT with endless legal arguments. They cannot reasonably expect the CRT to address them all. So, while I have reviewed all the parties’ materials and considered all their arguments, I have decided against addressing many of the issues they raise. If I do not address a particular argument in this decision, it is because the argument lacks any merit, is about something plainly irrelevant, or both."

The tribunal dismissed BW's defenses as baseless and awarded AQ $5,000 in damages and an additional $1,000 for time spent due to BW's submission of irrelevant evidence. The tribunal emphasized that arguments concocted by AI without legal basis would not be addressed.

The court noted that:

"both Lehr-Guthrie's and McDonald's briefs are replete with errors in their citations to case authority, such as repeated citation errors, references to nonexistent quotes, and incorrect statements about the cases (for instance, as noted above, the two cases McDonald cited for a proposition relating to the duty of impartiality don't even reference that duty). This suggests to us that the briefs may have been drafted with the use of generative artificial intelligence (GAI). “[U]sing a GAI tool to draft a legal document can pose serious risks if the user does not thoroughly review the tool's output.” Al-Hamim v. Star Hearthstone, LLC, 2024 COA 128, ¶ 32. Self-represented litigants must be particularly careful, as they “may not understand that a GAI tool may confidently respond to a query regarding a legal topic ‘even if the answer contains errors, hallucinations, falsehoods, or biases.’ ” Id. (citation omitted).4 We advise the parties that errors caused by GAI in future filings may result in sanctions. See id. at ¶ 41. "

The court warned that future errors caused by AI could result in sanctions.

"Compounding the problem, what generative AI lacks in precision, it more than makes up for in speed. Litigants who simply file the material that AI tools generate, without carefully reviewing it first for accuracy, have the potential to swamp courts with what appear at first glance to be legal arguments built on law and precedent, but which are in fact nothing of the sort. And not only are these problems in their own right, but they also heighten the two concerns the Court highlighted above—that defendants will be forced to spend more time and incur more costs parsing through copious baseless filings to defend an action, and that Courts will waste precious time doing the same in ruling on motions and moving matters along."

(Plaintiff acknowledged use of ChatGPT in a subsequent filing)

Plaintiff was eventually designated as vexatious litigant and his case dismissed with prejudice.

Defendant Holly Nelson, appearing pro se, filed a motion to dismiss that included a fabricated case citation, Finch v. Smith, which does not exist. The court inferred that Nelson used AI to draft the motion without verifying the accuracy of the citations. The court issued an order to show cause, requiring Nelson to justify why her filing does not violate Rule 11, or alternatively, to withdraw her motion. If she fails to do so, the court intends to strike her motion entirely.

In the case of Crespo v. Tesla, Inc., the pro se plaintiff, Leonardo Crespo, submitted discovery motions containing fabricated case citations and a false quote, which were identified as potentially generated by AI. The court ordered Crespo to show cause for these submissions and admitted to using AI in his filings. The court acknowledged Crespo's candor and imposed sanctions requiring him to apologize to the defendant's counsel and pay reasonable attorney's fees incurred by the defendant in addressing the fake citations.

(In a subsequent ruling, the court averred that the reasonable fees amount was 921 USD.)

"Generative AI can be beguiling, particularly when the task of representing yourself seems overwhelming. However, a litigant runs the risk that their case will be damaged, rather than helped, if they choose to use AI without taking the time to understand what it produces, and to confirm that it is both legally and factually accurate. "

Plaintiff Ravel Ferrera Parra, proceeding pro se, filed a lawsuit against the United States alleging financial harm due to misconduct by various judicial and governmental entities. The court dismissed the case for lack of jurisdiction, as the claims were not within the court's purview.

The court noted that Plaintiff's filings appeared to be assisted by AI, as evidenced by the rapid filing of responses tell-tale language ("Would you like additional affidavits, supporting exhibits, or further refinements before submission?"), the inclusion of fabricated case citations. "While Plaintiff’s use of AI, by itself, does not violate this Court’s Rules, Plaintiff’s citation to fake cases does."

The court further pointed out that:

"“It is no secret that generative AI programs are known to ‘hallucinate’ nonexistent cases.” Sanders, 176 Fed. Cl. at 169 (citation omitted). That appears to have happened here. When searching the Federal Claims Reporter for “Tucker v. United States, 71 Fed. Cl. 326 (2006),” Plaintiff’s citation brings the Court to the third page of Grapevine Imports, Ltd. v. United States, 71 Fed. Cl. 324, 326 (2006), a real tax case from this Court. Similarly, the AI used by Plaintiff in Sanders v. United States, 176 Fed. Cl. 163, 169 (2025) also made up a citation to a case called Tucker v. United States. Perhaps both AI programs hallucinated this case name based on the Tucker Act, this Court’s jurisdictional statute. Regardless, here, as in Sanders, the citation to a case called Tucker v. United States does not exist."

The court warned Plaintiff about the risks of using AI-generated content without verification but did not impose sanctions.

The court observed that the original motion to remand filed pro se by the plaintiff contained non-existent case law and falsified quotations. Although the court did not impose sanctions in this instance, it warned that future use of fake legal authority would result in a show cause order, including against the Counsel who later joined the case.

"Before addressing the merits of Schoene’s motion, the Court notes that Schoene cited several cases in her reply brief to support her motion to amend, including Butler v. Oregon, 218 Or. App. 114 (2008), Curry v. Actavis, Inc., 2017 LEXIS 139126 (D. Or. Aug. 30, 2017), Estate of Riddell v. City of Portland, 194 Or. App. 227 (2004), Hampton v. City of Oregon City, 251 Or. App. 206 (2012), and State v. Burris, 107 Or. App. 542 (1991). These cases, however, do not exist. Schoene’s false citations appear to be hallmarks of an artificial intelligence (“AI”) tool, such as ChatGPT. It is now well known that AI tools “hallucinate” fake cases. See Kruse v. Karlen, 692 S.W.3d 43, 52 (Mo. Ct. App. 2024) (noting, in February 2024, that the issue of fictitious cases being submitted to courts had gained “national attention”).6 In addition, the Court notes that a basic internet search seeking guidance on whether it is advisable to use AI tools to conduct legal research or draft legal briefs will explain that any legal authorities or legal analysis generated by AI needs to be verified. The Court cautions Schoene that she must verify the accuracy of any future citations she may include in briefing before this Court and other courts"

Referring to Ayinde, the judge held that "The principle is essentially the same - though I hasten to say that I would not push the analogy too far as to a factual comparison of the present case with that case and the error in the present case is not of the order of the misconduct in that case. However, appreciable judicial time was wasted on the issue - not least trying to find the source of the quotation. And it does illustrate:

• The vital importance of precision and accuracy in written submissions. That duty lies on lay litigants as much as on lawyers.
• That text in submissions formatted so as to convey that it is a direct and verbatim quotation from an identified source must be exactly that. Of course, it is permissible to edit the text (for example to exclude irrelevant content or by underlining for emphasis) but, if so, that it has been done must be apparent on the face of the document.
• That opposing parties are entitled to written submissions in good time to check them.

43. All that said, in a substantive sense, the issue is not vital to this case. The underlying proposition for which Mr Malone contends - that domestic courts must implement EU law - is uncontroversial. Not least for that reason, and in light also of the manner in which Mr Malone generally presented his case at the hearing, I am inclined to accept that there was no attempt or intention to mislead and accept also that Mr Malone has apologized for the error. It does not affect the outcome of the present motions."

"When the parties came before the court on 22 May 2025, there had been little time for the plaintiff, the first defendant and the court to examine the second defendant’s written submissions served late on the night before. It was nevertheless immediately apparent that the second defendant sought to rely upon authority and court rules which were not merely misstated but, in some circumstances, imaginary. I am satisfied that all of the judgments and rules referred to in the submissions of 21 May 2025 were misstated, non-existent, or both, and that Gen AI had been used to prepare these submissions.

An example was the citation of a decision of the Supreme Court of New South Wales described as “Wu v Wilks” (I will not provide the citation given in full, as there is a risk of it being picked up as genuine by other Gen AI: Luck v Secretary, Services Australia [2025] FCAFC 26 at [14]). There is no decision with this name, either in the Supreme Court of New South Wales or in any other jurisdictions. The caselaw citation given for “Wu v Wilks” belonged to a judgment on wholly unrelated material and the principles of law for which it was cited. All of the citations suffered similar problems.

I drew these issues to the attention of the second defendant and enquired whether she had used Gen AI in the preparation of her submissions and, if so, whether she was aware of the Practice Note. She acknowledged that she had done so but said this was because she had very little time to provide submissions in reply and was deeply distressed by these proceedings"

Claimant used Chat GPT to assist in drafting his grounds of appeal and skeleton argument. The documents included fabricated citations and misrepresented case summaries. Claimant admitted to using Chat GPT and apologized for the errors.

Compounding matters, the court suspected that the respondent had also used AI, since the cases cited in the Counsel's skeleton, though extant, did not support any of the propositions made - and Counsel was unable to explain how they got there.

"Artificial intelligence may ultimately prove a helpful tool to assist pro se litigants in bringing meritorious cases to the courts. In that way, artificial intelligence has the potential to contribute to the cause of justice. However, accessing any beneficial use of artificial intelligence requires carefully understanding its limitations. For example, if merely asked to write an opposition to an opposing party’s motion or brief, or to respond to a court order, an artificial intelligence program is likely to generate such a response, regardless of whether the response actually has an arguable basis in the law. Where the court or opposing party was correct on the law, the program will very likely generate a response or brief that includes a false statement of the law. And because artificial intelligence synthesizes many sources with varying degrees of trustworthiness, reliance on artificial intelligence without independent verification renders litigants unable to represent to the Court that the information in their filings is truthful."

In the case of J.R.V. v. N.L.V., the respondent, appearing in person, used a generative AI tool to prepare parts of her written argument. This resulted in the inclusion of citations to non-existent cases, known as 'hallucinations.' The claimant sought costs due to the need to research and respond to these false citations. The court acknowledged the issue but noted that the respondent was not represented by counsel and was unaware of the AI's capability to generate false citations. Moreover, the claimant was wrong as to the alleged non-existence of some citations. The court ordered the respondent to pay $200 in costs to the claimant.

"[49] Mr. Ohenhen submitted a statement of legal argument to the court in support of his arguments. In those documents, he referred to at least two non-existent or fake precedent court cases, one ostensibly from the Court of Appeal for Ontario and another ostensibly from the British Columbia Court of Appeal. In reviewing his materials after argument, I tried to access these cases and was unable to find them. I asked the parties to provide them to me.

[50] Mr. Ohenhen responded with a “clarification”, providing different citations to different cases. I asked for an explanation as to where the original citations came from, and specifically, whether they were generated by artificial intelligence. I have received no response to that query.

[51] While Mr. Ohenhen is not a lawyer with articulated professional responsibilities to the court, every person who submits authorities to the court has an obligation to ensure that those authorities exist. Simple CanLII searches would have revealed to Mr. Ohenhen that these were fictitious citations. Putting fictitious citations before the court misleads the court. It is unacceptable. Whether the cases are put forward by a lawyer or self-represented party, the adverse effect on the administration of justice is the same.

[52] Mr. Ohenhen’s failure to provide a direct and forthright answer to the court’s questions is equally concerning.

[53] Court processes are not voluntary suggestions, to be complied with if convenient or helpful to one’s case. The proper administration of justice requires parties to respect the rules and proceed in a forthright manner. That has not happened here.

[54] I have not attached any consequences to this conduct in this case. However, should such conduct be repeated in any court proceedings, Mr. Ohenhen should expect consequences. Other self-represented litigants should be aware that serious consequences from such conduct may well flow."

" Plaintiffs identify fictitious quotes and citations in their briefing to support their arguments. For example, Plaintiffs purport to quote language from S.H. Hold v. United States, 853 F.3d 1056, 1064 (9th Cir. 2017) and Green v. United States, 630 F.3d 1245, 1249 (9th Cir. 2011). (Reed 2, Dkt. No. 23 at 6.) However, the quoted language is nowhere found in those cases. Plaintiffs also cite to a case identified as “Horne v. Potter, 557 F.3d 953, 957 (9th Cir. 2009).” (Id. at 7.) That citation, however, is for a case titled Bell v. The Hershey Company. Plaintiffs appear to acknowledge they offered fictitious citations. (See Reed 2, Dkt. No. 27.) Plaintiffs are cautioned that providing fictitious cases and quotes will lead to sanctions. "

" This Court's local rules permit the use of generative artificial intelligence programs, but all parties, including pro se parties, must certify “that to the extent such a program was used, a human signatory of the document verified the accuracy of all generated text, including all citations and legal authority,” NECivR 7.1(d)(4)(B). The plaintiff's brief contains no such certification, nor does the plaintiff deny using artificial intelligence. See filing 27 at 9.
From the Court's review of the plaintiff's brief, most citations are not hallucinatory. But the plaintiff undeniably cites to cases that do not support the stated legal proposition. For example, the plaintiff asserts that “courts must ‘accept all factual allegations in the complaint as true.’ ” Filing 22 at 3. He attributes the quoted language to Braden v. Wal-Mart Stores, Inc., 588 F.3d 585, 595 (8th Cir. 2009). The quote is a boilerplate recitation of the standard on a motion to dismiss, and any number of cases contain that exact language—but not Braden, the case to which the plaintiff attributed the quote.

The plaintiff characterizes the mismatched citations as “[t]ypographical errors or minor misstatements.” Filing 27 at 9. That may be true for some of the mistakes—for example, the case Foman v. Davis, 371 U.S. 178 (1962) is correctly cited, and it immediately precedes a citation for the nonexistent “Foman v. Arnold,” which might be explained as a typographical error. See filing 22 at 13. But not all the mistakes are so innocent, such as the plaintiff's citation to the nonexistent “Brown v. Maple Tree Homes, Inc.,” purported to be a Nebraska case, the citation to which directs to a Wisconsin Supreme Court decision. Filing 22 at 12. This mistake resembles the hallucinatory citations seen in other cases, and is a mistake the plaintiff had an obligation to correct before filing his brief. See Strong v. Rushmore Loan Mgmt. Servs., LLC, No. 8:24-cv-352, 2025 WL 100904, at *6 (D. Neb. Jan. 15, 2025).

The plaintiff also asserts that while the citations may be erroneous, the statements of law are not. The Court agrees that the plaintiff made no substantive misrepresentations of law, so striking the brief is not the appropriate remedy. However, the plaintiff should take notice of the fact that such conduct, if repeated, could subject him to sanctions, even as a pro se litigant. See Ferris v. Amazon.com Servs., LLC, No. 3:24-cv-304, 2025 WL 1122235, at *2 (N.D. Miss. Apr. 16, 2025); Rubio v. D.C., No. 23-cv-719, 2024 WL 4957373, at *4 (D.D.C. Dec. 3, 2024); Vargas v. Salazar, No. 4:23-CV-4267, 2024 WL 4804091, at *3 (S.D. Tex. Nov. 1, 2024); Martin v. Hawai‘i, No. 24-cv-294, 2024 WL 3877013, at *2 (D. Haw. Aug. 20, 2024)."

"The pervasive misrepresentations of the law in Lucas' filings cannot be tolerated. It serves to make a mockery of the judicial process. It causes an enormous waste of judicial resources to try to find cited cases that do not exist and to determine whether a cited authority is relevant or binding, only to determine that most are neither.

In like fashion, Lucas' adversaries also must run to ground the nonexistent cases or address patently irrelevant ones. The adversaries must thus incur needless legal fees and expenses caused by Lucas' pervasive citations to nonexistent or irrelevant cases.

[...]

However, as previously noted Lucas appears to be judgment proof so monetary sanctions likely will not deter her from the abusive practices reflected in her filings and in her previously announced, consistently followed, abuse of the litigation proceedings created by the Individuals with Disabilities Education Act, 20 U.S.C. § 1400, et seq. (“IDEA”). So, the Court must find some other way to protect the interests of justice and to deter Lucas from the abuses which have come to mark her approach to participation as a defendant in the judicial process.

In this case, the most appropriate remedy is to strike Lucas' filings where they are burdensome by virtue of volume and exceed permitted page limits, where they are not cogent or understandable (when given the generous latitude afforded pro se litigants), and where they misrepresent the law by citing nonexistent or utterly irrelevant cases."

In a subsequent Opinion, the court declined to reconsider or review its findings, pointing out that:

"To begin, it is unclear what Lucas means by "contested" citations. The citations that the Court found to not exist are not "contested." They simply do not exist. There is no contesting that fact because the Court checked each citation that was referenced in its MEMORANDUM OPINION, exactly as Lucas cited them (and through other research means), and could not find any citation that matched what Lucas cited. That research demonstrates that the Court's

findings are, in fact, supported rather than "[u]nsupported." Id.

Then, in no way did the Court "wrongly assume[]" that these citations to nonexistent legal authority were "'fabricated' due to the use of generative AI." Id. The Court meticulously checked every citation that it held did not exist in those decisions. Those decisions were not based on "assumptions" but, instead, on the fact that either (1) no case existed under the reporter citation, case name, or quotation that Lucas used, or (2) a case with the reporter citation did exist but was to an entirely different case than the one cited by Lucas and had no relevancy to the issues of this case. ECF No. 170, at 520. And, there was no incorrect assumption that those nonexistent legal authorities were generated, hallucinated, or fabricated by AI because Lucas admitted, on the record, to using AI when writing her filings with the Court. The fact that her citations to nonexistent legal authority are so pervasive, in volume and in location throughout her filings, can lead to only one plausible conclusion: that an AI program hallucinated them in an effort to meet whatever Lucas' desired outcome was based on the prompt that she put into the AI program. As the Court described in its MEMORANDUM OPINION, this is becoming an alarmingly prevalent occurrence common

to AI programs. Id. at 23-26. It is exceedingly clear that it occurred here.

[...]

The MOTION also complains that the Court did not give Lucas an "opportunity to verify or correct citations." Id.

Wholly apart from the fact that it is the litigant's (pro se or represented) burden to verify citations, there is no reason to have accorded Lucas the opportunity to verify because the problem was extensive and pervasive across at least six filings. Moreover, the Court actually did what should have been done before the MOTION was filed by determining that those citations do not exist. No further verification is necessary. And, after a diligent search, if the Court could not find the legal authorities that Lucas purported to rely upon and present as real and binding, it is a folly to believe that Lucas' efforts at "correction" would have returned anything different. Further, she could have taken the opportunity in this MOTION to go through—citation by citation—and "verify" or "correct" them to demonstrate to the Court that its findings were, in fact, incorrect, rather than just baldly and without evidence claiming them to be so. She did not do that."

The Complainant, who was self-represented, used artificial intelligence to assist in preparing her submissions, which included fictitious case citations. The Commission noted the potential seriousness of relying on fabricated citations but did not impose any sanctions on the Complainant, as she was self-represented. The judge held:

"In relation to the issue of the Complainant's reference to what appears to be fictitious case authorities, this is potentially a serious matter because it can be viewed as an attempt to mislead the Commission. If an admitted legal practitioner were to do this, there would be grounds to refer the practitioner to the Legal Services Commission for an allegation of misconduct.

[...]

Given that the Complainant is self-represented, I intend to take the same approach that I adopted in Goodchild and simply afford that part of the Complainant's submissions that deals with the two authorities no weight in determining the two applications. "

" Directing the Court to nonexistent authorities wastes the Court’s time, a resource meant to serve the public, not be monopolized by a single litigant making baseless arguments. "

In an earlier decision, the court had already warned the plaintiff against "any further legal misrepresentations in future communications".

After the Applicant confessed having relied on ChatGPT for the written phase of the proceedings, hearing was adjourned and the Court "cautioned the Applicant on relying on ChatGPT for legal advice and suggested that the Applicant may wish to seek legal advice from a lawyer about his application."

The court ultimately found that one of the authorities illustrated "the risk in relying on ChatGPT and Generative AI for legal advice. The Applicant’s description of the decision in this case in his submissions, for which he used ChatGPT to prepare, is clearly wrong. "

The judge pointed out that the plaintiffs' written request included what appeared to be a direct quote from one of her own previous judgments. However, upon examination, she found that not only did this quote not exist in the cited judgment, but the judgment itself did not even address the legal question at stake.

" The Court has serious reason to believe that Plaintiff used artificial intelligence tools to assist in drafting her objection. While the use of such tools is not prohibited, artificial intelligence often cites to legal authorities, like Cabrera, that do not exist. Continuing to cite to non-existent cases will result in sanctions up to and including dismissal. "

After being ordered to show cause, plaintiff admitted having used AI for several filings.