AI Hallucination Cases

Counsel submitted a response to an Order to Show Cause that included a false quote attributed to none other than Mata v. Avianca, a case about hallucinations.

In May, the court asked Counsel to show cause why they should not be sanctioned for filing briefs with hallucinations - especially since they continued filing hallucinated submissions after being warned about it.

In their Answer, Counsel revealed that "specific citations and quotes in question were inadvertently derived from internal draft text prepared using generative AI research tools designed to expedite legal research and brief drafting".

In the Order, the court noted that Counsel "was not candid to the Court when confronted about his use of AI, stating that some of these documents were “prepared under time constraints,” when he had nearly two more weeks before the deadline to submit his responses." The judge was also unimpressed by Counsel's attempt to shift the blame to a paralegal.

Finally, in the fee dispute order, the court cited this database to point out that its approach to award costs and fees was appropriate, especially given the egregiousness of the claimant's conduct in this case. The judge also took into account "the significant, if indirect, monetary losses that may arise from nonmonetary sanctions in other cases, such as loss of business and loss of reputation, or the monetary loss borne by a client when a motion or even an entire case is adversely decided due to counsel’s misuse of AI."

"Before getting into the nitty gritty of the issue above, as indicated when making reference to authorities cited by counsels for the respondent to this court, it behoves to say out of all cases referred, were non-existing and did not support the preposition. I have to emphasize that all lawyers have duties to the court, to their clients and to the administration of justice, duty to faithfully represent the law to the court, duty not to fabricate case precedents and not to mis-cite cases for propositions that they do not support and duty to use technology, conduct legal research, and prepare court documents competently. [...]

Without being to repetitive, advocates have a duty to ensure they do not mislead the court, whether through their own actions or failures to act, or by enabling or participating in any misleading conduct by their clients. The proper functioning of the justice system relies on the court’s ability to trust the honesty and integrity of legal practitioners, as well as their professional responsibility to present only arguments that are legitimately supported."

Plaintiff's Counsel cited nine non-existent cases in a response to a motion to dismiss, which were generated using AI software. The court found this to be a violation of Rule 11, as the citations were not checked for accuracy. Counsel admitted the error, apologized, and explained the circumstances, including staff transitions and the use of AI. The court imposed a $1000 sanction, required Mr. Braddy to attend a CLE course on AI ethics, and ordered reimbursement of Defendant's attorney fees and costs.

The submitted document was also an unedited draft containing instructional comments and placeholder text (e.g., "It is recommended to consult with a lawyer...") characteristic of an AI-generated template. The court held:

"The defendants' counsel rightly argued that referring to non-existent citations is particularly burdensome, because it causes a waste of time in checking all the references. Therefore, in light of both the severity of these imaginary citations and the contempt shown by the document as a whole, it is necessary to award exemplary costs in favor of the defendants, as well as costs to the state treasury. " (translation by Gemini 2.5 Pro.)

The Respondent, a self-represented litigant, used generative AI tools for drafting and preliminary research, leading to the citation of a non-existent case, 'Fontaine v Canada, 2004 FC 1777', in his Motion Record. The Court found this to be a fabricated citation, and the (allegedly) intended citation pointed to an irrelevant case.

The court further pointed out that the Respondent had already been caught fabricating citations in a previous proceeding. Despite acknowledging use of AI, the respondent had also failed to provide the declaration on this point required by the AI Practice Direction. The Court ordered the removal of the Motion Record from the file. Costs of $500 CAD were awarded to the Applicant.

Counsel confessed having Claude Sonnet 4 to draft a legal submission, which included fabricated quotations from legal authorities. Counsel said he was pressed by time.

After holding that there is "no reason to distinguish between the submission of fabricated cases and the submission of fabricated quotations from real cases. In both postures, the attorney seeks to persuade the Court using legal authority that does not exist", the court held that Mr. Desmarais had violated Rule 11 of the Federal Rules of Civil Procedure by failing to verify the accuracy of the AI-generated content. Counsel was found to have acted in subjective bad faith, as he was aware of the potential for AI to hallucinate legal citations and failed to take corrective action even after the government pointed out the errors.

The court imposed a $1,000 monetary sanction and required Counsel to complete a CLE course on the ethical use of AI in legal practice and notify his client of the issue.

Counsel tried to blame the software he used, as well as an intern; court was unimpressed

This case was described and reviewed by Dean A. Blohm here.

Prior Order to Show Cause available here.

After reviewing - and dismissing - the factual allegations made by Counsel, and noting that they had submitted errata in parallel cases (dealing with other fabricated citations), the court swiftly concluded that they "have violated Rule 11 because they were not reasonable in certifying that the claims, defenses, and other legal contentions contained in Defendants’ Opposition to Motion in Limine [Doc. 283] were warranted by existing law or by a nonfrivolous argument for extending, modifying, or reversing existing law or for establishing new law."

Both counsel were sanctioned with a 3,000 USD fine, payable to the court.

In the case AQ v. BW, the applicant AQ claimed damages for the non-consensual sharing of an intimate image by the respondent BW. Both parties were self-represented. The tribunal found that BW shared an intimate image of AQ without consent, violating the Intimate Images Protection Act (IIPA). BW attempted to defend their actions by citing a fabricated version of CRTA section 92, which was identified as a hallucination likely generated by artificial intelligence. Judge held:

"16. I have considered my obligation to give sufficient reasons. I do not consider that obligation to include responding to arguments concocted by artificial intelligence that have no basis in law. I accept that artificial intelligence can be a useful tool to help people find the right language to present their arguments, if used properly. However, people who blindly use artificial intelligence often end up bombarding the CRT with endless legal arguments. They cannot reasonably expect the CRT to address them all. So, while I have reviewed all the parties’ materials and considered all their arguments, I have decided against addressing many of the issues they raise. If I do not address a particular argument in this decision, it is because the argument lacks any merit, is about something plainly irrelevant, or both."

The tribunal dismissed BW's defenses as baseless and awarded AQ $5,000 in damages and an additional $1,000 for time spent due to BW's submission of irrelevant evidence. The tribunal emphasized that arguments concocted by AI without legal basis would not be addressed.

"Further, these inaccuracies signal that Plaintiff's counsel may have used AI to assist in the drafting of Plaintiff's Response (or otherwise counsel produced exceptionally sloppy work). In this regard, this Court's Chambers Rules include “Disclosure and Certification Requirements” for use of “Generative Artificial Intelligence” and expressly provide that an attorney or party must disclose in any document to be filed with the Court “that AI was used and the specific AI tool that was used” and to “certify in the document that the person has checked the accuracy of any portion of the document drafted by generative AI, including all citations and legal authority.” See id.6 No such disclosure and certification has been made in this case. The Court's Rules further provide that an attorney will be responsible for the contents of any documents prepared with generative AI, in accordance with Rule 11 of the Federal Rules of Civil Procedure, and that the failure to make the disclosure and certification “may result in the imposition of sanctions.”"

The court noted that:

"both Lehr-Guthrie's and McDonald's briefs are replete with errors in their citations to case authority, such as repeated citation errors, references to nonexistent quotes, and incorrect statements about the cases (for instance, as noted above, the two cases McDonald cited for a proposition relating to the duty of impartiality don't even reference that duty). This suggests to us that the briefs may have been drafted with the use of generative artificial intelligence (GAI). “[U]sing a GAI tool to draft a legal document can pose serious risks if the user does not thoroughly review the tool's output.” Al-Hamim v. Star Hearthstone, LLC, 2024 COA 128, ¶ 32. Self-represented litigants must be particularly careful, as they “may not understand that a GAI tool may confidently respond to a query regarding a legal topic ‘even if the answer contains errors, hallucinations, falsehoods, or biases.’ ” Id. (citation omitted).4 We advise the parties that errors caused by GAI in future filings may result in sanctions. See id. at ¶ 41. "

The court warned that future errors caused by AI could result in sanctions.

"In a prior order, we struck portions of Walker's brief that included citations to nonexistent cases. We note that the arguments raised would not have changed the outcome of the appeal in any event. "

Counsel in charge apologised to the Court in a letter (available here), explaining that he had failed to supervise the work of his subordinates.

"Compounding the problem, what generative AI lacks in precision, it more than makes up for in speed. Litigants who simply file the material that AI tools generate, without carefully reviewing it first for accuracy, have the potential to swamp courts with what appear at first glance to be legal arguments built on law and precedent, but which are in fact nothing of the sort. And not only are these problems in their own right, but they also heighten the two concerns the Court highlighted above—that defendants will be forced to spend more time and incur more costs parsing through copious baseless filings to defend an action, and that Courts will waste precious time doing the same in ruling on motions and moving matters along."

(Plaintiff acknowledged use of ChatGPT in a subsequent filing)

Plaintiff was eventually designated as vexatious litigant and his case dismissed with prejudice.

"14    Here, the applicant's solicitor’s use of AI in the preparation of two court documents has given rise to cost, inconvenience and delay to the parties and has compromised the effectiveness of the administration of justice. But I do not consider the use of AI in this case means that it is appropriate to refer the solicitors’ conduct to the Victorian Legal Services Board. Here an inexperienced junior solicitor was given the task of preparing document citations for an amended pleading, and did so while working remotely and without access to the documents to be cited. In attempting to cite the relevant documents she used an (apparently AI-assisted) research tool which she considered had produced accurate citations when she previously used it. And as soon as Massar Briggs Law was told of the false citations the problem was addressed. The junior solicitor and the principal solicitor have apologised or expressed their regret to the other parties and the Court, and there was no suggestion that they were not genuine in doing so.

15    The junior solicitor took insufficient care in using Googe Scholar as the source of document citations in court documents, and in failing to check the citations against the physical and electronic copies of the cited documents that were held at Massar Briggs Law’s office. The error was centrally one of failing to check and verify the output of the search tool, which was contributed to by the inexperience of the junior solicitor and the failure of Mr Briggs to have systems in place to ensure that her work was appropriately supervised and checked. To censure those errors it is sufficient that these reasons be published."

Defendant Holly Nelson, appearing pro se, filed a motion to dismiss that included a fabricated case citation, Finch v. Smith, which does not exist. The court inferred that Nelson used AI to draft the motion without verifying the accuracy of the citations. The court issued an order to show cause, requiring Nelson to justify why her filing does not violate Rule 11, or alternatively, to withdraw her motion. If she fails to do so, the court intends to strike her motion entirely.

Counsel for the claimant admitted to using an AI tool to draft the initial legal document without verifying its content. This resulted in the creation of non-existent judicial precedents to support the claimant's case. The court found this to be a serious violation of procedural loyalty and good faith, as it attempted to deceive the court and the opposing party. Consequently, the claimant was fined 5% of the updated value of the case for litigating in bad faith, although the lawyer's apologies were partially accepted, preventing a higher fine.

Fake citation, in this brief, was to : Moms Against Poverty v. Dep’t of State, 2022 WL 17951329, at *3 . Case docket can be found here.

Counsel later confirmed having used ChatGPT and apologised.

" After the trial court entered a final judgment and decree of divorce, Nimat Shahid (“Wife”) filed a petition to reopen the case and set aside the final judgment, arguing that service by publication was improper. The trial court denied the motion, using an order that relied upon non-existent case law."

"We are troubled by the citation of bogus cases in the trial court's order. As the reviewing court, we make no findings of fact as to how this impropriety occurred, observing only that the order purports to have been prepared by Husband's attorney, Diana Lynch. We further note that Lynch had cited the two fictitious cases that made it into the trial court's order in Husband's response to the petition to reopen, and she cited additional fake cases both in that Response and in the Appellee's Brief filed in this Court. "

"[92] In Mavundla, the court emphasised the trite duty of legal practitioners not to mislead the court, whether through negligence or intent. This includes the duty to present an honest account of the law, which means (inter alia) not presenting fictitious or non-existent cases.24 In my view, it matters not that such cases were not presented orally, but were contained in written heads of argument. Written heads are as important a memorial of counsel’s argument as oral argument and, for purely practical reasons, are often more heavily relied upon by judges.

[...]

[95] In this case, counsel’s explanations bear out their submission that there was no deliberate attempt to mislead the court in relation to the use of incorrect case citations in the heads of argument. Their apologies are acknowledged. As is clear from Mavundla, however, even negligence in this context may have grave repercussions particularly to the administration of justice and, in appropriate circumstances, could constitute serious professional misconduct.

[96] As a consequence, it is appropriate to make the same order as in Mavundla, namely that the conduct of theapplicant’s legal practitioners is referred to the Legal Practice Council for investigation."

In the case of Crespo v. Tesla, Inc., the pro se plaintiff, Leonardo Crespo, submitted discovery motions containing fabricated case citations and a false quote, which were identified as potentially generated by AI. The court ordered Crespo to show cause for these submissions and admitted to using AI in his filings. The court acknowledged Crespo's candor and imposed sanctions requiring him to apologize to the defendant's counsel and pay reasonable attorney's fees incurred by the defendant in addressing the fake citations.

(In a subsequent ruling, the court averred that the reasonable fees amount was 921 USD.)

"Generative AI can be beguiling, particularly when the task of representing yourself seems overwhelming. However, a litigant runs the risk that their case will be damaged, rather than helped, if they choose to use AI without taking the time to understand what it produces, and to confirm that it is both legally and factually accurate. "

Plaintiff Ravel Ferrera Parra, proceeding pro se, filed a lawsuit against the United States alleging financial harm due to misconduct by various judicial and governmental entities. The court dismissed the case for lack of jurisdiction, as the claims were not within the court's purview.

The court noted that Plaintiff's filings appeared to be assisted by AI, as evidenced by the rapid filing of responses tell-tale language ("Would you like additional affidavits, supporting exhibits, or further refinements before submission?"), the inclusion of fabricated case citations. "While Plaintiff’s use of AI, by itself, does not violate this Court’s Rules, Plaintiff’s citation to fake cases does."

The court further pointed out that:

"“It is no secret that generative AI programs are known to ‘hallucinate’ nonexistent cases.” Sanders, 176 Fed. Cl. at 169 (citation omitted). That appears to have happened here. When searching the Federal Claims Reporter for “Tucker v. United States, 71 Fed. Cl. 326 (2006),” Plaintiff’s citation brings the Court to the third page of Grapevine Imports, Ltd. v. United States, 71 Fed. Cl. 324, 326 (2006), a real tax case from this Court. Similarly, the AI used by Plaintiff in Sanders v. United States, 176 Fed. Cl. 163, 169 (2025) also made up a citation to a case called Tucker v. United States. Perhaps both AI programs hallucinated this case name based on the Tucker Act, this Court’s jurisdictional statute. Regardless, here, as in Sanders, the citation to a case called Tucker v. United States does not exist."

The court warned Plaintiff about the risks of using AI-generated content without verification but did not impose sanctions.

The court observed that the original motion to remand filed pro se by the plaintiff contained non-existent case law and falsified quotations. Although the court did not impose sanctions in this instance, it warned that future use of fake legal authority would result in a show cause order, including against the Counsel who later joined the case.