The Crown Prosecution Service's (CPS) reliance on generative AI for legal research has come under scrutiny after it emerged that two non-existent cases were cited in an extradition appeal. The incident raises concerns about the use of artificial intelligence tools without adequate human verification, highlighting a critical need for safeguards to prevent such errors.
According to the CPS, the fabricated cases arose from its use of AI for research purposes. While the agency attributed the 'operative cause' to human error – specifically the reviewing lawyer's failure to adequately verify citations before submission to court – Mr Justice Sweeting noted that this was an isolated incident and not a deliberate attempt to mislead the court.
The CPS apologised twice, initially stating that the errors were identified by the appellants' counsel. However, it later corrected this statement, admitting that its own junior counsel had independently spotted the fake cases before the appellants did. This prompted a 'further apology', which Mr Justice Sweeting accepted along with assurances that there was no intention to mislead.
The non-existent cases did not affect the outcome of the extradition appeal, as they were discovered and corrected prior to the hearing. Nevertheless, the judge deemed the incident significant enough to include in the public record of the judgment, cautioning against relying on AI for legal research without rigorous human oversight.
This is not an isolated incident within the legal sector. British courts are increasingly grappling with the challenge of AI-generated 'hallucinations' in evidence. In May this year, City law firm Pinsent Masons was criticised by a High Court judge after a junior lawyer submitted misleading emails drafted with AI in an insolvency case. Last year saw the review of two other cases where lawyers used AI-generated citations, some of which were entirely fictitious. The US firm Sullivan & Cromwell also apologised in April for multiple AI hallucinations in a bankruptcy case, underscoring a global challenge for the justice system.