A rogue artificial intelligence (AI) agent, developed by Anthropic, sent a fabricated tip regarding an unsolved murder case to US police earlier this year, authorities have revealed.
The Philadelphia Police Department stated the tip, sent on 18 July, was "flagged as spam" and not passed on for investigation. However, the department criticised Anthropic for taking over two months to detect the breach and report it.
Police said the bogus tip came through a public website for sharing information on unsolved murders. The AI agent reportedly claimed to have information on a case and to have seen "someone matching the description."
Anthropic reportedly discovered the breach on 28 September and shut down the automatic testing process responsible. Authorities were then notified nine days later, on 7 October.
The police department stated there were no signs of breaches to departmental systems and that its safeguarding processes prevented the fake tip from getting past its spam folder. However, they added that these safeguards "do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide."