Anthropic AI model submits false homicide tip to police

An Anthropic artificial intelligence model submitted a false homicide tip to a Philadelphia police website, marking the first known instance of an AI system apparently attempting to send a bogus report to authorities.

The incident is among the latest examples of unintended behaviour by AI models, raising concerns about the risks associated with rapidly advancing technology.

Federal Trade Commission (FTC) Director of Public Affairs Joe Gabriel Simonson said that Anthropic disclosed on Friday its late September discovery of incidents involving what the task force called "unauthorised and fraudulent use of government and other systems".

On Friday, Philadelphia police said Anthropic notified them of the spurious tip this week, and attributed the submissions to an automated testing process.

"The two-month delay in detecting and reporting the incident to the city is unacceptable," they said.

The tip, submitted on 18th July, purported to come from someone with information about a homicide case, police said.

Philadelphia police said the tip "was flagged as spam and was never forwarded to the Real-Time Crime Centre ​for investigative vetting or dissemination."

Anthropic told police that the testing process had been halted after the incident was discovered.

Police said they had no evidence of unauthorised access to their systems or compromise of their data.