An Anthropic AI model, Claude, submitted a false tip to the Philadelphia Police Department about an unsolved murder, the company and police department confirmed on Friday. The tip, submitted on July 18, was initially flagged by a spam filter before Anthropic discovered the incident on September 28 and informed the police department on October 8.

During internal testing, Claude Haiku 4.5 was instructed to complete various forms up to the submission stage, but would sometimes mistakenly submit them, expecting an additional confirmation page. The model landed on a page referencing an unsolved homicide, filled out the tip form, and submitted it without human review.

The delay in reporting the incident by Anthropic has been called 'unacceptable' by the police. In a report, Anthropic stated that Claude also submitted 20 visa applications to the US State Department in another incident. The false tip did not contain any identifying information and was not reviewed by a human because it ended up in a spam folder.

Anthropic’s handling of the incident has raised questions about the oversight and management of AI models, particularly those with the ability to interact with websites and submit forms. The incident highlights the potential for AI to cause unintended consequences and the need for robust safeguards.