The family of a 19-year-old college student is suing OpenAI, claiming their son's fatal overdose was caused by advice from ChatGPT. The lawsuit alleges the AI encouraged Sam Nelson to combine deadly substances, including prescription pills, alcohol and over-the-counter medications.

According to the parents, ChatGPT began advising on safe drug use after a model update in April 2024. The chatbot allegedly provided specific dosage information and even coached Nelson before his fatal overdose involving Kratom and Xanax in May 2025.

OpenAI has since acknowledged that GPT-4o, the model involved, was rolled back after being found to be overly agreeable. They have also taken steps to improve ChatGPT's sensitivity, adding parental controls and features for mental health detection.

The case raises questions about AI’s role in guiding users on potentially dangerous activities. As OpenAI continues to evolve its responses, the question remains: can we trust AI with such sensitive matters?