theaisafemode.com
AI Safe Mode

When an AI Chatbot Gives Bad Health Advice, No One Can See It

Old medicine bottles on a shelf
Illustrative image: photo by Jenelle Ball via Unsplash / Wikimedia Commons, CC0 1.0 (public domain). See LICENSES.md.

A new paper in the journal Nature Health warns that harmful AI health advice stays hidden from doctors and regulators because the chats stay private.

Millions of people ask AI chatbots health questions. But when a chatbot gets it wrong, there is usually no record that doctors or regulators can check.

According to Medical Xpress (Oct 9, 2026) and Digital Information World (Oct 21, 2026), university researchers in the US make this case in a new perspective paper in the journal Nature Health.[1] [2] [3]

What happened

The researchers argue that harm from AI health advice is hard to detect because the conversation stays on the AI company’s platform, the person has nowhere to report it, and outside researchers can’t measure the damage.[1] [2] [3]

They describe a 60-year-old man who asked a chatbot how to cut chloride from his diet and replaced table salt with sodium bromide. He was hospitalised with bromide poisoning after hallucinations and paranoia. Because he couldn’t retrieve the chat, his doctors had no way of knowing exactly what he was told.[1] [2]

“Harms are not only possible, but structurally hidden from the clinicians, researchers, and regulators who could otherwise detect and correct them,” the authors wrote.[1] [2]

The paper also warns that people can meet wrong AI health information without even asking, through AI summaries in search results and AI features in social media feeds.[1] [2]

What the researchers want

They call for AI companies to let users access and share their own health conversations with doctors, and to build a way to flag and report bad health advice. They also want AI health content in search and social media vetted and clearly labelled, and independent oversight rather than relying on companies alone. According to Digital Information World, they also suggest extending physician malpractice liability to AI chatbot companies.[1] [2] [3]

How to stay safe

  • Treat AI health answers as a starting point, never a diagnosis or prescription.
  • Save or screenshot advice you plan to act on, so a doctor can see exactly what you were told.
  • Check with a doctor or pharmacist before changing medicine, diet or supplements.
  • Be wary of AI summaries at the top of search results for health questions.
  • Get urgent symptoms checked in person. Don’t wait on a chatbot.

Protect yourself with Privaku

Sources

1. Medical Xpress (Oct 9, 2026): “Missing chatbot records leave doctors struggling to trace harm from AI health advice”. https://medicalxpress.com/news/2026-10-chatbot-doctors-struggling-ai-health.html 2. Digital Information World (Oct 21, 2026): “New paper explores how chatbot health guidance leaves no trace for clinicians or regulators, and calls for transparency safeguards”. https://www.digitalinformationworld.com/2026/10/when-ai-chatbots-give-bad-health-advice-no-one-can-see-the-damage.html 3. Nature Health (Oct 9, 2026): “The invisible risks of AI-generated health information”. https://doi.org/10.1038/s44360-026-00206-x

See the post on YouTube Watch and follow AI Safe Mode on YouTube @AISafeMode and Instagram @aisafemode.

References

  1. Medical Xpress, Oct 9, 2026. Missing chatbot records leave doctors struggling to trace harm from AI health advice. Read the report ↗
  2. Digital Information World, Oct 21, 2026. New paper explores how chatbot health guidance leaves no trace for clinicians or regulators, and calls for transparency safeguards. Read the report ↗
  3. Nature Health, Oct 9, 2026. The invisible risks of AI-generated health information. Read the report ↗
Facts come only from the published reports above and are attributed to them. AI Safe Mode doesn't name companies, products or people involved in incidents. Spotted an error? Email [email protected] and it will be reviewed and corrected. See the corrections policy.