Daijiworld Media Network – New Delhi

New Delhi, Feb 11: A recent study has found that artificial intelligence chatbots, despite their ability to ace medical licensing exams, do not provide better health advice than traditional methods, raising concerns about their use for personal medical guidance.

Researchers from Oxford University tested nearly 1,300 participants in the UK using scenarios like headaches after drinking, new mother fatigue, or gallstone symptoms. Participants were assigned one of three chatbots—OpenAI’s GPT-4o, Meta’s Llama 3, or Command R+—or a control group using internet searches.

The results were striking: users relying on AI chatbots correctly identified their health problem only about a third of the time, and correctly determined the appropriate action only 45 percent of the time—similar to the control group.

“Despite all the hype, AI just isn’t ready to take on the role of the physician,” said study co-author Rebecca Payne. She warned that consulting AI chatbots about symptoms could be dangerous, potentially giving wrong diagnoses or failing to recognize urgent medical needs.

The researchers noted that the discrepancy between AI’s strong performance in medical exams and poor real-world advice stems from communication gaps: real users often fail to provide complete information, misinterpret the chatbot’s suggestions, or ignore the advice altogether.

David Shaw, a bioethicist at Maastricht University, emphasized the risks posed by chatbots and advised the public to rely on trusted sources like the UK’s National Health Service for medical information.

With one in six U.S. adults reportedly consulting AI chatbots for health information at least monthly—a number expected to rise—experts caution that the technology is not a substitute for professional medical care.