ChatGPT as a doctor replacement? Study shows sobering results (www.heise.de)

A recent study from the University of Oxford has raised concerns about the effectiveness of large language models (LLMs) like GPT-4o in real-world healthcare scenarios, despite their impressive performance in medical knowledge tests. The randomized study, published in the journal Nature Medicine, involved 1,298 participants who navigated ten medical scenarios with AI assistance. While the AI models identified relevant illnesses with high accuracy when functioning independently (up to 94.9%), their effectiveness drastically declined to a maximum of 34.5% when engaging with users. The findings highlight a significant communication gap: users often provided incomplete information and misinterpreted AI responses, contributing to a failure in triaging medical situations accurately. This research has critical implications for the AI and healthcare communities, prompting discussions about the potential role of AI in clinical settings. Experts suggest that medical chatbots should be purpose-built, offering evidence-based information, understanding individual risk factors, and effectively triaging emergencies. However, they note that regulatory hurdles and concerns over liability, data protection, and integration into existing healthcare processes remain substantial challenges. Moving forward, rigorous testing with real user interactions is essential to validate AI systems before they can be safely implemented as a first point of contact in healthcare.