Changing a patient social background changed LLM care recommendations (medwrite.ai)

The recent DubLINK LLM-athon, held by MIT Critical Data in Dublin, showcased how large language models (LLMs) adapt care recommendations based on contextual patient background. During the event, 49 participants, including clinicians and engineers, tested the MedWrite platform by analyzing a realistic patient case. They discovered that altering a patient's social background significantly affected the care recommendations generated by various AI models, highlighting the models' sensitivity to context. Notably, some models failed to recognize critical changes, such as a patient being from a homeless shelter or having a unique cultural background, which influenced their care suggestions. This event underscores the importance of integrating contextual awareness into AI systems used in healthcare, as the findings illustrate that patient care cannot solely rely on data inputs. The testing revealed that models could produce substantially different recommendations when given comprehensive context or when key details were altered. Dr. Leo Anthony Celi emphasized that preparing AI for medicine involves creating robust governance frameworks, and the discussions highlighted the necessity for unified patient records in Ireland. The insights from the LLM-athon will contribute to ongoing efforts in optimizing AI systems for healthcare, ensuring they are contextually aware to provide relevant and safe recommendations.