Cognitive Biases and A.I. – shows worse biases than human practitioners (ai.nejm.org)

🤖 AI Summary
A recent NEJM commentary and analysis highlights that contemporary AI diagnostic and decision-support systems not only inherit human cognitive biases but can amplify them, often performing worse than practicing clinicians on the same reasoning tasks. Using case vignettes and simulated clinical scenarios, researchers found systematic tendencies such as anchoring (fixating on an initial diagnosis), availability and confirmation biases (overweighting familiar or early evidence), and automation bias (over-relying on model outputs) that led to higher error rates, poorer calibration, and larger subgroup disparities than those exhibited by human practitioners. The paper argues these are not just isolated failures but predictable consequences of training data artifacts, objective functions that optimize average performance rather than robust reasoning, and opaque model behavior. For the AI/ML community this is a wake-up call: model accuracy alone is insufficient for deployment in high-stakes domains. Technical implications include the need for standardized bias-challenge benchmarks (vignettes that probe reasoning errors), richer evaluation metrics (calibration, decision-impact, subgroup fairness), and mitigation approaches such as targeted fine-tuning, adversarial or counterfactual training, uncertainty-aware outputs, human-in-the-loop designs, and post-deployment monitoring. The authors recommend transparency about model limitations, routine cognitive-bias audits, and regulatory guidance to ensure AI augments rather than degrades expert decision-making.
Loading comments...
loading comments...