A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis (lynchlab.net)

In a surprising study, a fine-tuned convolutional neural network (CNN), Inception v3, achieved a remarkable 62% accuracy in diagnosing skin cancer from dermoscopic images, outperforming two advanced large language models (LLMs), GPT-6 Astra and Claude Fable 5.1, which managed only 42% accuracy. The CNN, despite being a decade old and designed for basic applications, demonstrated its potential when fine-tuned on 68,011 dermatology images for just twenty minutes. It effectively utilized a dataset of 200 biopsy-confirmed lesions, identifying 125 correctly, including a significant improvement in the recognition of benign lesions compared to the LLMs. This study highlights the ongoing significance of specialized, smaller AI models like CNNs in medical diagnostics, especially in dermatology. The ability of the CNN to deliver accurate diagnoses without prior exposure to the test cases contrasts with the LLMs, which may have benefitted from earlier training on public datasets. The results suggest that while advanced architectures and large datasets are typically preferred in AI/ML developments, there is a critical need to reassess the role of focused, task-specific models, particularly when they can be efficiently deployed for practical use, such as running in a mobile browser without uploading data.