To be trustworthy, LLMs need to show their work (cen.acs.org)

A recent commentary highlights the importance of transparency in the use of large language models (LLMs) like AlphaFold for drug discovery. While AlphaFold and similar AI tools can quickly provide protein structures, this ease of access comes with significant drawbacks. The author, John Trant, emphasizes that understanding the process behind generated structures is crucial for valid scientific conclusions. Without a transparent “chain of custody” that connects input data to outputs, researchers cannot effectively challenge or validate the findings, leading to potential missteps in drug development. Trant argues that although AI and machine learning have revolutionized part of the drug discovery process, reliance on these models can pose risks. The lack of insight into the assumptions and decisions made during AI structure generation raises concerns about the accuracy of results. This commentary serves as a cautionary reminder for the AI/ML community, highlighting the necessity for both human expertise and rigorous validation methods in computational drug design to mitigate the unpredictable nature of AI-generated outputs.