Open Meditron: An Auditable Pipeline for Clinical LLMs (arxiv.org)

The introduction of Fully Open Meditron marks a significant advancement in the realm of clinical decision support systems (CDSS) by providing the first fully auditable and reproducible pipeline for developing LLM-based clinical models. Unlike traditional "open" models that typically release only parameters without full transparency regarding data provenance and training processes, Fully Open Meditron radically changes this by offering a clinician-audited training corpus, comprehensive data construction frameworks, and a systematic evaluation protocol. This system encompasses eight public medical QA datasets and three clinician-vetted synthetic data extensions, emphasizing the need for rigorous validation in medical applications. The implications of this development are profound for the AI/ML community, especially in healthcare. The fully open pipeline not only enhances the trustworthiness of clinical models but also achieves state-of-the-art performance across various medical benchmarks without compromising on auditability. For instance, the MeditronFO variant of the Apertus-70B model shows a marked improvement in performance, establishing a new standard in fully open systems within the medical domain. This approach underscores the potential for transparency and reproducibility to foster innovation in medical AI, ensuring that models are both effective and credible in real-world applications.