The uncritical adoption of AI in science is alarming – We need guard rails (www.nature.com)

The rapid integration of artificial intelligence, particularly large language models (LLMs), into the scientific community raises significant concerns about the quality and integrity of research. While AI tools have been marketed as a means to enhance productivity in academic writing, evidence suggests that reliance on these technologies might diminish the scientific merit of published work. Studies have shown that LLM-assisted papers often attract more citations and facilitate quicker advancement for researchers; however, they also produce outputs described as "AI slop," which include nonsensical references and images, leading to a saturation of the literature with superficially convincing but scientifically underwhelming research. Furthermore, the use of AI in scientific tasks threatens the development of crucial tacit knowledge among early-career researchers. As routine tasks become automated, the essential hands-on experiences that cultivate skills in scientific reasoning and oversight of AI-driven research may be lost, leaving future scientists ill-equipped to navigate a landscape increasingly dominated by AI tools. This situation prompts the scientific community to reconsider the dual objectives of scientific institutions: generating a repository of knowledge versus fostering a dynamic community of scientific thinkers. Without proper guardrails and a thoughtful approach to integrating AI, the risks to scientific innovation and training could outweigh the benefits of increased productivity.