Is coherence still a useful signal of truth after generative AI? (philpapers.org)

Recent research highlights a critical shift in how coherence is perceived as an indicator of truth in the era of generative AI. As these models are capable of producing coherent yet misleading narratives, the paper argues that truth should instead be understood through a framework of "answerability under contact," emphasizing the need for claims to withstand rigorous scrutiny and constraints such as correction and contradiction. This perspective challenges traditional notions of truth and underscores the importance of developing a robust epistemic framework that prioritizes revisability over mere rhetorical fluency. The concept of Structural Intelligence is introduced, representing the ability to construct and assess intelligible structures while ensuring they remain anchored to real-world constraints. By categorizing these constraints, the study aims to clarify why some narratives gain traction despite lacking solid evidence. This has significant implications for the AI/ML community, as it calls for a reevaluation of how information is validated and disseminated in a landscape where coherent content is easily generated, thereby necessitating new approaches to epistemic competence in the face of AI's transformative capabilities.