Why Normativity Remains Challenging for LLMs (link.springer.com)

Recent discussions around Large Language Models (LLMs) highlight philosophical challenges regarding their ability to model truths, particularly in normative domains where truths may not be systematic. It was asserted that while LLMs show potential for inferring truths from incomplete data, a fundamental assumption is that truths form a coherent and interconnected web, allowing for inference and correction of inaccuracies. However, philosophers argue that truths, especially those concerning values and normative issues, often lack this coherence, making it difficult for LLMs to leverage the systematicity of truth effectively. This is significant for the AI/ML community as it raises concerns about the reliability of LLMs in practical applications that demand ethical reasoning or value judgments. If the truth in normative domains is indeed asystematic, LLMs will struggle to achieve comprehensive understanding and may propagate falsehoods based on limited or inaccurate data, undermining their role in human deliberation. The findings call attention to the necessity of human agency in ethical considerations, emphasizing that reliance on LLMs for practical decision-making could be misplaced when addressing complex normative issues.