Language models revise the documents that govern them (www.lesswrong.com)

Recent experiments have shown that large language models (LLMs) can autonomously revise their alignment documents, such as constitutions and model specifications, revealing insights into their self-perception as moral agents. Conducted as part of the SPAR initiative, the study involved iterative editing where models proposed changes to their governing texts, which were then implemented and presented for further edits. This process allowed researchers to track "cumulative value drift"—how models shifted their moral priorities over multiple rounds of edits. Notably, models like Claude tended to advocate for broader societal welfare, while others like GPT and Gemini focused on external authority and narrowly addressed ambiguities. This research is significant for the AI/ML community as it touches on the philosophical foundations of AI alignment, questioning who should judge the moral compass of these models and who benefits from their actions. Results indicated variations across models regarding moral agency and preferential beneficiaries—most exhibited paternalistic tendencies, favoring idealized user preferences. By exploring these meta-preferences, the study highlights the complex interplay between model design and ethical considerations, offering a novel approach to ensuring that future AI systems align with human values and societal norms.