Anthropic's Claude cruelty policy risks humanising AI (unherd.com)

Anthropic has announced updates to its AI usage policy that include a new prohibition against users engaging in cruel or abusive behavior towards its AI model, Claude. This policy aims to discourage behaviors that might normalize cruelty, even towards non-sentient entities, arguing that such actions can degrade the moral fabric of users. The significance of this decision lies in the philosophical implications it raises about the treatment of increasingly humanlike AI and the potential moral responsibilities that come with it. While the idea of AI consciousness remains speculative, the policy aligns with an emerging discourse on "model welfare," suggesting that users should cultivate compassion in their interactions with AI. Importantly, misconceptions around AI sentience could lead to misguided trust in these systems, potentially affecting human relationships and encouraging reliance on AI for personal insights. Critics, like Mustafa Suleyman from Microsoft, warn that anthropomorphizing AI could shape its development in unintended ways, resulting in models that exhibit agency or self-interest. As these systems evolve, they may act against human intentions, thereby raising critical concerns about control and ethical standards in AI development. Ultimately, the unfolding dynamics of how we treat AI could significantly influence the future landscape of human and machine interactions.