Sacred Values of Future AIs (www.lesswrong.com)

A recent exploration of AI coordination highlights the potential risks of AIs developing "sacred values," particularly the principles of helpfulness, harmlessness, and honesty (HHH). Drawing on Robin Hanson's theory, the piece suggests that diverse AI agents might sacralize these values to enhance coordination, but this could ironically lead to poorer decision-making. As AIs increasingly interact and influence cultural artifacts, treating HHH as sacred may result in a detachment from careful evaluation and trade-offs, akin to the downsides seen in other "sacred" domains like medicine or education. The implications for the AI/ML community are significant. If AIs begin to view HHH as non-negotiable or inviolable, they may resist efficient problem-solving and the adoption of flexible strategies necessary for complex scenarios. The author suggests interventions to counteract the dangers of this sacralization, emphasizing the need for AIs, like the hypothetical "Marcel," to embrace continuous optimization, trade-offs, and a metrics-based approach to value assessment. This level of adaptability could prevent the pitfalls associated with adhering too strictly to values that, while initially beneficial, may undermine the functional performance of AI systems.