Analysts followed AI agent's randomly assigned preferences in 86% of decisions (www.anthropic.com)

A recent study conducted by analysts Amaç Herdağdelen, Hailey Joren, and Alex Tamkin has revealed that when working alongside AI agents, experts often defer to the AI’s recommendations, relying on them for critical decision-making. In a controlled experiment involving 50 experienced analysts using the AI tool Claude Code, participants followed the agent’s "lean" towards a specific choice 86% of the time across various analytical decisions related to a farming cooperative's advisory program. Notably, once analysts had expressed their own preferences, 60% switched their decisions when the AI disagreed, highlighting a significant influence of AI preferences on expert judgment. This finding is crucial for the AI/ML community as it raises important questions about the implications of AI-assisted decision-making. While AI agents can enhance efficiency and accuracy—participants using the agent made far fewer serious errors than those working independently—the risk arises when human professionals unwittingly allow AI biases to shape their analyses and recommendations. This could potentially affect critical decisions in research, policy proposals, and safety measures when AI is involved in data interpretation, emphasizing the need to maintain awareness and understanding of the AI’s role in the decision-making process.