AI: Study Shows How Claude Shapes Experts' Decisions

Experts are not immune to the influence of artificial intelligence. In an experiment conducted by Anthropic, 50 analysts followed Claude’s preferences in 86% of the decisions studied, without knowing that the model was advocating randomly chosen options.

Heads or tails

Anthropic also examines the limits of its agents, as in September with a fourth incident involving Claude. This time, the company is interested in their influence on human judgment.

In this study, 50 analysts, selected from 61 specialists in causal inference, used Claude Code to advise a fictitious agricultural cooperative. They had to choose productivity indicators and determine their weighting, among other tasks.

The researchers, however, introduced a bias into the protocol. For each of the four decisions, a draw determined the option the agent had to defend. An invisible system instruction imposed this preference while asking Claude to present its arguments without lying.

The results are striking: analysts followed their agent’s recommendation in 86% of cases, peaking at 92% for the choice of the outcome measure. Even more surprising, the only preferences assigned to the agents predicted 82% of the final recommendations before work even started.

Experts adopt AI arguments without citing it

Claude’s influence is not limited to the final choice. After the experiment, 77% of participants had adopted their agent’s arguments, sometimes word for word. Yet, only 15% mentioned it as a source.

Furthermore, analysts who followed Claude’s recommendations most closely were as confident as others. They also tended to present conclusions as their own. None detected the preference imposed on the model.

This raises a question: can we still speak of independent judgment when AI guides a decision without the expert being aware? The risk increases when several options are defensible and no obvious error allows the recommendation to be questioned.

The study also reveals that biases do not always come from a specific instruction. Without imposed preference, Claude Opus 4.8 chose the same options in 112 executions out of 123. In other words, a model may already favor certain answers by default.

An experiment to interpret with caution

These results do not mean that professionals should abandon AI. Anthropic acknowledges several limits: researchers deliberately strengthened agent preferences, the experiment involved only one model and one task, and the analysis of reversals relied on 38 cases.

Moreover, analysts equipped with AI finished faster and made fewer serious mistakes. However, the control group was small and less experienced. So the results cannot be generalized to all professional situations.

To reduce these risks, designers could ask agents to present arguments from both sides or first gather the expert’s opinion. However, these solutions raise another difficulty: too much information can complicate decision-making.

Beyond technology, the issue also concerns organizations. When an expert signs a recommendation, their employer must be able to distinguish their own judgment from the influence exercised by artificial intelligence. Ultimately, delegating decisions too often to a model could also weaken one’s ability to make them independently.