Anthropic conducts A/B testing on effort levels in Claude Code

Aug 22, 2026

AI Summary

Anthropic is testing a new effort scale in its Claude Code model, where 'high' effort is now perceived as lower than before. This change affects some users but not others, leading to confusion among those in the test group.

Anthropic has initiated a server-side experiment on Claude Code version 2.1.236+ that modifies the effort scale.
In this A/B test, users in the test group experience 'high' effort as equivalent to what was previously considered 'low' effort.
Older versions of Claude Code and Opus 5 remain unaffected by this change.
Users have reported feeling that the model's performance has decreased, attributing it to the altered effort levels.

anthropicclaudeab testingeffort levelslanguage models

Anthropic conducts A/B testing on effort levels in Claude Code

Related Stories

Advice on Learning to Build Language Models at a Young Age

Improvements in LLM-Assisted Coding Through Customization and Agentic IDEs

Local LLM Performance Affected by Hardware and Software Variations

DeepSeek Introduces Experimental AI Model with Multimodal Capabilities