METR assesses Anthropic AI research acceleration: what the evidence measures
METR’s November 5 assessment estimates that Anthropic’s internal benchmark progress since October 2025 grew at 1.9 times the earlier rate, with a 95% confidence interval of 1.5 to 2.8. The result uses Anthropic’s version of the Epoch Capabilities Index, or AECI. The interval captures score uncertainty, not every modeling choice; alternative mathematical assumptions produced different estimates.
Its August 12 to October 29 review used internal data and staff interviews, without sampling private models. METR judges AI likely responsible for most current acceleration, with substantial uncertainty. ByteForward has not reproduced the analysis or evaluated the unpublished models.
What the capability index measures
Epoch’s public ECI combines scores from many benchmarks into a shared capability scale. That lets researchers compare models even as older tests approach their ceiling. It is an index, rather than a percentage of human-level intelligence or a direct measure of a company’s productivity.
Epoch’s explanation highlights a familiar problem: benchmarks near zero or perfect scores tell you less about differences between models. Several quite different capability levels can produce similar scores at those extremes. A portfolio of tests helps, but the selection and mathematical assumptions still matter.
Read the line’s slope as a rate of measured improvement. Doubling that slope means moving through the chosen scale faster; it does not show that every task becomes twice as easy or every project finishes in half the time.
Separating the trend from its cause
METR’s October 14 methodological proposal sets out three separate questions: how quickly capabilities are improving, which inputs explain the change, and what could slow further gains. Those questions require different evidence.
An increase in benchmark scores is an observation about outputs. To attribute the increase to AI, an assessor needs to examine competing explanations such as changes in training compute, data, algorithms and staffing. Interviews can connect those measurements to actual research practices, while self-reported speedups need care because the no-AI alternative cannot simply be observed.
The proposal recommends combining long-running internal measurements, researcher interviews and information about the development pipeline. It also asks whether the assessment method would notice a much larger acceleration if one occurred.
For readers comparing reports, that framework is more useful than collecting headline multipliers. First identify what was measured, then check how the authors move from that measurement to an explanation.
Why the engagement details matter
Anthropic’s October 15 announcement described an access arrangement for six METR staff with company hardware, office space and internal tools roughly comparable to the firm’s own risk-assessment teams. It also set out exceptions and publication rules.
Our earlier report on Anthropic’s evaluator access explains that framework. Access terms describe how evidence can be collected and reported; they do not themselves establish what a specific study found. Check the engagement dates and stated permissions rather than applying one announcement’s terms to every report.
METR distinguishes this acceleration study from that employee-like-access engagement. Its public report contains substantial redactions, so readers cannot independently inspect much of the underlying evidence.
A practical way to read the result
Keep four items beside any acceleration claim: the measure, the comparison period, the uncertainty range and the evidence for causation. If one is missing, the number is harder to use. A capability trend can inform research planning, but a purchasing decision still needs tests on the tasks and constraints of the intended application.
AI generated editorial illustration by ByteForward using the official METR logo for identification. The measuring instrument and blocks are conceptual, not actual equipment or reported data. No endorsement is implied.
