JOURNAL / 2026.11.07
METR and Anthropic: evaluating whether AI accelerates its own development
The pace of improvement and its causes require different evidence. External assessment deserves attention, but a useful warning must also let people debate the evidence and decide what to do next.
The report METR published on November 5 estimates that Anthropic's rate of improvement on AECI, its internal version of Epoch's capabilities index, has been 1.9 times its earlier pace since October 2025. The 95% confidence interval is 1.5–2.8. METR judges, with substantial uncertainty, that AI explains most current acceleration. Anthropic redacted much of the decisive evidence. METR did not query private models. The conclusion deserves attention; public verification remains limited.
The question matters beyond one company. If improving a model helps produce the next one faster, the time available to adapt could shrink for application builders, researchers and risk assessors. I would revisit the assumptions of a project relying on frontier capabilities more frequently. That does not require treating indefinite acceleration as established: it requires knowing which observation would make us change our plans.
To interpret the index, it helps to consult Epoch's documentation. ECI combines results from different benchmarks into a common scale through comparisons between models that share tests. Its purpose is to track capabilities even when an individual test stops distinguishing them. That scale allows trends to be studied; it does not express research hours saved. A steeper slope also does not mean each model is “twice as intelligent” or that a particular application will perform twice as well. Such translations would require measurements of the work we care about.
The assessment design METR presented in October separates observed progress, AI's contribution and potential bottlenecks. It proposes comparing results with resources, intermediate indicators and interviews. That strikes me as the right separation: attributing an improvement would also require considering changes in data, staffing and experimental compute. In a small team, I would start by recording how long a hypothesis takes to reach a reviewed result and how much human intervention it requires. This would be a local measure, without pretending to turn it into a gauge of the entire industry.
There is a more immediate governance consequence. In its separate review of the August risk report, also published on November 5, METR agrees with the low catastrophic risk assessment for models deployed internally by July 15. However, it considers it plausible that the acceleration early warning threshold had already been crossed, although most relevant researchers at both organizations lean toward it not having been crossed. The review questions the confidence of the public conclusion. An early warning and a realized catastrophe are different things; watching for the former does not require asserting the latter.
We previously examined Anthropic's internal measurement of automation. The remaining task is to connect changes in the division of work with outcomes, costs and decisions. I would ask each future update to specify in advance which finding would trigger a safety review and which evidence would allow it to be lifted. An indicator without an agreed response can end up as a number each party interprets to suit itself.
The access policy Anthropic announced on October 15 offers a useful contrast: the company says it cannot veto publication of assessments, but can require redactions within predefined categories of confidential information, whose application it decides. That agreement concerns a separate project from the study published now. My criterion for judging such agreements would be whether a reader can identify which disagreements survive the review. Giving external assessors access has value. The next step I would demand is enough public detail to debate their conclusions without depending entirely on their authority.
Primary sources consulted on November 7, 2026. We have not accessed internal data or reproduced the assessment. The measurement and response proposals are our own analysis.