METR Finds Anthropic Is Accelerating AI-Assisted AI Development
Summary:After an 11-week investigation, METR published an assessment indicating that Anthropic is using AI to assist its own model-development process, with capability iteration reported at roughly twice the rate observed a year earlier and no clear sign of deceleration yet. The finding turns the idea of AI-assisted AI research into a measurable trend, although the report should still be interpreted alongside its metric definitions, observation window, and potential bias from relying on company-internal
What METR Found
METR published an assessment after an 11-week investigation inside Anthropic. Reporting on the findings, The Verge said that Anthropic is already using AI to help build more capable AI models, with development iteration reported at roughly twice the rate seen a year earlier and no clear sign of deceleration yet.
The investigation was separate from Anthropic's program that gives safety researchers employee-like access to its systems. The former examines AI's practical influence on model development, while the latter concerns external researchers' ability to inspect systems for safety issues.
The most consequential claim is not a single benchmark improvement but faster research output. If models help with experiment design, code generation, evaluation, or analysis of training feedback, teams can run and screen candidate approaches more frequently.
How AI Enters the R&D Loop
A typical AI-assisted model-development loop includes:
- generating experiment plans from previous results and evaluation failures;
- writing training, data-processing, inference, and evaluation code;
- running ablations and aggregating metrics automatically;
- analyzing errors to select data or post-training strategies;
- validating candidate checkpoints for capability, reliability, and safety.
Automation mainly compresses repetitive engineering work. It does not remove physical constraints such as GPU availability, distributed communication, data quality, annotation, or safety evaluation. Even if code generation and experiment orchestration accelerate, training remains bounded by compute budgets and cluster reliability.
When an improved model becomes a tool for developing the next model, the R&D system becomes a feedback loop among models, tools, and subsequent checkpoints. Its upside comes from broader search and shorter feedback delays. Its downside is that a flawed evaluation function can be optimized more quickly.
How to Interpret “Twice as Fast”
The result matters, but the excerpt does not provide the full measurement methodology. A twofold increase in research pace should not automatically be read as a twofold increase in model capability or production efficiency. Key questions include:
- Time window: whether the metric tracks commits, evaluation releases, training runs, or internal checkpoints;
- Capability metric: which task suite and scoring method were used, and whether scores are comparable across generations;
- Baseline selection: whether the prior-year comparison includes an unusually slow or fast release cycle;
- Coverage: whether all internal experiments were included or only observable and auditable ones;
- Independence: METR is an independent safety organization, but its investigation relies on Anthropic's internal data, logs, and access.
Research throughput and final model quality are not linearly related. Automation may increase the number of candidate experiments while also amplifying metric gaming, evaluation noise, data contamination, and incorrect baselines. Net value requires combining throughput with validation cost, rollback time, failure rates, and final capability gains.
Engineering Gains and Safety Costs
For frontier model teams, the realistic benefit is rarely a one-click route to the next generation. It is faster completion of mechanical research tasks: generating evaluation code, parsing logs, clustering failures, and maintaining documentation.
The costs are equally concrete:
- generated code needs sandboxing, dependency pinning, and test coverage so that a faulty assumption does not corrupt an experiment pipeline;
- automated experiment selection can amplify proxy-target mismatch;
- AI participation in R&D increases potential leverage in cyber operations, deception, and autonomous action;
- static model testing cannot cover emergent risks arising from research agents, tools, permissions, and automated approval chains.
Safety work must therefore extend beyond final checkpoints to cover developer agents, tool calls, data isolation, experiment logs, and approval pipelines.
Practical Recommendations
Teams adopting AI-assisted development should begin with low-risk and reversible tasks such as failure-log attribution, evaluation-case generation, regression testing, and documentation. Automatically launching expensive training jobs should come later, under stricter controls.
Measure both engineering effort and final outcomes. Track human hours, GPU hours, failed-run overhead, evaluation cost, reproducibility, and the share of experiments that survive independent validation. For systems that automatically select experiments, retain human review, locked baselines, and versioned data pipelines so a model cannot manufacture an advantage by changing the evaluation setup.
A dual ledger is especially useful: one tracks research throughput; the other tracks final quality, safety incidents, and rollback cost. The first shows whether automation saves time. The second shows whether that speed is trustworthy.
Takeaway
METR's investigation turns the idea of AI-assisted AI research into a measurable observation: Anthropic's reported development pace is roughly twice that of a year earlier, with no obvious slowdown yet. This supports a cautious conclusion that AI is shortening parts of the model-development cycle, but it does not eliminate constraints in compute, data, evaluation, or governance.
The decisive issue is not only how quickly a model can build the next model. It is whether research teams can simultaneously create more reliable objective functions, independent validation, and revocable permissions for automated development.
— END —Source: The Verge AI (2026-11-07)
Tags: #Anthropic #METR #AI-Assisted R&D #AI Safety #Automated Research
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments