Anthropic Embeds METR Evaluators With Employee-Like Access in Three-Tier External Risk Review
Summary
- • Anthropic onboarded six METR staff with employee-like access to assess its AI risk practices
- • METR can publish findings; Anthropic cannot veto but may require redaction of narrow categories
- • SecureBio is reviewing the chem/bio section of Anthropic's August risk report
- • Anthropic funds Accenture directly, drawing independence criticism; METR is not funded by Anthropic
Details
Anthropic began giving a METR team employee-like access to assess its risk practices
Six METR employees were onboarded with hardware, office space and internal tools roughly on par with Anthropic's own risk assessment teams.
METR will publish analysis of Anthropic's risk levels, incidents, and practices
Anthropic cannot veto publication but can require redaction of narrow predefined categories such as customer data, privileged information and model architecture. METR can characterize redactions publicly.
Anthropic is not funding METR's work
This supports the independence criterion that evaluators have no equity or financial dependence on the company being evaluated.
SecureBio is reviewing the chem/bio weapons section of Anthropic's August risk report
This is a narrowly scoped, first-tier investigation; SecureBio could take on broader access in the future.
Anthropic funds Accenture directly as an evaluator, drawing independence criticism
Direct payment from the evaluated company raised concerns on social media; Anthropic says no funding system exists yet and long-term wants pooled or government funding.
Anthropic is in early talks with other evaluators, including Hugging Face
The program is intended to expand beyond METR, SecureBio and Accenture; access level for others is unclear.
Framework has three tiers: narrow investigations, publish-freely evaluators, employee-like access
Tier 1 is narrow risk investigations (SecureBio), tier 2 can publish without approval, tier 3 has employee-like access (METR). Anthropic likens it to bank supervision.
Move follows Anthropic signing the White House Accord on Super Intelligence in late September
The voluntary accord encouraged external auditors and was signed by executives at Google, Meta, OpenAI, SpaceX and Nvidia.
Policy = oversight framework, Financials = funding, Partnership = evaluator relationships, Strategy = program design, Context = background
What This Means
Anthropic is testing outside oversight in which evaluators sit inside the company and can publish findings, much like bank supervisors. Credibility depends on unsettled details such as redaction power, who pays, and real access. With no industry standards, it could become a template for other labs or regulators.
Sentiment
Mostly impressed with the embedded oversight model, some skepticism on long-term credibility
“Anthropic has placed 6 safety evaluators from the outside nonprofit METR inside the company, with access to internal systems, tools and staff roughly equal to that of its own risk teams... METR will publish its own verdicts... Anthropic cannot block them, only redact a few predefined categories.”
“Yesterday I asked whether any frontier lab would volunteer for an audit it cannot control. Anthropic just answered. METR gets employee level access and publishes its own assessments with no veto. That is the strongest version of outside scrutiny we have seen so far.”
“Six METR researchers now work inside Anthropic with company hardware, office space and internal access roughly on par with its own risk team. Anthropic can't veto what they publish, only redact narrow stuff like customer data. I'm waiting for the first report it won't enjoy.”
“Anthropic announces embedded safety evaluators from METR. They get the same access as Anthropic's own risk teams, and Anthropic is calling on every other frontier lab to do the same.”
Split
~70/30 positive/skeptical on whether the access model delivers credible independence vs. potential influence.