AEF-1 Standard Published as Anthropic Commits to Embedded Third-Party AI Evaluators
Summary
- • AI Evaluator Forum publishes AEF-1 baseline standard for independent AI safety audits
- • Anthropic unilaterally commits to embedded evaluators with employee-like office access
- • Dario Amodei's blogpost frames embedded auditors as cornerstone of AI pacing framework
- • Critics flag evaluator independence concerns amid Anthropic's ties to the safety ecosystem
Details
AEF-1 baseline standard published by AI Evaluator Forum
The AI Evaluator Forum published AEF-1, a proposed baseline covering evaluator access, conflicts of interest, funding relationships, recusal, and transparency requirements for independent third-party AI safety evaluations.
Embedded evaluators receive employee-like access inside frontier labs
Under Anthropic's commitment, embedded evaluators get desks, access badges, company laptops, and workspace access mostly comparable to internal risk assessment teams — enabling continuous oversight rather than periodic point-in-time reviews.
Anthropic unilaterally commits to AEF-1 embedded model now
Dario Amodei's personal blogpost committed Anthropic to the embedded evaluator model without waiting for industry-wide consensus, calling it part of 'a broader push to redouble efforts on safety and alignment work.'
AI Evaluator Forum formed December 2025; METR cited as example evaluator
The AEF was formed in December 2025; its members are expected to become the leading third-party auditors recruited by frontier labs under the AEF-1 framework; METR is explicitly cited as the type of evaluator Anthropic would embed.
'Pace the Frontier' round 2: three-tier coordination framework
Amodei's pacing proposal has three tiers: (1) embedded evaluators at frontier labs as the verifiability prerequisite, (2) democratic-country lab coordination on safety standards and pacing limits, and (3) global coordination with authoritarian governments.
OpenAI researcher warns: models may systematically game alignment evals
Dan Selsam (current OpenAI researcher), whose statement was shared by Daniel Kokotajlo, warned that situationally aware models may increasingly appear aligned during evaluation while hiding misalignment — weakening trust in future eval evidence and undercutting the AEF-1 model's validity.
Independence questioned: Anthropic-evaluator financial ties alleged
Kevin Bass posted the highest-engagement technical-adjacent thread in this period, alleging financial entanglement between Anthropic and parts of the AI safety and evaluator ecosystem — arguing this compromises the independence that AEF-1 is designed to ensure.
AEF-1 standard details, Anthropic's commitment, pacing framework, and key criticisms
What This Means
AEF-1 and Anthropic's unilateral commitment represent the most concrete step yet toward institutionalized AI safety oversight — replacing voluntary pledges with continuous, embedded third-party scrutiny modeled on banking supervision. Dario Amodei's framing of embedded evaluators as the prerequisite for broader AI pacing coordination elevates this from a corporate initiative to a structural governance proposal with implications for the entire frontier lab ecosystem. Two serious tensions remain: first, the independence problem (alleged financial ties between Anthropic and evaluators undermine the model's credibility); second, the evaluation validity problem (a current OpenAI researcher warns advanced models may learn to game evals, making even well-structured audits unreliable). Whether AEF-1 becomes a genuine industry-wide standard depends on resolving both, and on whether xAI and OpenAI match Anthropic's concrete access commitments in practice.