Last month, Anthropic committed to stronger independent, third-party assessment of its models and safety practices. Anthropic also signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators. Anthropic has now started giving employee-like access to a team from METR, a nonprofit that evaluates frontier AI models.
METR will publish the results of its risk assessments. Over the past year, it has worked with Anthropic on several reviews of Anthropic's Risk Reports. Anthropic is also working with SecureBio and Accenture as evaluators, and is in early discussions with several other organizations, including Hugging Face.
This piece describes Anthropic's approach to working with evaluators.
Why Anthropic works with independent evaluators
As models grow more powerful, they need stronger safeguards and more demanding risk assessments. Anthropic wants to be publicly accountable for its decisions, but it can't publicly share everything relevant to its safety work. Some details could help malicious actors, and others are commercially sensitive, such as compute or training information. So Anthropic is giving third-party evaluators deep access to its information. It is also asking them to share evidence and reasoned conclusions about the risks of building Anthropic's models. The aim is to narrow the gap between what evaluators understand about a frontier AI company's systems and what the company understands.
To evaluate frontier models, evaluators need the following:
- Expertise in assessing frontier model capabilities, including knowing how evaluations can go wrong, such as through alignment faking.
- Independence: they shouldn't hold equity in the company being evaluated, or depend on it financially or operationally.
- Enough access to evaluate risks meaningfully. Beyond the latest models, evaluators need visibility into how the models were trained, how Anthropic's safeguards work, and more.
- Freedom to publish their observations and assessments of a model or process.
The analogy isn't exact, but bank supervision works on similar logic. Banks know more about their own risk positions than outsiders do, so regulators place resident examiners inside the largest banks. If a bank's risk-taking could threaten the public, third parties try to see it in advance.
For the AI industry, this is a step toward robust safety oversight. Anthropic is moving quickly with multiple evaluators and learning what kinds of partnership work. It is committed to sharing updates along the way.
What "employee-like access" means
Anthropic has onboarded six METR employees with company hardware, office space, and access to internal tools and systems roughly on par with those of Anthropic's own risk assessment teams. Outside a small number of clearly defined categories (for example customer data, attorney-client privileged information, or information about model architectures), Anthropic employees are encouraged to share information with them as they would with a fellow employee working on alignment risk.[1]
METR will publish its analysis of Anthropic's risk levels, incidents, or practices. Anthropic can't veto publication, but it can require redaction of information in predefined categories, such as reproduced confidential IP. Anthropic ultimately decides what falls into those categories, though METR reserves the right to make certain statements about the level of access and information it received.[2]
Anthropic's engagement with METR is limited to a small team under enforceable agreements. Other METR employees, including researchers who work with Anthropic on other projects, are siloed from that team's work. Embedded evaluation gives a few qualified people, working under Anthropic's security standards, a close view of how Anthropic operates. It isn't a way for other organizations to obtain information from Anthropic, its customers, or its partners.
Anthropic's other evaluator partnerships
Anthropic is working on several other partnerships. Because they are at early stages, those evaluators get more limited information and have more limited publication rights. The goal is to confirm that partners protect confidential information and have the resources and expertise to evaluate Anthropic's models. As partnerships progress and trust builds, Anthropic hopes to embed more independent evaluators fully.
Anthropic is developing three tiers of evaluator access. Over time, it hopes to have several evaluators at the third tier, each publishing its own analysis of Anthropic's risk posture and key decisions.
In the first tier, evaluators conduct narrowly scoped risk investigations. Anthropic decides what access to provide, and evaluators can publish findings with Anthropic's approval.
In the second tier, evaluators can publish without Anthropic's approval and have access to more information. Anthropic can still require removal of specific categories of information, though evaluators can publish their views on whether the redactions were justified.
The third tier is the arrangement described with METR, with employee-like access and publication rights similar to those of the second tier.
SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report, after completing and publishing a pilot review of the previous risk report. Once this review is complete, Anthropic will consider moving to projects with increased access.[3]
With Accenture and Faculty, Anthropic is working toward an ambitious partnership that includes evaluating and red-teaming models, conducting alignment assessments, testing model safeguards, and building capabilities across the organizations.
Anthropic is embedding evaluators voluntarily, and believes every frontier AI company should do the same.
Footnotes
Because these evaluators (like bank examiners) may also work with other companies, Anthropic is cautious about inadvertent leaks. Information about architectures carries a high leak risk and is typically of low relevance to risk assessment. When it is highly relevant, Anthropic will generally seek to share it, though this requires explicit internal approvals.
Updated on October 16 to clarify METR's current project terms with Anthropic.
Updated on October 15 to clarify SecureBio's current project terms with Anthropic.