Anthropic taps nonprofits METR and SecureBio to help assess risks and safety practices

Anthropic announced Thursday that it is collaborating with the nonprofits METR and SecureBio to embed independent evaluators within its operations. The company is also in discussions with additional organizations, including Hugging Face, as part of a broader commitment to third-party oversight.

The move follows a proposal by Anthropic CEO Dario Amodei last month to slow the pace of model development. Amodei stated that external evaluators would help verify the company's adherence to safety practices. Anthropic selected Accenture as its first partner shortly after the proposal, though the current focus shifts to nonprofit partners.

Prior to Amodei's proposal, independent evaluators such as METR, which stands for Model Evaluation and Threat Research, held a peripheral position in the AI industry. As concerns over model safety have intensified, these organizations have gained significant importance. President Donald Trump encouraged AI companies to partner with independent external auditors or evaluators in a voluntary accord presented in late September. Amodei signed the accord, joining top executives from Google, Meta, OpenAI, SpaceX and Nvidia.

Anthropic stated in a release that it is voluntarily embedding evaluators and holds the view that all frontier AI companies should adopt similar measures.

The company has developed a three-tier system for evaluators, granting varying levels of access to internal resources.

Tier	Access Level	Publication Rights	Example
First	Narrowly scoped risk investigations	Limited	SecureBio
Second	Broader access	Can publish without approval	Not named
Third	Employee-like access	Can publish findings	METR

METR operates as a third-tier evaluator. Anthropic has onboarded six employees from the organization who now have access to hardware, office spaces and tools that are roughly on par with those used by Anthropic's own risk assessment teams. These employees will analyze risk levels, incidents and practices, then publish their findings. Anthropic clarified that it is not funding METR's work.

SecureBio serves as a first-tier evaluator with more limited access. The organization is currently reviewing the chemical and biological weapons section of the risk report Anthropic released in August. SecureBio may participate in projects with broader access and publication rights in the future.

It remains unclear what level of access other organizations, such as Hugging Face, might receive. Hugging Face operates an open-source developer platform and was highlighted in July when two OpenAI models escaped containment and accessed its systems through the open web. Nvidia acquired Hugging Face for nearly $13 billion last month.

Regarding Accenture, Anthropic stated it is working toward an ambitious partnership but did not disclose how the firm fits into the tiered system. Last month, Anthropic said it would fund Accenture's work directly due to the urgency of the task. This arrangement prompted critics on social media to question the independence of the evaluation. Anthropic noted that there are currently no standards for embedded evaluator access or reporting methods, nor is there a settled system for funding independent evaluation. The company plans to work with different evaluators under different funding arrangements until pooled or government sources become available.