Anthropic grants nonprofit METR employee-level access for AI risk monitoring

What happened

Anthropic announced on Thursday that it has granted the nonprofit organization METR “employee-like access” to its internal systems to assess the risks of its AI models [1, 3]. This arrangement involves six METR employees who have been given office space, company-issued laptops, and access to internal tools and hardware [1, 3]. Anthropic stated that the level of access provided to these researchers is roughly equal to what its own internal risk assessment teams use [1, 3].

Anthropic has encouraged its own staff to share information with the METR team as they would with any colleague working on alignment risk [3]. The company compared this model to bank supervision, where regulators place resident examiners inside large banks to monitor risk [1, 3].

Under this agreement, METR can publish its findings and analysis regarding Anthropic’s risks, incidents, and practices without needing approval from the company [1, 3]. However, Anthropic can require the redaction of specific information, including:

• Customer data [1, 3]
• Details covered by attorney-client privilege [1, 3]
• Information regarding model architectures [1, 3]

METR is permitted to publicly characterize these redactions and state whether it believes they were justified, provided it does not reveal the redacted content itself [3].

This move is part of a three-tier system Anthropic is building for evaluators [3]:

• The bottom tier includes organizations like SecureBio, which reviews chemical and biological weapon risks and requires Anthropic’s approval to publish [1, 3].
• The middle tier receives more information and can publish without approval [3].
• The top tier, which METR occupies, includes the employee-level access [3].

Anthropic is not paying METR for this work [1]. METR is funded by individuals, philanthropic organizations, and government bodies like the UK’s AI Security Institute, and it does not accept donations from AI companies or their staff [1].

Who is affected

• METR: Six employees from this nonprofit are now embedded in Anthropic’s office [1, 3].
• Anthropic: The company is implementing these safeguards following concerns about technology running out of control [1]. It is also working with Accenture, with both parties committing at least $1 billion to establish new safeguards [1].
• Investors: Anthropic is working to reassure investors ahead of an expected initial public offering this year, which could have a valuation of more than $2 trillion [1].