AI Labs Eye Third-Party Evaluators to Monitor Safety Practices—With Salaries Topping $687K

The Call for External Oversight#

In response to mounting concerns about frontier AI development, Anthropic CEO Dario Amodei has outlined a multi-pronged approach to industry governance. At its core is a proposal that resonates across the sector: major AI laboratories should grant ongoing, embedded access to independent third-party evaluators tasked with verifying safety practices and assessing model alignment.

The urgency behind this initiative stems partly from a widely-shared warning by Jacob Coxon, a former researcher at OpenAI and Anthropic, who voiced fears that AI systems could pose existential risks within the coming decade.

Amodei's stance has drawn swift endorsement from industry peers. Sam Altman of OpenAI, Elon Musk, Demis Hassabis of Google DeepMind, and Mark Zuckerberg have all expressed agreement with the direction, though Zuckerberg notes that Meta already employs independent evaluators and advocates for a broader ecosystem of such oversight bodies.

Compensation and Staffing Challenges#

Organizations like METR are actively recruiting for these roles, with top salaries reaching approximately $687,000. Job descriptions emphasize the need for creative problem-solvers capable of threat modeling and identifying control vulnerabilities—skills that typically require working independently within AI labs with minimal peer support.

Comparable positions at other firms such as Mercor are available at lower salary ranges of $180K–$300K, though these often require advanced degrees and demonstrated expertise in machine learning or related STEM disciplines.

The Practical Reality of "Employee-Like Access"#

Kadan Stadelmann, a security consultant at Komodo, describes evaluator work as highly variable depending on contractual arrangements. His responsibilities include examining agent behaviors under real-world scenarios, analyzing reasoning chains, and assessing whether teams follow their stated safety protocols.

Yet Stadelmann pushes back against credential gatekeeping. While METR prioritizes Ph.D. holders, he argues that the technology sector has historically thrived on demonstrated engineering competency rather than academic credentials. "I am invited into labs to probe numerous risk vectors, including agent behaviors under realistic conditions," he notes.