Facebook
Britain's News Portal
Around The Clock
BREAKING
Loading latest headlines…

Anthropic and OpenAI Propose Embedding Third-Party AI Safety Evaluators

Anthropic and OpenAI have proposed embedding third-party safety evaluators within their AI labs, offering unprecedented access to their systems.

  • Anthropic CEO Dario Amodei proposed embedding third-party evaluators to report safety incidents and assess AI model alignment.
  • OpenAI CEO Sam Altman also committed to the practice, which could change how the industry works with outside research groups.
  • Evaluators welcome the proposal but seek clearer details and legislative backing to ensure their independence.

Anthropic CEO Dario Amodei has proposed embedding third-party evaluators within frontier AI companies, granting them the authority to report safety incidents, assess AI model alignment, and publicly share their findings. Amodei stated that Anthropic would provide independent evaluators, such as METR and Redwood Research, with unprecedented access to its systems.

OpenAI CEO Sam Altman has also committed to this practice. This move could significantly alter how the AI industry collaborates with external research groups.

Third-party evaluators have generally welcomed the proposal, but they emphasise the need for detailed plans and, ideally, legislative support to ensure their independence. Researchers suggest that deeper access, including to intermediate model versions during training, is crucial to uncover problematic behaviour that might be concealed during final model testing.

However, it remains unclear when and how Anthropic and OpenAI will implement this access. Neither company has specified which evaluators they will work with, when they will be embedded, the number of evaluators, the exact systems and information they can access, or what information can be disclosed to the public.

Why this matters: This proposal could enhance transparency and oversight in the development of advanced AI models, potentially improving their safety and alignment.

Related Articles

Get the news that matters.

Join thousands of readers getting the best of British news straight to their inbox.