Skip to content

OpenAI Launches First Round of Embedded Assessments with Four External Research Groups

5 reports 3 reporting sources first reported Updated 40 minutes ago

Understand the event

AI overview

OpenAI has launched its first round of embedded assessments, giving four external research groups—Apollo Research, METR Evals, Transluce AI, and GoodfireAI—deep access to its internal models and systems to investigate alignment and test safeguards, according to OpenAI and co-founder Greg Brockman. In a pilot, OpenAI will host six Apollo Research safety evaluators to examine whether models covertly work against developers' intentions or undermine their own or successor models' alignment and control mechanisms, OpenAI and Apollo said. Apollo said it will receive relevant training code, datasets, reinforcement-learning rollouts, and intermediate checkpoints, but its access to staff and information is limited to agreed claims, specific models, and time periods. OpenAI also reserves the right to withhold information for technical infeasibility, security risks, or IP protection, and while Apollo can publish independent conclusions, OpenAI may limit the public evidence supporting them. GoodfireAI will embed interpretability researchers to use its platform to analyze OpenAI's model internals, OpenAI said. The company stated that GoodfireAI's work focuses on training run monitoring, debugging, and alignment to help OpenAI researchers identify and make safety improvements.

Generated by AI from reports · Updated 27 minutes ago

Read next

More changes in today’s briefingGemini Connects to Google Wallet for Personal Finance Insights

Developments

2 developments

  1. 11/17 00:03 (UTC+8) · 1 reports
    OpenAI Partners with GoodfireAI on Model Interpretability Research
    OpenAI · X:@GoodfireAI will embed interpretability researchers to use their platform to analyze our models’ internals. Their work focuses on training run monitoring, debugging, and alignment, helping our researchers identify and make safety improvements.
  2. 11/17 00:03 (UTC+8) · 4 reports
    OpenAI Launches Its First Round of Embedded Assessments
    OpenAI · X:OpenAI Works With Four Research Groups on Embedded Assessments

Report timeline

Follow the reports to explore different sides of the event.

11/17
  1. OpenAI to Embed Apollo Researchers in Pilot on AI Subversion

    OpenAI will host six Apollo Research safety evaluators in a pilot examining whether models try to undermine their own or successor models’ alignment and control mechanisms. Apollo says it will receive relevant training code and datasets, reinforcement-learning rollouts, and access to intermediate checkpoints, but its access to staff and information is limited. It can publish independent conclusions, while OpenAI may limit the evidence available to support them publicly.

  2. Greg Brockman Announces OpenAI’s First Round of Embedded Assessments

    Greg Brockman announces OpenAI’s first round of embedded assessments, quoting OpenAI’s description of work with Apollo Research, METR Evals, Transluce AI, and Goodfire AI. The assessments give outside experts deep access to OpenAI’s internal models and systems to investigate alignment and test safeguards.

  3. @GoodfireAI will embed interpretability researchers to use their platform to analyze our models’ internals. Their work focuses on training run monitoring, debugging, and alignment, helping our researchers identify and make safety improvements.

    @GoodfireAI will embed interpretability researchers to use their platform to analyze our models’ internals. Their work focuses on training run monitoring, debugging, and alignment, helping our researchers identify and make safety improvements.

  4. Apollo Research to Assess OpenAI Models for Covert Behavior

    Apollo Research will assess whether OpenAI's models could covertly act against their developers’ intentions and whether safeguards would detect that behavior. The assessment will examine evidence from training, safety testing, and internal use.

  5. OpenAI Works With Four Research Groups on Embedded Assessments

    OpenAI is working with Apollo Research, METR Evals, Transluce AI, and Goodfire AI on embedded assessments that give outside experts deep access to its internal models and systems. The experts will investigate alignment and test safeguards as part of OpenAI’s ongoing alignment and safety research.

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.