OpenAI will honor a shared robots.txt signal for AI training preferences

What happened

OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.

Read next

• More changes in today’s briefingMistral releases improved Mistral Large 4 open weights for coding and cybersecurity

Reported by 2 sources

The story so far

OpenAI, Google, Microsoft, and Anthropic have backed a shared robots.txt signal for publishers to express AI scraping preferences across participating companies; Anthropic has already adopted it, while the others plan to add support. OpenAI says it will implement the signal in the coming months, and Google and Microsoft plan support in 2027. The shared signal gives website operators a way to communicate their preferences to multiple providers. OpenAI says it will honor the signal for AI training. Microsoft says it will offer separate controls for opting out of AI training or AI grounding, and that websites will remain eligible to appear in Bing search results regardless of those preferences.