Skip to content
K = 0.731Subscribe
Viewing history for · This page does not vary by date2026年10月26日 周一‹ Previous dayNext day ›AccelerationBack to now

OpenAI, Google, Microsoft, and Anthropic Back Shared robots.txt Signal for AI Scraping Preferences

3 reports 3 reporting sources Updated 1 day ago

Understand the event

AI overview

OpenAI, Google, Microsoft, and Anthropic have backed a shared robots.txt signal for publishers to express AI scraping preferences across participating companies; Anthropic has already adopted it, while the others plan to add support. OpenAI says it will implement the signal in the coming months, and Google and Microsoft plan support in 2027. The shared signal gives website operators a way to communicate their preferences to multiple providers. OpenAI says it will honor the signal for AI training. Microsoft says it will offer separate controls for opting out of AI training or AI grounding, and that websites will remain eligible to appear in Bing search results regardless of those preferences.

Generated by AI from reports · Updated 1 day ago

Latest development10/31 04:26 (UTC+8)
OpenAI, Google, Microsoft, and Anthropic back shared AI scraping opt-out standardThe Verge · AI

Read next

More changes in today’s briefingConfigure the ChatGPT desktop app with Amazon Bedrock in AWS GovCloud (US)

Report timeline

Follow the reports to explore different sides of the event.

10/31
  1. TechCrunch · AI
    TechCrunch Suggests Last-Minute Halloween Costumes for Tech Nerds

    TechCrunch offers last-minute tech-themed Halloween costume ideas, including dressing as recursive self-improvement, a cute AI agent, an AI usage limit notification, or a PHP vulnerability. The article also suggests costumes inspired by Mark Zuckerberg’s T-shirts and Bryan Johnson’s “GLP-Bryan” post.

  2. The Verge · AI
    OpenAI, Google, Microsoft, and Anthropic back shared AI scraping opt-out standard

    OpenAI, Google, Microsoft, and Anthropic say they will support a shared policy token in robots.txt that lets publishers express AI scraping preferences across companies. Anthropic has already adopted the signal, Google and Microsoft plan support next year, and OpenAI says implementation will come in the coming months. Microsoft will distinguish between AI training and AI grounding preferences while keeping websites eligible to appear in Bing search results.

10/30
  1. OpenAI News
    OpenAI will honor a shared robots.txt signal for AI training preferences

    OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

Current within comparable coverage
9
Peak within comparable coverage
1710/31 05:00 (UTC+8)
24-hour change
-50%

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.