🔍 Read the full analysis: How To Set The AI Training Opt-Out—and Understand Its Boundaries on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will honor a shared robots.txt signal asking them not to use a site’s content for generative AI training. The signal is voluntary, does not block AI answers based on live web content, and is not yet an official internet standard; Meta has voiced support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The move could let publishers express the preference once instead of maintaining separate training rules for each company, but the signal is voluntary and does not cover every use of web content.
The proposed rule uses the user-agent name __usage_ai-training, followed by Disallow: / in a robots.txt file. It expresses a site owner’s preference against training use; it does not technically prevent a company from accessing the site. The shared signal is set out in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” written by Anthropic standards lead Kevin Kelley and published October 30.
The companies have given different timelines. Anthropic says it has started implementing support, though it says deployment across its services will take time. OpenAI says it aims to implement the signal in the coming months. Google’s documentation points to early 2027, while Microsoft says it will honor the preference from February 1, 2027, as it crawls sites.
Meta has expressed support for publishers’ ability to state preferences, but spokesperson Andy Stone did not say whether the company will honor this specific signal. The source report says it found no announcement from Apple, Amazon, xAI or Perplexity. Existing company-specific rules, including GPTBot, Google-Extended and ClaudeBot, remain in place while adoption proceeds.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
For publishers, a common rule could reduce the work of expressing a training preference across multiple services. Instead of relying only on separate crawler names and company-specific instructions, site operators can add a shared signal. Its practical value will depend on which companies adopt it, when they do so and how they interpret it.
The distinction between a preference and enforcement matters. Robots.txt communicates instructions to crawlers that choose to honor them; it is not a technical barrier. The rule also addresses model training, not all AI-related uses. A site may want to allow crawling for search or for AI systems that retrieve current pages to answer questions while declining use of its material for model training.
Microsoft’s announcement illustrates that separation: alongside its training preference, it is introducing a separate page-level choice for AI Grounding, meaning content used to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both preferences take effect February 1, 2027. The company also says neither choice guarantees inclusion, traffic or payment. The shared training signal itself does not establish compensation for publishers.
Products Worth Considering
Dell UltraSharp 32 U3226QER 4K Thunderbolt Hub Monitor
Samsung Essential S3 27-Inch Curved FHD Monitor
Dell P2725H 27-Inch Full HD Monitor
As an affiliate, we earn on qualifying purchases.
How Existing Publisher Controls Fit
Before this announcement, site operators generally had to use different controls for different services. Those included named crawler directives such as GPTBot, Google-Extended and ClaudeBot; Microsoft also used page-level NoArchive and NoCache tags for certain preferences. The new signal is intended to provide a common way to state a training preference, not to erase those existing controls immediately.
Google says that if a site uses the new token and has no separate Google-Extended rule, it plans to interpret the shared preference as disallowing Google-Extended. Google says the change will not affect Google Search. Microsoft says its new preferences will replace NoArchive and NoCache as training or grounding preferences after March 1, 2027. Site owners using those tags should check Microsoft’s instructions and plan for that date rather than assuming the older tags will retain the same meaning.
The proposal is connected to the Internet Engineering Task Force’s AI Preferences working group, but the document remains an individual Internet-Draft. The source report says it has no formal standing in the IETF standards process. At an October 21 working-group meeting, a chair said the group had not declared consensus on anything.
““has already started implementing support for this draft””
— Anthropic
Adoption and Coverage Remain Open
It is not yet clear how many AI companies beyond the four named in the announcements will honor the rule, or how consistently they will apply it across their products and services. Meta’s support statement is not an implementation commitment, and the source report found no public commitment from several other major technology and AI companies.
The proposal’s standards status is also unresolved: publication as an Internet-Draft does not make it an adopted IETF standard. The companies’ stated dates are plans, not evidence that every service has already changed its behavior. The signal also cannot remove material from models that have already been trained, and it does not itself settle how companies handle content used to produce AI answers from live web pages.
Watch for Rollout Dates and Rules
Publishers considering the opt-out can add the shared user-agent and disallow lines to robots.txt, while keeping existing company-specific rules until each service’s support and instructions are clear. They should decide separately whether to permit training, search crawling and AI answer use; a training opt-out should not be assumed to control all three.
The next milestones are the companies’ rollouts: OpenAI’s planned implementation in the coming months, Google’s target of early 2027, and Microsoft’s February 1, 2027 start date. Microsoft says NoArchive and NoCache stop counting as training or grounding preferences after March 1, 2027. Anthropic says its implementation is under way, but has not given a date for full support across its services. Adoption by Meta and other companies, along with the IETF working group’s consideration of the draft, remains to be seen.
Key Questions
What should a website add to request an AI training opt-out?
The proposed robots.txt rule uses User-Agent: __usage_ai-training with Disallow: /. It communicates a preference against training use to services that honor the signal; it does not block access technically.
Does the shared rule block AI answers based on a site?
No. The proposed signal concerns generative AI model training, not every use of web content. Microsoft announced a separate AI Grounding preference for content used to inform some AI answers.
When will the companies honor the signal?
Timelines differ. Anthropic says implementation has started, OpenAI aims to adopt the signal in the coming months, Google targets early 2027, and Microsoft says its policy begins February 1, 2027.
Has Meta committed to honoring the opt-out?
Not in the statement described by the source report. Meta expressed support for publishers’ ability to state preferences, but did not say whether it would honor this specific signal.
Is this an official internet standard?
No. The signal appears in an individual Internet-Draft connected to the IETF’s AI Preferences working group. The source report says the draft has no formal standing in the IETF standards process.
Source: ThorstenMeyerAI.com
