OpenAI and Google plan a shared AI-training signal for websites
OpenAI and Google have announced plans to recognize a shared AI-training preference in robots.txt, but neither announcement says the new mechanism is operating today. OpenAI is aiming for the coming months. Google is aiming for 2027. For publishers, the news is a proposed way to communicate one preference to multiple participating providers, not a reason to mistake a future control for a deployed one.
The companies published their announcements on October 30. OpenAI says its existing crawler controls will continue, while Google describes how the shared signal would interact with Google-Extended. Those implementation details matter more than the shorthand description of a universal AI opt-out.
What is planned, and when?
| Participant | Announced timing | Important limit |
|---|---|---|
| OpenAI | Coming months | Implementation details still to follow. Current controls remain. |
| 2027 | Explicit Google-Extended instructions still matter. | |
| Microsoft | February 1, 2027, as crawled | Separate page-level grounding preferences and migration requirements. |
| Technical proposal | Internet-Draft dated October 30 | Work in progress, not a ratified standard. |
Google plans to treat a shared training opt-out like a Google-Extended disallow instruction when Google-Extended is not explicitly specified. Its company-specific control would continue alongside the broader signal. Google also says sites specifically allowing Google-Extended are eligible for grounding responses and clicks in Gemini Apps. Eligibility is not a promise of traffic.
Microsoft adds a separate grounding choice
In an October 30 announcement, Microsoft says it will honor new training and grounding preferences from February 1, 2027, as it crawls. Training can be specified in robots.txt or per page; page-level training settings override the shared setting. Grounding is a separate, page-level choice. Neither setting changes eligibility for traditional Bing search, subject to normal crawling and indexing requirements.
Microsoft’s DisallowExtended grounding option permits Microsoft-operated experiences but excludes its third-party grounding services. Both preferences otherwise default to Allow. Existing NoArchive/NoCache users face a transition: those tags stop expressing training or grounding preferences after March 1, 2027. Sites using those tags as of December 15, 2026 and preserving grounding restrictions must publish page preferences and record a temporary Bing Webmaster Tools choice by then. There is no temporary training choice. The grounding choice lasts until recrawl or September 30, 2027, whichever comes first.
This is a forthcoming Microsoft implementation, not evidence that the shared control works today. The announcement does not guarantee inclusion, traffic or payment.
A preference token is not a new crawler
The linked Machine-Readable Training Preference Signal draft proposes the token __usage_ai-training. It uses ordinary robots.txt groups and existing Allow and Disallow syntax to attach preferences to paths. It does not create a new crawling directive.
The distinction is between what a provider may retrieve and how content is intended to be used. In the proposal, a crawler still interprets rules for its own product token separately. A wildcard user-agent group does not match the policy token, and absence of the policy token means no preference has been expressed through this particular mechanism. That is not a statement about every other instruction or agreement.
The IETF Datatracker lists this as an individual Internet-Draft by Kevin Kelley, dated October 30. Its notice explicitly says it is not endorsed by the IETF and has no formal standing in the standards process. Provider support for a proposal and completion of an internet standard are different milestones.
Training and search already have separate controls
OpenAI’s current crawler documentation distinguishes GPTBot, which crawls content that may be used for model training, from OAI-SearchBot, which supports ChatGPT search. The settings are independent. A publisher can allow search crawling while disallowing GPTBot under that documented system.
ChatGPT-User is another case: it handles certain user-initiated visits, rather than automatic web crawling, and OpenAI says robots.txt rules may not apply. A training preference should therefore not be described as a switch that prevents every AI-related visit or every possible use of a page.
What publishers should take from the announcement
The practical question is not just whether a website has said yes or no to AI. It is which provider, which purpose, which content and which implementation date that choice refers to. A useful internal review can keep those four questions separate before anyone considers a configuration change.
For example, an editorial team might want its reporting discoverable in search while choosing a different position on training use. That is a hypothetical publishing decision, not a test of this draft or a recommended robots.txt configuration. The existing provider documentation and the eventual implementation notices are the evidence needed to map that decision to an actual control.
The development is worth watching because shared signaling could reduce repeated administration. Its usefulness will depend on participation and implementation, not simply on the presence of a new token in a file. These announcements do not establish universal compliance or a legal guarantee. For now, treat the shared mechanism as forthcoming and keep current crawler controls distinct from the proposal.
