OpenAI Commits to Honoring Shared robots.txt Signal, Letting Websites Express Training Preferences to Multiple AI Providers Uniformly
OpenAI will honor a shared robots.txt signal, __usage_ai-training, that lets websites communicate AI training preferences to participating providers. Google plans to support the same proposal in early 2027.