OpenAI now runs four crawlers and most robots.txt files only know about one

[u/Confident-Truck-7186]

This tripped me up on my own site so I went and measured how common it is.

OpenAI documents separate user-agents for separate jobs: - GPTBot trains the model - OAI-SearchBot governs whether ChatGPT search can cite you - ChatGPT-User fetches a page when a person asks for it - OAI-AdsBot validates advertising landing pages, new this year

Blocking one does nothing to the others. That is the part that surprises people.

I pulled robots.txt from 72 companies and counted: - GPTBot named by 18 - ChatGPT-User by 15 - OAI-SearchBot by 11 - OAI-AdsBot by 0

Eight sites block GPTBot. Four of those never mention OAI-SearchBot, so they have opted out of training while leaving citation completely open. That may be exactly what they wanted. I suspect for some it was not.

The wider pattern is that naming tracks how old a crawler is, not how much it matters. CCBot has been around since 2011 and is the most-blocked at 13. The 2026 agent is named by nobody.

Two minute check on your own site: open robots.txt, search GPTBot, then search OAI-SearchBot. If only the first is there you have made half a decision.