Skip to content
crawlwise.

AI crawler · training · OpenAI

What is GPTBot?

OpenAI's bot that collects pages for foundation-model training. When someone asks ChatGPT a question in search mode, a different bot does the fetching — OAI-SearchBot and ChatGPT-User.

How this helps you show up

Leaving GPTBot allowed does not change your Google rankings. It can put your pages in the pool of material models learn from, which some brands want when they care how an industry gets described. If your goal is citations and traffic from ChatGPT answers, focus on OAI-SearchBot and ChatGPT-User instead — those are the ones that need to reach your site today.

If it cannot reach your page

You only opt out of future training use. Answers and citations in ChatGPT search still depend on the retrieval bots above, so blocking GPTBot alone does not make you "safe" from AI answers — it just skips training.

The facts

robots.txt token
GPTBot
Owner
OpenAI
Product it serves
ChatGPT
Category
training — building model training data
Training opt-out
Documented — only affects training-style use, not every AI product
User agent as sent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3; +https://openai.com/gptbot

robots.txt is a request, not a lock. Our checker fetches your page as GPTBot and reports what actually came back — status code, challenge page, or real HTML.

Check your site in a minute

The AI crawler checker tests GPTBot plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the robots.txt tester.

Other AI crawlers