AI crawler profile
What is GPTBot? How to allow or block OpenAI's crawler
Operator
OpenAI
Purpose
AI model training
robots.txt
Respects robots.txt
Official docs
GPTBot is OpenAI's web crawler for gathering training data. Content it collects may be used to improve future GPT models. It is OpenAI's broadest crawler, and the one most site owners mean when they talk about "blocking ChatGPT" — though blocking it does not remove you from ChatGPT's live search answers, which use OAI-SearchBot instead.
Does GPTBot respect robots.txt?
OpenAI documents GPTBot's IP ranges and states it honors robots.txt disallow rules.
Block GPTBot with robots.txt
User-agent: GPTBot Disallow: /
Explicitly allow GPTBot
User-agent: GPTBot Allow: /
Should you block GPTBot?
Blocking GPTBot opts your content out of OpenAI's model training — a legitimate choice for original content you don't want reproduced by AI. The trade-off: models trained without your content are less likely to know your brand or recommend it unprompted. There is no direct traffic loss today, since training crawlers don't send visitors. See our guide on whether to block AI bots for the full decision framework.
Check and monitor GPTBot on your site
- AI Crawler Access Checker — see whether your current robots.txt allows or blocks GPTBot.
- robots.txt Generator for AI Bots — build a robots.txt with per-bot rules for all 15 known AI crawlers.
- AI Bot Log Analyzer — paste server logs and count real GPTBot hits, entirely in your browser.
Frequently asked questions
What is GPTBot?
GPTBot is OpenAI's web crawler for collecting AI training data. Crawls content to train OpenAI's models.
Does GPTBot respect robots.txt?
OpenAI documents GPTBot's IP ranges and states it honors robots.txt disallow rules.
How do I block GPTBot?
Add "User-agent: GPTBot" followed by "Disallow: /" to your robots.txt file. The change takes effect the next time the bot fetches your robots.txt.
Does blocking GPTBot hurt my Google rankings?
No. GPTBot is separate from Googlebot, which handles Google Search indexing. Blocking GPTBot has no effect on your traditional search rankings.
How can I tell if GPTBot is crawling my site?
Search your server access logs for the string "GPTBot". Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including GPTBot, with per-path breakdowns.
More OpenAI agents
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-07-15.