AI crawler profile
What is PerplexityBot? How to allow or block Perplexity's crawler
Operator
Perplexity
Purpose
AI search & live answers
robots.txt
Partially respects robots.txt
Official docs
PerplexityBot builds the index behind Perplexity's cited, search-style answers. Perplexity is an answer engine rather than a model trainer, so being crawled by PerplexityBot is primarily about visibility: it determines whether your pages can appear as sources in Perplexity results.
Does PerplexityBot respect robots.txt?
Perplexity says PerplexityBot respects robots.txt, but independent investigations (notably Cloudflare's 2024–2025 reports) have observed undeclared fetching that bypassed blocks.
Block PerplexityBot with robots.txt
User-agent: PerplexityBot Disallow: /
Explicitly allow PerplexityBot
User-agent: PerplexityBot Allow: /
Should you block PerplexityBot?
For most sites, no. PerplexityBot powers visibility: blocking it removes your pages from the cited answers Perplexity's assistant shows its users — the AI-era equivalent of de-indexing yourself from a search engine. Block it only if you deliberately don't want that audience. Our guide on blocking AI bots covers the trade-offs in detail.
Check and monitor PerplexityBot on your site
- AI Crawler Access Checker — see whether your current robots.txt allows or blocks PerplexityBot.
- robots.txt Generator for AI Bots — build a robots.txt with per-bot rules for all 15 known AI crawlers.
- AI Bot Log Analyzer — paste server logs and count real PerplexityBot hits, entirely in your browser.
Frequently asked questions
What is PerplexityBot?
PerplexityBot is Perplexity's agent for AI search and live answers. Crawls and indexes pages for Perplexity's answer engine.
Does PerplexityBot respect robots.txt?
Perplexity says PerplexityBot respects robots.txt, but independent investigations (notably Cloudflare's 2024–2025 reports) have observed undeclared fetching that bypassed blocks.
How do I block PerplexityBot?
Add "User-agent: PerplexityBot" followed by "Disallow: /" to your robots.txt file. The change takes effect the next time the bot fetches your robots.txt.
Does blocking PerplexityBot hurt my Google rankings?
No. PerplexityBot is separate from Googlebot, which handles Google Search indexing. Blocking PerplexityBot has no effect on your traditional search rankings, but it does remove your pages from the AI answers Perplexity's assistant serves to its users.
How can I tell if PerplexityBot is crawling my site?
Search your server access logs for the string "PerplexityBot". Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including PerplexityBot, with per-path breakdowns.
More Perplexity agents
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-07-15.