Loading
AI crawler profile
User-agent string to match in robots.txt and server logs: PerplexityBot
Operator
Perplexity
Purpose
AI search & live answers
robots.txt
Partially respects robots.txt
Official docs
PerplexityBot builds the index behind Perplexity's cited, search-style answers. Perplexity is an answer engine rather than a model trainer, so being crawled by PerplexityBot is primarily about visibility: it determines whether your pages can appear as sources in Perplexity results.
Perplexity says PerplexityBot respects robots.txt, but independent investigations (notably Cloudflare's 2024–2025 reports) have observed undeclared fetching that bypassed blocks.
The user-agent string above is self-declared, so anything can send it. Perplexity publishes PerplexityBot's IP ranges as JSON, which is what makes a rule verifiable: check the request's IP against the published prefixes instead of trusting the name. Requests claiming to be PerplexityBot from outside those ranges are spoofed — useful to know whether you're allowing or blocking it.
User-agent: PerplexityBot Disallow: /
User-agent: PerplexityBot Allow: /
PerplexityBot's robots.txt compliance is unreliable, so a robots.txt rule alone may not stop it. These enforce the block. Matching on the user-agent string still trusts a self-declared header — pair it with an IP check against the published ranges above for a rule that can't be spoofed.
if ($http_user_agent ~* "PerplexityBot") {
return 403;
}RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} PerplexityBot [NC]
RewriteRule .* - [F,L](http.user_agent contains "PerplexityBot")
grep -i "PerplexityBot" /var/log/nginx/access.log | wc -l
For most sites, no. PerplexityBot powers visibility: blocking it removes your pages from the cited answers Perplexity's assistant shows its users — the AI-era equivalent of de-indexing yourself from a search engine. Block it only if you deliberately don't want that audience. Our guide on blocking AI bots covers the trade-offs in detail.
PerplexityBot is Perplexity's agent for AI search and live answers. Crawls and indexes pages for Perplexity's answer engine.
Perplexity says PerplexityBot respects robots.txt, but independent investigations (notably Cloudflare's 2024–2025 reports) have observed undeclared fetching that bypassed blocks.
Add "User-agent: PerplexityBot" followed by "Disallow: /" to your robots.txt file. The change takes effect the next time the bot fetches your robots.txt.
No. PerplexityBot is separate from Googlebot, which handles Google Search indexing. Blocking PerplexityBot has no effect on your traditional search rankings, but it does remove your pages from the AI answers Perplexity's assistant serves to its users.
Search your server access logs for the string "PerplexityBot" — for example: grep -i "PerplexityBot" /var/log/nginx/access.log | wc -l. Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including PerplexityBot, with per-path breakdowns.
You don't — PerplexityBot isn't a tool you run. It's Perplexity's own crawler, operated by Perplexity, that visits your site from their infrastructure. The only control you have over it is whether you allow or block it, via robots.txt or a server rule. If you're looking to crawl other sites yourself, you'd write your own crawler or use a crawling library; sending "PerplexityBot" as your user agent would be impersonating Perplexity.
Almost certainly not. Crawl-delay was never part of the original robots.txt specification — Google has publicly said it ignores the directive, and the AI crawlers that model their parsers on Google's do the same. Perplexity publishes no position on crawl-delay for PerplexityBot, so treat it as unsupported. If PerplexityBot is hitting your site harder than you want, rate-limit it at the server or CDN instead: a Cloudflare rate-limiting rule or an nginx limit_req zone matched on the user agent will actually be enforced, whereas a crawl-delay line is only a request that this agent likely never reads.
Perplexity makes no enforceable commitment that PerplexityBot honors robots.txt, so the plain answer may be that it simply does not. Before assuming that, rule out the ordinary causes: a cached copy of robots.txt from before your change, a user-agent line that does not actually match, or a block placed in a group this agent does not read. If the hits continue after those are excluded, robots.txt is not going to stop this agent and you need a firewall or CDN rule that returns 403.
Treat it as no unless Perplexity says otherwise. Rendering JavaScript costs an order of magnitude more than fetching HTML, and most AI crawlers — unlike Googlebot, which runs a full headless Chrome — read the raw HTML response and stop there. The practical consequence: anything your page loads client-side after the initial response is likely invisible to PerplexityBot. If your main content is client-rendered, server-render it or pre-render it so the text exists in the first response.
Perplexity publishes them as JSON at https://www.perplexity.ai/perplexitybot.json, updated as their infrastructure changes. Fetch that file rather than hard-coding the prefixes, and match the request IP against it to confirm a visit is genuinely PerplexityBot. This is the only reliable check — the user-agent header is self-declared, so a scraper can send "PerplexityBot" while coming from anywhere.
There is no fixed schedule. PerplexityBot serves live answers, so its crawl rate tracks demand — pages that come up in Perplexity conversations get fetched more often, and a site nobody asks about may see it rarely or never. Frequency also rises with how often your pages change and how many links point at them. The only way to know for your own site is to measure it: grep your access log for the user agent, or paste the log into our AI Bot Log Analyzer, which breaks hits down by date and path in your browser.
Perplexity publishes it at https://docs.perplexity.ai/guides/bots. That page is the authoritative source for the user-agent string and Perplexity's stated crawling policy, and Perplexity also publishes PerplexityBot's IP ranges as JSON at https://www.perplexity.ai/perplexitybot.json so you can verify requests rather than trusting the header.
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-09-13.