Your robots.txt says what AI bots may do — your access logs show what they actually do. This analyzer parses pasted log lines entirely in your browser and reports which AI crawlers are visiting, how often, and which pages they're fetching.
Why it matters for AI search: AI crawler traffic is the ground truth of your GEO efforts. If OAI-SearchBot and PerplexityBot are actively fetching your pages, your content is being indexed for AI answers; if only training bots like GPTBot and CCBot show up, you're feeding models without gaining search-style visibility; if nothing shows up at all, your content isn't on AI platforms' radar yet — three very different situations demanding different responses, and indistinguishable without looking at logs. The per-path breakdown adds another layer: seeing which pages AI bots prioritize tells you what they consider your most valuable content.
How this tool works: paste raw access-log lines — Apache or nginx combined format works best, but any format containing user-agent strings is fine — and the analyzer matches each line against the same continuously updated AI bot registry that powers our other crawler tools: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, CCBot, Bytespider, Applebot-Extended, Amazonbot, Meta-ExternalAgent, and cohere-ai. You get total hit counts per bot, each bot's operator and purpose (training vs. search), and the top paths each bot requested. Everything runs client-side: your logs never leave your browser, which matters since access logs contain visitor IP addresses.
Limitations: the analyzer matches known user-agent substrings, so bots spoofing browser user agents or crawlers not yet in our registry won't be counted — the registry is refreshed monthly, but brand-new crawlers can lag a few weeks. Browser-based parsing also has practical size limits: paste up to a few hundred thousand lines comfortably, but multi-gigabyte log files should be pre-filtered (e.g. grep -i "gptbot\|claudebot\|perplexity" access.log) before pasting. And remember that a log sample is a snapshot — AI crawler activity is bursty, so trends across weeks are more meaningful than any single day's count. To change what these bots are allowed to do, pair this tool with the robots.txt Generator for AI Bots.