AI crawler profile
What is Meta-ExternalAgent? How to allow or block Meta's crawler
Meta-ExternalAgent is Meta's crawler for collecting training data for its Llama models and Meta AI products. It is distinct from facebookexternalhit (which fetches link previews), so you can block AI training without breaking how your links render on Facebook and Instagram.
Does Meta-ExternalAgent respect robots.txt?
Meta documents the agent and states it respects robots.txt rules.
Block Meta-ExternalAgent with robots.txt
User-agent: Meta-ExternalAgent Disallow: /
Explicitly allow Meta-ExternalAgent
User-agent: Meta-ExternalAgent Allow: /
Should you block Meta-ExternalAgent?
Blocking Meta-ExternalAgent opts your content out of Meta's model training — a legitimate choice for original content you don't want reproduced by AI. The trade-off: models trained without your content are less likely to know your brand or recommend it unprompted. There is no direct traffic loss today, since training crawlers don't send visitors. See our guide on whether to block AI bots for the full decision framework.
Check and monitor Meta-ExternalAgent on your site
- AI Crawler Access Checker — see whether your current robots.txt allows or blocks Meta-ExternalAgent.
- robots.txt Generator for AI Bots — build a robots.txt with per-bot rules for all 15 known AI crawlers.
- AI Bot Log Analyzer — paste server logs and count real Meta-ExternalAgent hits, entirely in your browser.
Frequently asked questions
What is Meta-ExternalAgent?
Meta-ExternalAgent is Meta's web crawler for collecting AI training data. Crawls content to train Meta's AI models.
Does Meta-ExternalAgent respect robots.txt?
Meta documents the agent and states it respects robots.txt rules.
How do I block Meta-ExternalAgent?
Add "User-agent: Meta-ExternalAgent" followed by "Disallow: /" to your robots.txt file. The change takes effect the next time the bot fetches your robots.txt.
Does blocking Meta-ExternalAgent hurt my Google rankings?
No. Meta-ExternalAgent is separate from Googlebot, which handles Google Search indexing. Blocking Meta-ExternalAgent has no effect on your traditional search rankings.
How can I tell if Meta-ExternalAgent is crawling my site?
Search your server access logs for the string "Meta-ExternalAgent". Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including Meta-ExternalAgent, with per-path breakdowns.
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-07-15.