# What is Meta-ExternalAgent?

> Meta-ExternalAgent (Meta): what it does, how it helps your site show up in AI products, and how to test access on your own URL.

Canonical: https://crawlwise.site/learn/crawler/meta-externalagent

AI crawler · training · Meta

Meta's crawler for AI products built on Llama and related systems.

### How this helps you show up

Feeds Meta's AI stack. Link previews in Facebook or Instagram use other fetchers — this is about AI products, not social cards.

### If it cannot reach your page

Meta's AI training and retrieval pipelines will not pick up new pages from your site.

### The facts

**robots.txt token**

Meta-ExternalAgent

**Owner**

Meta

**Product it serves**

Meta AI

**Category**

training — building model training data

**Training opt-out**

Documented — only affects training-style use, not every AI product

**User agent as sent**

`Mozilla/5.0 (compatible; Meta-ExternalAgent/1.0; +https://developers.facebook.com/docs/sharing/webmasters/crawler)`

robots.txt is a request, not a lock. Our checker fetches your page as Meta-ExternalAgent and reports what actually came back — status code, challenge page, or real HTML.

### Check your site in a minute

The [AI crawler checker](https://crawlwise.site/tools/ai-crawler-check) tests Meta-ExternalAgent plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the [robots.txt tester](https://crawlwise.site/tools/robots-tester).

### Other AI crawlers

- [GPTBot](https://crawlwise.site/learn/crawler/gptbot) — OpenAI
- [OAI-SearchBot](https://crawlwise.site/learn/crawler/oai-searchbot) — OpenAI
- [ChatGPT-User](https://crawlwise.site/learn/crawler/chatgpt-user) — OpenAI
- [ClaudeBot](https://crawlwise.site/learn/crawler/claudebot) — Anthropic
- [Claude-SearchBot](https://crawlwise.site/learn/crawler/claude-searchbot) — Anthropic
- [Claude-User](https://crawlwise.site/learn/crawler/claude-user) — Anthropic
- [Google-Extended](https://crawlwise.site/learn/crawler/google-extended) — Google
- [PerplexityBot](https://crawlwise.site/learn/crawler/perplexitybot) — Perplexity
- [Perplexity-User](https://crawlwise.site/learn/crawler/perplexity-user) — Perplexity
- [CCBot](https://crawlwise.site/learn/crawler/ccbot) — Common Crawl
- [Bytespider](https://crawlwise.site/learn/crawler/bytespider) — ByteDance
- [Applebot-Extended](https://crawlwise.site/learn/crawler/applebot-extended) — Apple
- [Amazonbot](https://crawlwise.site/learn/crawler/amazonbot) — Amazon
- [cohere-ai](https://crawlwise.site/learn/crawler/cohere-ai) — Cohere
- [Diffbot](https://crawlwise.site/learn/crawler/diffbot) — Diffbot
- [Timpibot](https://crawlwise.site/learn/crawler/timpibot) — Timpi