AI crawler · retrieval · Perplexity
What is PerplexityBot?
Perplexity's index crawler. Answers include citations, and this bot is how your pages get into that index.
How this helps you show up
If you want Perplexity to recommend or cite you, this bot needs a clean 200 response. Strong headlines and factual opening paragraphs matter because the answer often pulls from the top of the page.
If it cannot reach your page
Perplexity answers without you in the source list — your competitor's page may sit there instead.
The facts
- robots.txt token
- PerplexityBot
- Owner
- Perplexity
- Product it serves
- Perplexity
- Category
- retrieval — indexed so an answer can cite it
- Training opt-out
- Retrieval / user fetch — usually keep open for citations
- User agent as sent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
robots.txt is a request, not a lock. Our checker fetches your page as PerplexityBot and reports what actually came back — status code, challenge page, or real HTML.
Check your site in a minute
The AI crawler checker tests PerplexityBot plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the robots.txt tester.
Other AI crawlers
- GPTBot — OpenAI
- OAI-SearchBot — OpenAI
- ChatGPT-User — OpenAI
- ClaudeBot — Anthropic
- Claude-SearchBot — Anthropic
- Claude-User — Anthropic
- Google-Extended — Google
- Perplexity-User — Perplexity
- CCBot — Common Crawl
- Bytespider — ByteDance
- Applebot-Extended — Apple
- Meta-ExternalAgent — Meta
- Amazonbot — Amazon
- cohere-ai — Cohere
- Diffbot — Diffbot
- Timpibot — Timpi