AI crawler · retrieval · Perplexity
What is Perplexity-User?
A one-off fetch for a specific user question, not a full re-index.
How this helps you show up
Another chance to be the source at the moment someone asks. Worth allowing for the same reason you would not block a referred visitor.
If it cannot reach your page
That question gets answered from whatever sources were already reachable.
The facts
- robots.txt token
- Perplexity-User
- Owner
- Perplexity
- Product it serves
- Perplexity
- Category
- retrieval — indexed so an answer can cite it
- Training opt-out
- Retrieval / user fetch — usually keep open for citations
- User agent as sent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
robots.txt is a request, not a lock. Our checker fetches your page as Perplexity-User and reports what actually came back — status code, challenge page, or real HTML.
Check your site in a minute
The AI crawler checker tests Perplexity-User plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the robots.txt tester.
Other AI crawlers
- GPTBot — OpenAI
- OAI-SearchBot — OpenAI
- ChatGPT-User — OpenAI
- ClaudeBot — Anthropic
- Claude-SearchBot — Anthropic
- Claude-User — Anthropic
- Google-Extended — Google
- PerplexityBot — Perplexity
- CCBot — Common Crawl
- Bytespider — ByteDance
- Applebot-Extended — Apple
- Meta-ExternalAgent — Meta
- Amazonbot — Amazon
- cohere-ai — Cohere
- Diffbot — Diffbot
- Timpibot — Timpi