# What is Claude-SearchBot?

> Claude-SearchBot (Anthropic): what it does, how it helps your site show up in AI products, and how to test access on your own URL.

Canonical: https://crawlwise.site/learn/crawler/claude-searchbot

AI crawler · retrieval · Anthropic

Anthropic's retrieval crawler for Claude search — it finds pages that Claude can quote or link to.

### How this helps you show up

Keep this one happy. Readable HTML, real text on the page (not only in images), and a robots.txt that allows Claude-SearchBot are the boring basics that get you cited.

### If it cannot reach your page

Claude search will not quote a page its crawler cannot load. You stay invisible in that product even if your Google SEO is fine.

### The facts

**robots.txt token**

Claude-SearchBot

**Owner**

Anthropic

**Product it serves**

Claude search

**Category**

retrieval — indexed so an answer can cite it

**Training opt-out**

Retrieval / user fetch — usually keep open for citations

**User agent as sent**

`Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +Claude-SearchBot@anthropic.com)`

robots.txt is a request, not a lock. Our checker fetches your page as Claude-SearchBot and reports what actually came back — status code, challenge page, or real HTML.

### Check your site in a minute

The [AI crawler checker](https://crawlwise.site/tools/ai-crawler-check) tests Claude-SearchBot plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the [robots.txt tester](https://crawlwise.site/tools/robots-tester).

### Other AI crawlers

- [GPTBot](https://crawlwise.site/learn/crawler/gptbot) — OpenAI
- [OAI-SearchBot](https://crawlwise.site/learn/crawler/oai-searchbot) — OpenAI
- [ChatGPT-User](https://crawlwise.site/learn/crawler/chatgpt-user) — OpenAI
- [ClaudeBot](https://crawlwise.site/learn/crawler/claudebot) — Anthropic
- [Claude-User](https://crawlwise.site/learn/crawler/claude-user) — Anthropic
- [Google-Extended](https://crawlwise.site/learn/crawler/google-extended) — Google
- [PerplexityBot](https://crawlwise.site/learn/crawler/perplexitybot) — Perplexity
- [Perplexity-User](https://crawlwise.site/learn/crawler/perplexity-user) — Perplexity
- [CCBot](https://crawlwise.site/learn/crawler/ccbot) — Common Crawl
- [Bytespider](https://crawlwise.site/learn/crawler/bytespider) — ByteDance
- [Applebot-Extended](https://crawlwise.site/learn/crawler/applebot-extended) — Apple
- [Meta-ExternalAgent](https://crawlwise.site/learn/crawler/meta-externalagent) — Meta
- [Amazonbot](https://crawlwise.site/learn/crawler/amazonbot) — Amazon
- [cohere-ai](https://crawlwise.site/learn/crawler/cohere-ai) — Cohere
- [Diffbot](https://crawlwise.site/learn/crawler/diffbot) — Diffbot
- [Timpibot](https://crawlwise.site/learn/crawler/timpibot) — Timpi