AI crawler · retrieval · Cohere
What is cohere-ai?
Cohere's crawler for retrieval and training in Cohere-powered apps.
How this helps you show up
Smaller reach than OpenAI or Google, but straightforward: if partners build on Cohere and you want to be quotable, the bot needs access.
If it cannot reach your page
Cohere-backed apps will not see new pages from you.
The facts
- robots.txt token
- cohere-ai
- Owner
- Cohere
- Product it serves
- Cohere
- Category
- retrieval — indexed so an answer can cite it
- Training opt-out
- Documented — only affects training-style use, not every AI product
- User agent as sent
Mozilla/5.0 (compatible; cohere-ai/1.0; +https://cohere.com/bot)
robots.txt is a request, not a lock. Our checker fetches your page as cohere-ai and reports what actually came back — status code, challenge page, or real HTML.
Check your site in a minute
The AI crawler checker tests cohere-ai plus the other answer-engine bots on one URL. No account. Want robots.txt logic only? Use the robots.txt tester.
Other AI crawlers
- GPTBot — OpenAI
- OAI-SearchBot — OpenAI
- ChatGPT-User — OpenAI
- ClaudeBot — Anthropic
- Claude-SearchBot — Anthropic
- Claude-User — Anthropic
- Google-Extended — Google
- PerplexityBot — Perplexity
- Perplexity-User — Perplexity
- CCBot — Common Crawl
- Bytespider — ByteDance
- Applebot-Extended — Apple
- Meta-ExternalAgent — Meta
- Amazonbot — Amazon
- Diffbot — Diffbot
- Timpibot — Timpi