Does privacymanager.io block AI crawlers?
privacymanager.io serves a bot challenge to 1, and allows 16 (checked 2026-10-01).
The blocks are on research and corpus crawlers rather than on the assistants people use to search, so citation in ChatGPT, Claude and Perplexity is not affected by them.
Every crawler we tested
| Crawler | Used for | Verdict | What we saw |
|---|---|---|---|
| GPTBot OpenAI | ChatGPT (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| OAI-SearchBot OpenAI | ChatGPT search (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| ChatGPT-User OpenAI | ChatGPT (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| ClaudeBot Anthropic | Claude (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Claude-SearchBot Anthropic | Claude search (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Claude-User Anthropic | Claude (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Google-Extended Google | Gemini (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| PerplexityBot Perplexity | Perplexity (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Perplexity-User Perplexity | Perplexity (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| CCBot Common Crawl | Common Crawl (research) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Bytespider ByteDance | ByteDance (training) | Challenged | Challenged: robots.txt permits it, but the fetch got a bot challenge page instead of the content. |
| Applebot-Extended Apple | Apple Intelligence (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Meta-ExternalAgent Meta | Meta AI (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Amazonbot Amazon | Alexa (training) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| cohere-ai Cohere | Cohere (retrieval) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Diffbot Diffbot | Diffbot (research) | Allowed | Allowed: robots.txt permits it and the page loaded. |
| Timpibot Timpi | Timpi (research) | Allowed | Allowed: robots.txt permits it and the page loaded. |
Homepage SEO snapshot
From the Crawlwise SEO Index run, checked 2026-10-05. One fetch of the homepage, so it describes that page and not the whole site.
| Homepage | https://liveramp.com/ |
|---|---|
| HTTPS | yes |
| Title | 31 characters |
| Meta description | 150 characters |
| Canonical | points to itself |
| Noindex | no |
| H1 elements | 1 |
| Language | en |
| JSON-LD types | Organization, ImageObject, ContactPoint |
| hreflang alternates | 4 |
| /sitemap.xml | valid XML sitemap |
| Response time | 91 ms |
How this was measured
We read privacymanager.io’s robots.txt, resolved it with an RFC 9309 matcher for each crawler’s product token, then fetched the homepage once as each crawler. A 403 or a challenge page is a block even when robots.txt allows the crawler; a timeout or rate limit is “not established”, never a block. One probe, from one location, on 2026-10-01, so a rule changed since then is not reflected here. Full methodology.
Check your own site
The free checker runs the same probe on any URL: robots.txt resolved for every AI crawler, then a live fetch as each one, so a CDN or firewall block shows up even when robots.txt looks fine.
Need it across every page? Site crawls start at $4.99 a month, and Pro adds monitoring that re-checks your URLs on a schedule.
Other sites
- rapid7.com — blocks none of 17
- comodoca.com — blocks none of 17
- appier.net — blocks none of 17
- thousandeyes.com — blocks none of 17
- truste.com — blocks none of 17
- wyzecam.com — blocks none of 17