Skip to content
crawlwise.

SEO audits in OpenAI Codex via MCP

Codex can run Crawlwise audits, AI crawler checks and site crawls as MCP tools, then fix what they find in your repo. The server speaks streamable HTTP at https://crawlwise.site/mcp.

1. Add the server

One command adds the Crawlwise MCP server.

bash
codex mcp add crawlwise --url https://crawlwise.site/mcp
codex mcp login crawlwise

Or in ~/.codex/config.toml

bash
[mcp_servers.crawlwise]
url = "https://crawlwise.site/mcp"

2. Sign in

The first tool call opens the Crawlwise sign-in page. Approve it and the tools stay connected. If your client cannot do OAuth, create a key at /account/api (Pro and above) and send it as Authorization: Bearer cw_live_….

3. Check it worked

Restart Codex and run /mcp. You should see check_ai_crawlers and audit_page.

4. No MCP? Pipe the CLI

The CLI works without an account for a first look (1 page a day). Its llm format is written for an agent: only defects, each citing its check.

bash
npx crawlwise audit https://example.com --format llm

Prompts to try

  • Audit https://example.com with crawlwise and fix the priority items. Re-audit until none are left.
  • Use crawlwise to check whether GPTBot, ClaudeBot and PerplexityBot can read https://example.com.
  • Crawl my verified site with crawlwise and list broken internal links and redirect chains.

The agent should fix only what is under priority. Items marked not measured are tests that returned no result, not defects.

OAuth is the default. A Bearer header in config.toml also works if you already have a key.

All agents · Integrations · MCP reference · Claude Code · Gemini CLI · GitHub Copilot · ChatGPT · OpenCode · GitHub Action