Skip to content

Crawl

Multi-page crawl

Discover URLs on your site, fetch them in chunks, and store the link graph for Structure and internal linking.

Run a crawl before Structure or cross-page checks. Later tools read from the latest crawl.

Step by step

Step 1. Set the page limit

How to use it: Enter how many URLs to fetch. The cap is a budget — hitting it stops cleanly with whatever was crawled.

Step 1 — Set the page limit

Step 2. Start the crawl

How to use it: Press Start crawl after you review the credit estimate. Progress shows crawled vs queued until it finishes.

Step 2 — Start the crawl

Step 3. Open Structure

How to use it: When the crawl completes, switch to the Structure tab to see depth, duplicates, and the link graph.

Step 3 — Open Structure