Wix SEO audit: robots.txt, AI crawlers and what to check

Wix handles most technical SEO for you, which is why its robots.txt editor is easy to overlook and easy to get wrong. This audit checks the file, what crawlers actually receive, and the page settings Wix leaves to you.

Where do you edit robots.txt on Wix?

In your site’s dashboard, open SEO & GEO, scroll to Tools and settings, click Robots.txt Editor and then View File. Write your directives, click Save Changes and confirm. Wix calls this an advanced feature and its Customer Care will not help with changes you make, so keep a copy of what you started with.

What does the default file allow?

Wix says the default file allows search engine bots to access all of your site’s pages. Password-protected pages, members-only pages and pages you have set not to be indexed can still be kept from bots. Wix’s article does not mention AI crawlers or a dedicated setting for them, so any AI rule is a robots.txt rule you add yourself.

How do you allow or block an AI crawler?

Add a group for the crawler’s token:

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

That opts out of Anthropic’s training crawler while letting the crawler behind Claude’s search results fetch your pages. Each AI company runs separate crawlers for training and for live answers; which AI crawlers to allow explains the difference, and what is an AI crawler covers the basics.

Reset to Default exists for a reason. Use it the moment a robots.txt change does something you did not expect.

Adnan Arodiya, Crawlwise

What else should a Wix audit check?

• Per-page SEO settings. Give every important page its own title and meta description; dynamic pages need their patterns set so each item gets a distinct title.
• Pages hidden from search. A page set not to be indexed should be one you meant to hide; check the setting on pages that earn no traffic.
• The sitemap. Wix generates one; submit it in Search Console and Bing Webmaster Tools, and after a robots.txt change, resubmit your homepage so search engines refresh their cached copy sooner, as Wix suggests.
• Content in the HTML. Text inside images, galleries or embedded apps may not be readable to a crawler, so keep key answers in real text.

Check it on your own Wix site

Run the free AI crawler checker on your homepage and one product or article page. It reads your robots.txt the way each crawler does, then fetches the page as GPTBot, ClaudeBot, PerplexityBot and 14 others, so a block added by a CDN, firewall or app shows up even when the file looks right. The robots.txt tester shows which rule decides a given path, and a full audit adds titles, canonicals, structured data, hreflang and performance for the same URL.

To see how popular sites set these rules, look any of them up in the site-by-site AI crawler results.

What a plan actually costs

The free tools stay free. You can open them, run them, and leave without an account. A plan is for the moment you want the report saved, a few sites watched, or the same checks from the API. The price on this table is the price at checkout. A yearly plan is ten months of the monthly price, so two months are on us.

A single-page audit is about 1.25 credits, and that includes the live probe of answer-engine crawlers. You see the estimate before anything runs. If a hold is not used, it comes back to your balance. Extra credits, when you already subscribe, are $9.99 for 20.

Frequently asked questions

Where is the robots.txt editor in Wix?

In the dashboard, open SEO & GEO, scroll to Tools and settings, click Robots.txt Editor and then View File.

Does Wix have a setting for AI crawlers?

Wix’s robots.txt help article does not mention one. To allow or block a specific AI crawler, add a User-agent group in the robots.txt editor.

How do I undo a robots.txt change on Wix?

Open the editor, click View File, then Reset to Default and confirm. The file returns to Wix’s default.

Sources

Platform documentation was read on 2026-10-08. Platforms change these pages and settings without notice, so check the original before you act on a detail.

• Wix Help Center — Editing your site’s robots.txt file
• RFC 9309 — Robots Exclusion Protocol
• Crawlwise — The AI Crawler Access Report