# SEO audit methodology > Our SEO audit methodology, published: every check Crawlwise runs, what it measures, how, what it cannot measure, and whether it counts toward the score. Canonical: https://crawlwise.site/methodology The SEO audit methodology behind every check this product can report: what it measures, how it was determined, and what it cannot measure. The rules are published as data at [/methodology/checks.json](https://crawlwise.site/methodology/checks.json) so they can be read, diffed and argued with without reading the engine. Ruleset version 7 · definitions version 1 · 34 checks · scored: 22 ## How to read this page **Scored** means the check can count towards the checklist score when it applies and is measurable. A check marked not scored is reported and never counted — accessibility and generative-search findings are shown because they are worth knowing, not because they change a number. The score’s denominator is always shown beside it, and a page where nothing scorable applied has no score rather than a zero. Each scored check counts by its stated impact (high 3, medium 2, low 1), and a pass earns all of it, a warning half and a failure none. A page that tells Google not to index it, or that robots.txt blocks for Googlebot, is held at 40; a page without HTTPS at 60. Reports saved before methodology 7 keep the equal-weight score they were saved with. **Not measured** is a first-class answer. Where a check could not be evaluated the report says so, and it is excluded from the score rather than counted as a failure. ## SEO Whether search engines can discover, crawl, and understand this page. - [HTTPS connection](https://crawlwise.site/methodology#https) scored - [Page title](https://crawlwise.site/methodology#title) scored - [Meta description](https://crawlwise.site/methodology#description) scored - [Indexing directives](https://crawlwise.site/methodology#index) scored - [Canonical URL](https://crawlwise.site/methodology#canonical) scored - [Main heading](https://crawlwise.site/methodology#h1) scored - [Mobile viewport](https://crawlwise.site/methodology#viewport) scored - [Image alternative text](https://crawlwise.site/methodology#alt) scored - [Descriptive link labels](https://crawlwise.site/methodology#link-text) scored - [Structured data formats](https://crawlwise.site/methodology#schema-presence) scored - [JSON-LD syntax](https://crawlwise.site/methodology#schema-json) scored - [JSON-LD context](https://crawlwise.site/methodology#schema-context) scored - [Common schema properties](https://crawlwise.site/methodology#schema-fields) scored - [robots.txt discovery](https://crawlwise.site/methodology#robots) scored - [Googlebot access in robots.txt](https://crawlwise.site/methodology#robots-googlebot) scored - [Sitemap discovery](https://crawlwise.site/methodology#sitemap) scored - [Reserved image dimensions](https://crawlwise.site/methodology#image-size) scored - [HTML response weight](https://crawlwise.site/methodology#html-size) scored - [Server response time](https://crawlwise.site/methodology#response-time) scored - [Mobile Lighthouse performance](https://crawlwise.site/methodology#lighthouse-mobile) scored - [Desktop Lighthouse performance](https://crawlwise.site/methodology#lighthouse-desktop) not scored - [Synchronous head scripts](https://crawlwise.site/methodology#blocking-scripts) scored - [Image loading strategy](https://crawlwise.site/methodology#image-loading) scored ## Answer clarity Whether the page answers the question a reader arrived with, and whether that answer is reachable. - [Text available in source](https://crawlwise.site/methodology#geo-readable) not scored - [Content structure](https://crawlwise.site/methodology#geo-outline) not scored - [Snippet controls](https://crawlwise.site/methodology#aeo-snippet-controls) not scored - [Opening answer passage](https://crawlwise.site/methodology#aeo-answer-lead) not scored ## AI search The evidence a generative answer could draw on, plus any visibility that has actually been measured. - [Author or contributor signal](https://crawlwise.site/methodology#geo-author) not scored - [Published or updated date](https://crawlwise.site/methodology#geo-date) not scored - [Links to supporting sources](https://crawlwise.site/methodology#geo-sources) not scored - [About or contact discovery](https://crawlwise.site/methodology#geo-identity) not scored - [Reading cost for an AI agent](https://crawlwise.site/methodology#geo-agent-reading) not scored ## Additional improvements Worth doing for readers and assistive technology. Not search ranking signals, and never scored as such. - [Heading structure](https://crawlwise.site/methodology#a11y-heading-structure) not scored - [Document language](https://crawlwise.site/methodology#a11y-document-language) not scored ## Search ### HTTPS connection `https` **What it measures.** Whether the URL that answered is served over HTTPS, after following redirects. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the certificate is valid for every hostname the page loads from. - Whether the HTTP address still resolves and answers. **What to do.** Serve the site over HTTPS and redirect the HTTP address to it. Free tool: [mixed-content](https://crawlwise.site/tools/mixed-content) ### Page title `title` **What it measures.** Whether the head contains a title element and how long it is. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether Google will use it. Google rewrites titles it considers unhelpful, whatever this says. - Whether the title reads well, which is a judgement about your copy. **What to do.** Write one title per page, specific to that page, and put the distinguishing words first. Free tool: [serp](https://crawlwise.site/tools/serp) ### Meta description `description` **What it measures.** Whether a meta description is present and its length. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether it will be shown. Google frequently writes its own snippet from page text. - Whether a missing description costs you clicks — that depends on the query. **What to do.** Add a meta description. A missing one is a missed opportunity rather than an error. Free tool: [serp](https://crawlwise.site/tools/serp) ### Indexing directives `index` **What it measures.** The robots meta tag and X-Robots-Tag as sent, read as data rather than as prose. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether a page is actually indexed. That requires Search Console or an index check. - Whether a directive came from your code or a CDN rule. **What to do.** Remove a noindex you did not intend. A deliberate one should stay. Free tool: [robots-tester](https://crawlwise.site/tools/robots-tester) ### Canonical URL `canonical` **What it measures.** Whether the page declares a canonical and what it points at. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Which URL wins when two pages disagree. Google decides that. - Whether a canonical sent as an HTTP header says something different — see the headers tool. **What to do.** Declare a self-referencing canonical unless duplicates genuinely exist. Free tool: [canonical-checker](https://crawlwise.site/tools/canonical-checker) ### Main heading `h1` **What it measures.** Whether the source HTML contains exactly one h1 and its text. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the heading is visually hidden or styled to look like something else. - Whether the outline reads well to a person. **What to do.** Use one h1 that describes the page, and let the sections below it be h2s. Free tool: [structured-data](https://crawlwise.site/tools/structured-data) ### Mobile viewport `viewport` **What it measures.** Whether a viewport meta tag is present. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the layout actually works on a phone. That needs rendering at a mobile width. - Whether a CSS media query does the same job instead. **What to do.** Add a viewport meta tag with width=device-width and an initial scale of 1. Free tool: [meta-tags](https://crawlwise.site/tools/meta-tags) ### Image alternative text `alt` **What it measures.** How many images in the source HTML carry an alt attribute, including empty ones. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the text describes the image, which no tool can judge. - Whether an image is decorative, where an empty alt is the correct answer. **What to do.** Give informative images real alt text and decorative ones an empty alt. Free tool: [structured-data](https://crawlwise.site/tools/structured-data) ### Descriptive link labels `link-text` **What it measures.** The anchor text of internal links, looking for generic labels like "click here" or "read more". **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the destination is any good. - Whether a generic label is fine in context, which it sometimes is. **What to do.** Name the destination in the link text. "View the pricing page" beats "click here". Free tool: [broken-link-checker](https://crawlwise.site/tools/broken-link-checker) ### Snippet controls `aeo-snippet-controls` **What it measures.** Whether directives limit how much of the page a search engine may show. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the directive is being honoured, which the engine does not report. - Whether limiting the snippet is the right choice for you. **What to do.** Remove nosnippet or max-snippet limits if you want the full text quotable. Free tool: [meta-tags](https://crawlwise.site/tools/meta-tags) ### Opening answer passage `aeo-answer-lead` **What it measures.** Whether the page opens with a self-contained passage that answers the obvious question. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether that passage is the one anyone would quote. - Whether a quote would ever be displayed — that is not something a page controls. **What to do.** Open with a direct answer in two or three sentences, before the build-up. Free tool: [query-fanout](https://crawlwise.site/tools/query-fanout) ## Structured data ### Structured data formats `schema-presence` **What it measures.** Which structured data formats appear: JSON-LD, microdata, RDFa, and their @type values. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the markup is valid. That is the JSON-LD syntax check. - Whether the values are true statements about the business. **What to do.** Add JSON-LD for the entity the page represents, when the page actually represents one. Free tool: [schema-validator](https://crawlwise.site/tools/schema-validator) ### JSON-LD syntax `schema-json` **What it measures.** Whether each JSON-LD block parses as JSON. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the parsed markup is eligible for a rich result. That is a feature-requirements question. - Whether microdata or RDFa on the page is valid — neither is parsed here. **What to do.** Fix the JSON. A block that does not parse is invisible to every consumer. Free tool: [schema-validator](https://crawlwise.site/tools/schema-validator) ### JSON-LD context `schema-context` **What it measures.** Whether each JSON-LD block declares an @context of schema.org. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the types used inside it exist in the vocabulary. An unknown type parses and means nothing. - Whether a nested node inherits the context correctly, beyond the depth we walk. **What to do.** Add "@context": "https://schema.org" to every block. Free tool: [schema-validator](https://crawlwise.site/tools/schema-validator) ### Common schema properties `schema-fields` **What it measures.** Which commonly expected properties are present on each entity type. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the values are correct. - Whether these properties are the ones Google asks for today — that list changes without notice. **What to do.** Fill in the properties the entity genuinely has. Never invent a value to satisfy a checker. Free tool: [schema-validator](https://crawlwise.site/tools/schema-validator) ## Discovery files ### robots.txt discovery `robots` **What it measures.** Whether /robots.txt answers and whether its content parses as a robots file. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the rules inside it are the ones you want. Use the robots tester for a specific URL and crawler. - Whether a CDN or WAF answers differently for a bot. **What to do.** Serve a robots.txt that parses, and point it at your sitemap. Free tool: [robots-tester](https://crawlwise.site/tools/robots-tester) ### Googlebot access in robots.txt `robots-googlebot` **What it measures.** Whether robots.txt lets Googlebot crawl this exact URL, using the Googlebot group and the longest matching rule (RFC 9309). A robots.txt that answers a server error counts as a block, because Google stops crawling until it recovers. A block holds the score at 40. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether Google has indexed the page. Search Console's URL Inspection says that. - Whether a firewall or CDN refuses Googlebot even though robots.txt allows it. **What to do.** Remove or narrow the rule that blocks the URL, unless you mean to keep it out of Google. Free tool: [robots-tester](https://crawlwise.site/tools/robots-tester) ### Sitemap discovery `sitemap` **What it measures.** Whether /sitemap.xml answers and how many URLs it contains. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the listed URLs work. The sitemap validator samples them. - Whether every page on the site is listed. **What to do.** Publish a sitemap, keep it current, and declare it in robots.txt. Free tool: [sitemap-validator](https://crawlwise.site/tools/sitemap-validator) ## Performance ### Reserved image dimensions `image-size` **What it measures.** How many images in the source declare width and height attributes. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - The real layout shift this page causes. That is a field measurement, and it needs traffic. - Whether a CSS rule or aspect-ratio already reserves the space. **What to do.** Set width and height on images, or reserve the space in CSS. Free tool: [vitals](https://crawlwise.site/tools/vitals) ### HTML response weight `html-size` **What it measures.** The byte size of the HTML response as received, compressed or not as sent. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the page is slow. HTML is usually a small part of the total, and assets are separate requests. - The rendered size after JavaScript runs. **What to do.** Send less HTML, or compress it. Check the content-encoding header first. Free tool: [vitals](https://crawlwise.site/tools/vitals) ### Server response time `response-time` **What it measures.** Time to first byte for the audited request, from our machine. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - What a visitor experiences. One measurement from one location is a sample, not a field number. - Whether the delay is your origin, your CDN, or the network between. **What to do.** Cache at the edge, and look at the server side if the first byte is slow everywhere. Free tool: [headers](https://crawlwise.site/tools/headers) ### Mobile Lighthouse performance `lighthouse-mobile` **What it measures.** The Lighthouse performance score from a Google PageSpeed Insights mobile lab run: 90 and above passes, 50 to 89 warns, below 50 fails. Its individual suggestions are listed beside it but are never scored. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - What real visitors experience. A lab run is one simulated load, not Core Web Vitals field data. - A stable number. Lighthouse scores move a few points between runs of the same page. **What to do.** Work through the measured opportunities, largest saving first, and retest. Free tool: [vitals](https://crawlwise.site/tools/vitals) ### Desktop Lighthouse performance `lighthouse-desktop` **What it measures.** The same Lighthouse lab run on a desktop profile, reported for comparison. Google ranks on the mobile version, so only the mobile score counts. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - What real visitors experience. A lab run is one simulated load, not Core Web Vitals field data. - A stable number. Lighthouse scores move a few points between runs of the same page. **What to do.** Fix the mobile opportunities first; most also improve the desktop score. Free tool: [vitals](https://crawlwise.site/tools/vitals) ### Synchronous head scripts `blocking-scripts` **What it measures.** Script tags in the head with neither async nor defer. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the script is actually blocking in practice, which depends on where and how it is inserted. - The cost in milliseconds. That is a lab measurement. **What to do.** Add async or defer, or move the script out of the head. Free tool: [vitals](https://crawlwise.site/tools/vitals) ### Image loading strategy `image-loading` **What it measures.** Which images declare loading="lazy" and which declare fetchpriority. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether lazy-loading is correct for a particular image. The largest one above the fold should not be lazy. - The effect on Core Web Vitals, which needs a measurement. **What to do.** Lazy-load below-the-fold images and leave the hero image eager. Free tool: [vitals](https://crawlwise.site/tools/vitals) ## Generative search ### Text available in source `geo-readable` **What it measures.** How much readable text is present in the HTML the server returns. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether that text is any good. - What a browser would show after JavaScript runs — a client-rendered page looks thin here and may be fine. **What to do.** Serve the content in the HTML if you want anything outside a browser to read it. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Content structure `geo-outline` **What it measures.** The heading outline of the document as sent. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the structure reflects the argument the page makes. - Anything about headings a script inserts later. **What to do.** Use headings that describe their sections, in a single sensible hierarchy. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Author or contributor signal `geo-author` **What it measures.** Whether the page declares an author, by name, by link, or in structured data. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether that person exists or knows the subject. - Whether a byline is appropriate — plenty of good pages correctly have none. **What to do.** Attribute the page to a real person and link to something that shows why they would know. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Published or updated date `geo-date` **What it measures.** Whether the page declares a published or modified date a machine can read. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the date is true. - Whether the content is current — a fresh date on stale text is worse than none. **What to do.** Declare a modified date that changes when the content actually changes. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Links to supporting sources `geo-sources` **What it measures.** Outbound links, and whether any point at primary or reference sources. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the sources support the claim. That is a reading of the page. - Whether a link to a .gov domain is relevant to your subject. **What to do.** Link to the primary source for each claim that has one. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### About or contact discovery `geo-identity` **What it measures.** Whether the page or its navigation links to an about or contact page. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the business is real or reachable. - Whether the entity is who it says it is. **What to do.** Link to a page that says who publishes this site and how to reach them. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Reading cost for an AI agent `geo-agent-reading` **What it measures.** The tokens the page costs as HTML and as the Markdown an agent fetch tool keeps, and whether the page links a Markdown version. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - How any one agent converts the page; each uses its own rules. - Whether an engine reads, ranks or cites the page. No engine documents using either figure. **What to do.** Keep the facts and answers in the HTML as text inside
. A Markdown version is optional. Free tool: [ai-agent-view](https://crawlwise.site/tools/ai-agent-view) ## Accessibility and reader experience ### Heading structure `a11y-heading-structure` **What it measures.** Whether heading levels are sequential and no level is skipped. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the document is usable with a screen reader, which needs a real assistive technology. - Whether a skipped level matters on a particular page. **What to do.** Do not skip levels. Use h3 because the section is a subsection, not because it looks right. Free tool: [visibility](https://crawlwise.site/tools/visibility) ### Document language `a11y-document-language` **What it measures.** Whether the html element declares a language and whether it is a valid language tag. **How it was determined.** Parsed from the response the server returned, recorded on the scan, and re-read from the stored report rather than re-derived later. **What it cannot measure.** - Whether the page is actually written in that language. - What to do about a page that mixes languages, which the specification leaves to the author. **What to do.** Set lang on the html element to the language the page is written in. Free tool: [meta-tags](https://crawlwise.site/tools/meta-tags) ## Checks this page does not define Three families are generated from data rather than listed above: the per-crawler AI access findings (one per crawler probed, named in the report), the per-feature rich-result findings (one per schema type Google supports), and the cross-page findings from a whole-site crawl. Their rules live in `lib/checks/` and `lib/crawl/` and are versioned by the same ruleset number. Withdrawn checks are listed in `lib/scope.ts`. A saved report that contains one is filtered at read time rather than rewritten, because a stored report is a record of what was true on the day it ran. ## Open source, deliberately The rules are published, not the engine. Anyone can read what each check does, implement the same check, and disagree with ours on the evidence. That is the point: a finding you cannot inspect is a finding you have to take on trust. [Read the definitions as JSON](https://crawlwise.site/methodology/checks.json) [What crawlwise cannot tell you](https://crawlwise.site/limits)