# GEO and AI search glossary: 60 terms in plain English

> Plain definitions of the terms in GEO and AI search work — AI crawlers and tokens, robots.txt and snippet controls, query fan-out, grounding, citations, share of voice and how to measure AI visibility.

Canonical: https://crawlwise.site/guides/ai-search/geo-ai-search-glossary
Last updated: 2026-10-29

This glossary defines the terms that come up in GEO and AI search work — from GPTBot and query fan-out to grounding, citations and share of voice — in two or three plain sentences each, with a link to the deeper guide where we have one. Where a term has no official definition, we say so.

by Adnan Arodiya

Updated October 29, 2026 · 11 min read

## Key takeaways

- Training crawlers, search crawlers and user-triggered fetchers are three different things.
- Google-Extended and Applebot-Extended are robots.txt tokens, not crawlers.
- A citation is a link; a mention is your name in the answer. Record both.
- Every AI visibility number is a sample of questions, engines and moments.

AI search arrived with a lot of new vocabulary, some of it precise and some of it marketing. This glossary gives each term two or three plain sentences and links to a deeper page where we have one. Where a term has no official definition, the entry says so instead of inventing one.

## The big ideas

**Generative engine optimization (GEO).** Making your pages easy for AI answer engines to find, read and cite. The term comes from a 2023 research paper by Aggarwal and colleagues. See [what is GEO](https://crawlwise.site/guides/ai-search/what-is-geo).

**Answer engine optimization (AEO).** The older industry name for getting your page used as the direct answer, such as a featured snippet. Today it is used almost interchangeably with GEO; neither has an official definition.

**AI SEO and LLMO.** Two more labels for the same goal; LLMO stands for large language model optimization. Read what a vendor means by them rather than relying on the name.

**Answer engine.** A product that answers a question in written prose, with or without links: ChatGPT search, Perplexity, Claude, Gemini, or Google’s AI Overviews and AI Mode.

**Large language model (LLM).** The model that writes the answer. On its own it answers from what it learned in training; combined with search, it can read and cite current pages.

## AI crawlers and robots.txt tokens

**AI crawler.** Any automated client run by an AI company that fetches web pages. There are three kinds — training, search and user-triggered — and each needs its own decision. See [what is an AI crawler](https://crawlwise.site/guides/ai-search/what-is-an-ai-crawler).

**Training crawler.** Fetches pages that may be used to train future models, such as GPTBot, ClaudeBot and CCBot. Blocking one opts you out of training, not out of answers.

**Search crawler.** Builds the index an assistant searches before it answers, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot. Blocking one can remove you from that product’s search answers.

**User-triggered fetcher.** Fetches one page because a person asked for it, such as ChatGPT-User, Claude-User and Perplexity-User. Several vendors say these may not follow robots.txt.

**GPTBot.** OpenAI’s training crawler. Disallowing it is a training opt-out and does not affect ChatGPT search. [More on GPTBot](https://crawlwise.site/learn/crawler/gptbot).

**OAI-SearchBot.** OpenAI’s search crawler, “used to surface websites in search results in ChatGPT’s search features.” OpenAI says opted-out sites are not shown in ChatGPT search answers. [More](https://crawlwise.site/learn/crawler/oai-searchbot).

**ChatGPT-User.** OpenAI’s fetcher for certain user actions in ChatGPT and Custom GPTs. OpenAI says robots.txt rules may not apply to it. [More](https://crawlwise.site/learn/crawler/chatgpt-user).

**ClaudeBot.** Anthropic’s training crawler. [More](https://crawlwise.site/learn/crawler/claudebot).

**Claude-SearchBot.** Anthropic’s search crawler. Anthropic says disabling it “may reduce your site’s visibility and accuracy in user search results.” [More](https://crawlwise.site/learn/crawler/claude-searchbot).

**Claude-User.** Anthropic’s fetcher for pages a person asks Claude about. Anthropic says blocking it prevents retrieval for those questions. [More](https://crawlwise.site/learn/crawler/claude-user).

**PerplexityBot.** Perplexity’s crawler for surfacing and linking sites in its results; Perplexity says it “is not used to crawl content for AI foundation models.” [More](https://crawlwise.site/learn/crawler/perplexitybot).

**Perplexity-User.** Perplexity’s fetcher for pages a user asks about. [More](https://crawlwise.site/learn/crawler/perplexity-user).

**Google-Extended.** A robots.txt token with no user agent of its own. It controls whether content Google crawls may be used for Gemini training and for grounding, and Google says it “does not impact a site’s inclusion in Google Search.” [More](https://crawlwise.site/learn/crawler/google-extended).

**Applebot-Extended.** Apple’s token for opting out of training its foundation models. Apple says it “does not crawl webpages,” and pages that disallow it can still appear in search results. [More](https://crawlwise.site/learn/crawler/applebot-extended).

**CCBot.** Common Crawl’s crawler, which builds an open web archive anyone can download. Common Crawl documents a standard robots.txt block. [More](https://crawlwise.site/learn/crawler/ccbot).

**Bytespider.** ByteDance’s crawler. We could not find English-language documentation of its purpose, so we describe what it does only where a source exists. [More](https://crawlwise.site/learn/crawler/bytespider).

**Meta-ExternalAgent.** Meta’s crawler, which Meta says crawls “for use cases such as training foundation AI models or improving products by indexing content directly.” [More](https://crawlwise.site/learn/crawler/meta-externalagent).

## Files, tags and controls

**robots.txt.** A plain-text file at the root of a host that tells crawlers which paths they may fetch. It is a request that documented crawlers choose to follow. See [does robots.txt control AI crawlers](https://crawlwise.site/guides/ai-search/does-robots-txt-control-ai-crawlers).

**RFC 9309.** The 2022 standard for robots.txt. It sets how groups and rules are matched, including that product tokens match case-insensitively.

**User-agent group.** A block of robots.txt rules for one or more crawler tokens. A crawler follows only the most specific group that names it, so the `*` group is a fallback, not a base layer.

**Training opt-out.** Blocking a training crawler or token while leaving search crawlers allowed. It is a deliberate business choice, and Crawlwise labels it as one rather than as a defect.

**llms.txt.** A proposed Markdown file that gives AI agents a map of a site. Google says Search ignores it, and Ahrefs reports that 97% of published llms.txt files were never fetched at all in a given month. See [does llms.txt affect Google Search](https://crawlwise.site/guides/ai-search/does-llms-txt-affect-google-search).

**noindex.** A robots meta tag or header that asks search engines to keep a page out of their index. A crawler has to be able to fetch the page to see it. [How we check indexing directives](https://crawlwise.site/learn/check/index).

**X-Robots-Tag.** The HTTP header form of robots directives such as `noindex` and `nosnippet`, useful for files that have no HTML head.

**nosnippet, max-snippet and data-nosnippet.** Google’s snippet controls. Google lists them, with noindex, as the way to limit what its AI features show from your pages. See [AI Overviews and AI Mode controls](https://crawlwise.site/guides/ai-search/ai-overviews-ai-mode-controls) and [our snippet check](https://crawlwise.site/learn/check/aeo-snippet-controls).

**Bot challenge.** A page a CDN or firewall serves instead of your content, asking the client to prove it is a browser. Crawlers do not solve these, so a challenge blocks them even when robots.txt allows them.

## How AI search builds an answer

**Retrieval.** Finding documents to read before writing an answer. A page that is not in the engine’s index cannot be retrieved.

**Retrieval-augmented generation (RAG).** Writing an answer from retrieved documents rather than from the model’s memory alone. It is the general pattern behind AI search answers that cite sources.

**Grounding.** Google uses the word for supplying Search index content to a model at the moment it answers. Google-Extended governs grounding in Gemini Apps and Vertex AI.

**Query fan-out.** Splitting one question into several related searches and building one answer from the results. Google says AI Overviews and AI Mode may use it. See [what is query fan-out](https://crawlwise.site/guides/ai-search/what-is-query-fan-out).

**Fan-out query.** One of the sub-searches generated during fan-out. Google does not publish them, so any list of “Google fan-out queries” is a guess.

**Deep Search.** A capability in Google’s AI Mode that takes fan-out further; Google says it “can issue hundreds of searches.”

**AI Overviews.** Google’s AI-written summary shown above some search results, with supporting links. A page must be indexed and eligible for a snippet to be one of them. See [how to show up in AI Overviews](https://crawlwise.site/guides/ai-search/how-to-show-up-in-google-ai-overviews).

**AI Mode.** Google’s conversational AI search, built on the same index. Google says it uses query fan-out.

**Passage.** A section of a page that makes sense when quoted on its own. Answer engines quote passages, not whole pages, so the first lines of each section matter.

**Citation.** A link to a source shown with an AI answer. Being cited means your page was used; it does not mean your brand was named.

**Mention.** Your name in the text of an AI answer. Crawlwise counts a mention only when the name appears in the answer itself, not when it appears only in the list of sources.

## Your pages

**Source HTML.** The HTML your server returns before any JavaScript runs. Most AI vendors do not document rendering JavaScript, so this is the safe version to optimize. [Our check for text in the source](https://crawlwise.site/learn/check/geo-readable).

**Opening answer passage.** The first lines under a heading, written as a direct answer to the question the heading asks. [How we read it](https://crawlwise.site/learn/check/aeo-answer-lead).

**Structured data.** Machine-readable labels for what a page contains, usually JSON-LD using the schema.org vocabulary. Google says it is not required for generative AI search. See [structured data in the AI search era](https://crawlwise.site/guides/ai-search/structured-data-ai-search).

**JSON-LD.** The structured data format Google recommends where your setup allows: a script block of JSON in the page, separate from the visible HTML.

**Organization markup and sameAs.** Structured data naming your company, logo, site and official profiles. The `sameAs` list should name only profiles that are really yours. [Organization schema explained](https://crawlwise.site/learn/schema/organization).

**Entity.** A thing a system can identify without ambiguity: your company, your product, a person. Consistent names and descriptions across your site and profiles make that easier.

**Author and date signals.** Who wrote a page and when it last changed. Useful to readers; no AI vendor documents them as a citation lever. [Author check](https://crawlwise.site/learn/check/geo-author) and [date check](https://crawlwise.site/learn/check/geo-date).

**Supporting sources.** Links from your page to the evidence behind its claims. They help a reader check you. [How we check them](https://crawlwise.site/learn/check/geo-sources).

## Measuring AI visibility

**AI visibility.** How often AI answers mention or cite you for the questions you care about. It is always a sample: the questions someone chose, the engines covered, the moments they ran.

**Prompt tracking.** Asking the same set of questions on a schedule and recording the answers. Most AI visibility tools are built around it; see [the best AI visibility tools](https://crawlwise.site/blog/site-audits/best-ai-visibility-tools).

**Share of voice.** How often you are mentioned compared with named competitors, across a set of prompts and engines.

**Cited sources.** The pages and domains an engine cites for your topic. Often the most useful output of any check, because it is a list of sites worth being mentioned on.

**Snapshot.** One check at one moment. AI answers change between runs, so a single snapshot is not a trend.

**GEO score.** A single number some tools publish for AI readiness or visibility. Crawlwise does not publish one, because the weights cannot be checked — see [why there is no score](https://crawlwise.site/learn/honest-measurement).

**Search generative AI performance report.** A Search Console report Google announced on June 3, 2026, showing how often your URLs appeared in generative AI features in Search and Discover, by page, country, device and date.

**AI Assistant channel.** A GA4 default channel for visits from AI assistants such as ChatGPT, Gemini and Copilot. AI Overviews and AI Mode stay in Organic Search. See [tracking AI traffic in GA4](https://crawlwise.site/guides/visibility/track-ai-traffic-in-ga4).

**utm_source=chatgpt.com.** The tag OpenAI says ChatGPT adds to links it sends, so ChatGPT visits keep their source in analytics.

**Zero-click search.** A search that ends without a visit to any website, because the answer was on the results page or in the AI answer. Analytics cannot see it; only impression reports and checks of the answers can.

> Most of this vocabulary describes one chain: can the crawler read the page, does the page answer the question, and did the answer use it.
>
> Adnan Arodiya, Crawlwise

## How Crawlwise tests this

The free [AI crawler checker](https://crawlwise.site/tools/ai-crawler-check) covers the crawler and token entries above for one URL: it reads robots.txt with an RFC 9309 matcher, sends one live request per crawler, and labels a training opt-out separately from an accidental block. A full [audit](https://crawlwise.site/) covers the page entries: source HTML, the opening answer passage, snippet controls, structured data and sources.

Every check, what counts as a block, and what the checks cannot see are written up on the [methodology page](https://crawlwise.site/methodology).

## What a plan actually costs

The free tools stay free. You can open them, run them, and leave without an account. A plan is for the moment you want the report saved, a few sites watched, or the same checks from the API. The price on this table is the price at checkout. A yearly plan is ten months of the monthly price, so two months are on us.

| Plan | Monthly | Credits | A good fit when |
| --- | --- | --- | --- |
| Starter | $4.99 | 10 | You look after one site and check it now and then |
| Pro | $14.99 | 40 | A few sites, plus the API and a handful of watched URLs |
| Studio | $49.99 | 150 | Client work that would burn through Pro mid-month |
| Agency | $99.99 | 400 | Many locations, reported under your own name |

A single-page audit is about 1.25 credits, and that includes the live probe of answer-engine crawlers. You see the estimate before anything runs. If a hold is not used, it comes back to your balance. Extra credits, when you already subscribe, are $9.99 for 20.

## Frequently asked questions

### Are GEO, AEO and AI SEO official terms?

No. They are industry labels with no standards body behind them. GEO comes from a 2023 research paper; the others grew out of SEO practice.

### What is the difference between a citation and a mention?

A citation is a link to a source shown with an AI answer. A mention is your name in the answer text. You can have either without the other, so it is worth recording both.

### Which glossary terms matter most for a small site?

Search crawler, robots.txt, server HTML and opening answer passage. Those decide whether an answer engine can read and quote you at all; the measurement terms only matter once they are right.

## Sources

Every page below was read on 2026-10-29. Vendors change prices, limits and documentation without notice, so check the original before you act on a detail.

- [Aggarwal et al. — GEO: Generative Engine Optimization (arXiv 2311.09735, accepted to KDD 2024)](https://arxiv.org/abs/2311.09735)
- [OpenAI — Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots)
- [Anthropic — Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- [Perplexity — Perplexity crawlers](https://docs.perplexity.ai/guides/bots)
- [Google Search Central — Google common crawlers (Google-Extended)](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)
- [Apple — About Applebot](https://support.apple.com/en-us/119829)
- [Meta — Meta web crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers)
- [Common Crawl — CCBot](https://commoncrawl.org/ccbot)
- [RFC 9309 — Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html)
- [Google Search Central — AI features and your website](https://developers.google.com/search/docs/appearance/ai-features)
- [Google Search Central — AI optimization guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide)
- [Google — AI in Search: Going beyond information to intelligence (May 20, 2025)](https://blog.google/products/search/google-search-ai-mode-update/)
- [Google Search Central Blog — Introducing Search Generative AI performance reports in Search Console (June 3, 2026)](https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports)
- [Analytics Help — Default channel group](https://support.google.com/analytics/answer/9756891)
- [OpenAI Help Center — Publishers and developers FAQ](https://help.openai.com/en/articles/12627856-publishers-and-developers-faq)
- [Ahrefs — 10 SEO trends I’ve seen firsthand in 2026 (Ryan Law, June 18, 2026)](https://ahrefs.com/blog/seo-trends/)

## Crawlwise crawler check vs checking by hand

| What you get | Crawlwise | By hand |
| --- | --- | --- |
| Reads robots.txt the way crawlers do (RFC 9309) | Yes, per crawler token | By eye; group precedence is easy to misread |
| Fetches the page as each AI crawler | Yes, one live request per agent | One curl at a time |
| Separates a training opt-out from an accidental block | Labelled separately | Up to you |
| Sees blocks that only happen in other regions | No — one location, one moment | Only from your own server logs |

## Meet the author

Adnan Arodiya

Crawlwise

Writes about what Crawlwise actually measures: on-page evidence, crawler access, and performance signals — with the limits stated up front.

## Related guides

[What is GEO? Generative engine optimization vs SEO vs AEO, explained simply](https://crawlwise.site/guides/ai-search/what-is-geo): GEO, AEO and SEO describe overlapping jobs. Here is what is genuinely new, and what is the same work with a new name.

[What is query fan-out, and why no tool can show you Google’s real fan-out queries](https://crawlwise.site/guides/ai-search/what-is-query-fan-out): AI search splits one question into many searches. Google does not publish them, so here is what you can and cannot know.

[How to get your website mentioned by ChatGPT: what you can control and what you can’t](https://crawlwise.site/guides/ai-search/how-to-get-mentioned-by-chatgpt): Four things you can control, three you cannot, and how to check where you stand without guessing.