Generative engine optimization: what it is and how to actually do it

Somewhere around 2024, people who write about content marketing needed a name for "make ChatGPT mention my company" and they landed on generative engine optimization, GEO. We build an audit tool, not a content agency, so our interest in GEO is narrower and more literal: which specific, checkable things on a page make an AI model more or less likely to read it, understand it, and cite it. This is that list, with the reasoning behind each item.

What generative engine optimization actually is

Traditional search optimization gets a page ranked in a list of ten blue links. GEO gets a page cited, quoted, or paraphrased inside a generated answer, whether that answer comes from ChatGPT, Perplexity, Google's AI Overviews, or an agent doing research on your behalf. The output isn't a ranking position anymore, it's a sentence in someone else's answer with your domain attached (or not attached, which is its own problem).

That shift changes what "winning" looks like. You can't check your GEO rank the way you check a SERP position, because the same query produces a different generated answer every time, sometimes citing you and sometimes not. What you can control is everything upstream of that: whether the answer engine's crawler was allowed to fetch your page, whether it could parse the content once it did, and whether the specific passage that answers the query is written in a way that survives being lifted out of context and dropped into a chat window.

It's worth being honest about scale before you rearrange your content strategy around this. For most sites, organic search traffic still dwarfs anything referred from ChatGPT or Perplexity, by a wide margin. GEO is a real and growing channel, not yet a replacement channel. Treat everything below as additive to your existing SEO work, not a pivot away from it.

How answer engines actually pick sources

Each of the big three works differently, and the differences matter for what you optimize.

OpenAI runs three separate crawlers with three separate purposes, documented in its overview of OpenAI crawlers: GPTBot collects training data, OAI-SearchBot indexes pages for ChatGPT's search feature, and ChatGPT-User fetches a page live when a user's prompt triggers a browsing action. You can block training and still show up in live answers, because they're gated separately in robots.txt.

Anthropic does the same split for Claude: ClaudeBot trains, Claude-User and Claude-SearchBot handle search and live fetches. Anthropic's own crawler documentation walks through exactly what each one does and how to block them individually.

Google folds AI Overviews and AI Mode into the existing Search index rather than running a separate crawl. If Googlebot can already index a page, it's eligible to be summarized. Google's public guidance on AI features and your website says the ranking systems are shared with classic Search, so there's no separate "AI SEO" index to chase, just the same one with a generative layer on top that increasingly favors sources that are easy to extract a clean answer from.

Perplexity is different again: a smaller, more curated index than Google's, biased toward pages that are well-cited, current, and easy to quote directly. It leans on retrieval plus synthesis rather than a training corpus, so freshness and clear sourcing matter more than raw domain authority.

The common thread: every one of these systems has to fetch your page (or already have it indexed), parse it, and pull out a self-contained chunk that answers a specific question. Block the fetch and none of the rest matters. Bury the answer under five paragraphs of preamble and the model has more work to do finding it, which mechanically lowers the odds it gets used.

None of these companies publish a ranking algorithm the way Google eventually documented parts of Search. What we do have is their own crawler and product documentation, which tells you the mechanics even when it won't tell you the weighting: which bots exist, what each one is for, and roughly what shape of content each system is built to retrieve. That's enough to act on even without a leaked scoring formula.

GEO vs SEO: the overlap is bigger than the hype admits

Most GEO advice you'll read reduces to "write clear, authoritative, well-structured content," which is also just good SEO advice from the last decade. Content marketers didn't invent a new discipline so much as repackage E-E-A-T for a new distribution channel. If your site already ranks well because it's fast, crawlable, and genuinely useful, you're most of the way to being GEO-ready.

The parts that are actually new:

• AI crawlers are a separate allow list from search crawlers. You can rank #1 in Google and still be invisible to ChatGPT if GPTBot or OAI-SearchBot is blocked in robots.txt. This is the single most common GEO gap we see, usually left over from a blanket "block all bots" rule written before these user agents existed.
• Passages get lifted out of context, so each one has to stand alone. A search snippet links back to the full page for context. A generated answer often doesn't, or the link is secondary to the summarized text. Write the answer to "what is X" in the first sentence of the section about X, not as the payoff of a five-paragraph buildup.
• Machine-readable summaries are a new surface. llms.txt and clean Markdown/JSON alternatives to your HTML give agents a shortcut. Nothing forces an answer engine to use them yet, but they're cheap to publish and nobody's arguing they hurt.
• Freshness reads differently. Perplexity and AI Overviews both weight recency more heavily than classic organic ranking does for non-news queries, because a stale answer is a worse user experience in a single generated paragraph than it is buried on page two of results.

What's still unproven

We'd rather undersell this than oversell it. Nobody outside OpenAI, Anthropic, Google, and Perplexity actually knows the retrieval weighting, and most of the GEO advice circulating right now, including plenty from vendors selling GEO tools, is inference dressed up as certainty. Two things worth flagging:

• Citation counts are noisy and non-reproducible. Ask the same question twice and you can get different sources cited, even seconds apart, because these are generative systems sampling from a distribution, not deterministic lookups. Track trends over weeks, not single queries.
• llms.txt adoption by the model providers themselves is unconfirmed. It's a community-driven convention, not a standard any answer engine has committed to reading. We recommend it because it's cheap and plausible, not because we have evidence a specific engine consumes it.

The checklist

Run through this in order. Each one is independently checkable, and most of them are a squirrelscan audit rule.

1. Confirm AI crawlers can reach you. Open robots.txt and check for Disallow: / under GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, and Google-Extended. If you want to opt out of model training but stay answerable, block the training-only bots (GPTBot, ClaudeBot, Google-Extended) and explicitly allow the live-fetch and search bots.

2. Publish an llms.txt. A plain Markdown file at your domain root: an H1 title, a one-line summary, then linked sections pointing at your most useful pages. The format is defined at llmstxt.org. It costs about ten minutes to write and gives an agent a map of your site without crawling the whole thing.

3. Structure content so single sections answer single questions. Lead each H2 with the direct answer, then explain. Compare "Our platform was built by a team who spent years thinking deeply about developer workflows before we ever wrote a line of code, and eventually that led us to..." against "squirrelscan is a CLI that audits websites against 249+ rules." The second sentence is the one that survives being pasted into a chat window. This is the single highest-leverage content change for GEO and it's also just better writing.

4. Add schema markup for the content type you actually have. Schema.org structured data (Article, FAQPage, Product, HowTo) gives answer engines an unambiguous signal about what a page is, on top of whatever they infer from the prose. FAQPage schema in particular maps almost one-to-one onto the question-and-answer format these systems generate, which is why we ship a faq block on every guide like this one. It doesn't guarantee a citation, but it removes one source of ambiguity.

5. Keep dates current and visible. A visible "updated" date, and content that's actually current when you set that date, matters more to answer engines than it ever did to classic search ranking. A page that says it was reviewed last month reads as more trustworthy to a retrieval system biased toward recency than one with no date at all, even if the underlying content hasn't changed much.

6. Make sure content renders without JavaScript. Several AI crawlers fetch raw HTML and don't execute client-side JavaScript. If your main content only appears after a script runs, hydrates behind a loading spinner, or is injected by a client-side framework after the initial payload, it's invisible to those crawlers regardless of how good the writing is. Server-side rendering or static generation isn't a GEO-specific recommendation, it's table stakes that a lot of modern JavaScript-heavy sites quietly fail.

7. If you're behind bot protection, use signed requests instead of blanket blocks. Cloudflare and Shopify both support Web Bot Auth, a scheme that verifies a crawler's identity cryptographically (via HTTP Message Signatures, RFC 9421) instead of trusting a spoofable user-agent string. It lets you allow specific verified crawlers without opening the door to every bot that lies about its identity.

How to audit this instead of guessing

Every item above is something we check automatically. squirrelscan's agent experience rules cover AI crawler access, llms.txt presence and format, whether your main content survives without JavaScript, and whether an LLM can parse the page cleanly, alongside the 249+ other SEO, performance, and content rules that still matter because GEO didn't replace them.

curl -fsSL https://install.squirrelscan.com | sh
squirrel audit https://example.com --format llm

The --format llm flag renders the report as clean, agent-readable text instead of a dashboard, which is a small demonstration of the same principle this whole guide is arguing for: structure your output for the reader that's actually going to consume it, human or model.

If you'd rather point an agent at it directly than run the CLI yourself, the llms.txt validator checks that one file in seconds. For the surrounding pieces, we've broken out dedicated guides on AI crawler configuration, what it takes to get cited in ChatGPT specifically, and llms.txt in more depth. If you're evaluating the broader GEO tool landscape beyond a technical audit, see our honest roundup of GEO tools.

Run a full audit and see exactly where your site stands on all of this, not just the parts we covered above.