Generative engine optimization checker: test your whole site, not one URL

A generative engine optimization checker tests whether answer engines can reach, read and quote your site. Most free ones grade a single page. This one crawls the whole site, and you can run it now:

curl -fsSL https://install.squirrelscan.com | bash
squirrel audit https://example.com --rule-include ax

It's free when you run it signed out. At the default quick level it reads up to 25 pages from your sitemap; add --level full for up to 500, or --max-pages for up to 10,000. If you'd rather stay in a browser, the website audit tool runs a full audit in the cloud; that path needs a free account and spends the free plan's monthly credits.

Six free GEO checkers, run on one site

On 23 October 2026 we ran every free GEO checker we could reach without an account against one site: squirrelscan.com, our own. Six ran. Six more asked for an email, a registration or a CAPTCHA, or led into a sign-up flow, before showing a result, so we left those (LLM Pulse, Pixelmojo, Otterly.ai's crawler simulation, Indexly, sden.ai and Scrunch's free audit).

Checked 23 October 2026. Scores ran from 55 out of 100 (LightSite) to 99 (Frase) for the same homepage, which tells you more about the rubrics than about the site. They didn't agree on basic facts either: nslookup.io and Goodie found no Organization schema, GEO Ready found it, and Outrun found it but said the logo was missing.

Free checkers are no longer all one-page tools: Goodie sampled 10 inner pages and LightSite analysed 26, while the other four stopped at the homepage or a single URL. None of the six rendered JavaScript to compare the result with the raw HTML. Each counted the text in the raw response instead, which is a reasonable proxy and not much more. Only Goodie named the bots our edge turns away.

The block that robots.txt does not show

Our robots.txt allows every crawler: User-Agent: *, Allow: /, with only /reports/ disallowed. Ask the server, though, and the answer depends on who is asking. Re-checked with curl on 23 October 2026, squirrelscan.com returned:

So the declared policy and the server disagree. A checker that reads only robots.txt reports GPTBot, ClaudeBot and CCBot as allowed, and four of the six did exactly that. squirrel audit flagged it through ax/agent-blocking: "GPTBot is blocked (HTTP 403) while the browser gets 200". It did not report ClaudeBot or CCBot, because that rule only probes as GPTBot and Claude-User. Goodie's 22-agent probe caught more than our rule did. Our free AI crawler checker now runs that comparison for 22 AI user agents, next to the robots.txt verdict for each.

One caveat on method. These requests carried each bot's user-agent string from an ordinary connection. Bot-management products can also verify crawlers by IP address, so the vendors' own crawlers may get a different answer, and we have not tested that.

Where one-page checkers stop

The homepage is the page most likely to be fine. Marketing sites are hand-tuned at the root and templated everywhere else, so the rendering problem that matters turns up on the product or docs template, and a homepage check never loads one. Sampling 10 or 26 pages helps; a crawl of the sitemap helps more.

Rendering can't be tested by reading HTML alone. Establishing whether content exists for a crawler that skips JavaScript means comparing the raw response with the rendered DOM. Counting words in the raw HTML gets you halfway.

And a score answers the wrong question between deploys. What you want to know on Tuesday is whether the site still clears the bar you set on Monday, which needs a check that runs unattended and fails the build when it doesn't.

The checklist a full GEO check runs

Most of these rules are in the agent experience (AX) category, which --rule-include ax scopes a run to; the AX rules overview lists them all. The ai/* rules also count as AX and run only when you are signed in. The schema and content rules at the end belong to their own categories, so run a full audit to include them. Each rule below runs on a free signed-out audit unless it says otherwise.

Crawler access, per named bot

ax/ai-crawlers classifies AI crawlers into four groups and reports which ones robots.txt blocks at the site root, meaning a Disallow: / that no Allow: / re-permits. The group matters more than the count. Training crawlers such as GPTBot and ClaudeBot collect pages for a vendor's future models, and blocking them costs you no citations in today's answers. AI-search indexers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot decide whether you show up in AI answers with citations. User-action fetchers such as ChatGPT-User, Claude-User and Perplexity-User serve a live request a person made inside an assistant. Archive crawlers, CCBot and the Internet Archive's bots, feed the shared corpora. Blocking a search indexer or a user-action fetcher is rarely what anyone meant. It tends to happen when someone assumes a vendor ships one bot rather than three.

ax/agent-blocking covers what robots.txt cannot show. It requests the homepage as GPTBot and as Claude-User and compares each response with a browser request, so a firewall rule that returns 403 or a challenge page surfaces here. It probes the homepage only, and only those two user agents.

ax/pay-per-crawl uses the same homepage probe and reports monetized access, telling Cloudflare's Pay Per Crawl headers apart from an x402 JSON payment body. Charging bulk crawlers is a business decision. Charging a user-action fetcher bills a person who is waiting for an answer.

The raw HTML an agent receives

ax/content-without-js compares the raw HTML with a separately rendered DOM and flags main content that only appears after JavaScript runs. The render happens in the cloud, so this rule needs a signed-in run and skips on a free local audit.

ax/token-weight estimates the token cost of the page HTML and the share of it that is visible text, and warns below 15 percent. In our 23 October run on squirrelscan.com it flagged five pages, the homepage at about 6 percent. A page like that spends most of an engine's token budget on wrapper divs, inline scripts and utility classes.

A map an agent can follow

ax/llms-txt checks for /llms.txt and /llms-full.txt at the domain root and validates the basic Markdown format. ax/markdown-response checks whether the site honours Accept: text/markdown or publishes a .md variant, which removes the markup cost instead of trimming it. ax/agents-md looks for an AGENTS.md, the instructions file a coding agent reads when it works on the site's repository. ax/well-known-agent detects MCP server cards, A2A agent cards and agent-skills manifests under .well-known, and flags a leftover ai-plugin.json from the ChatGPT plugins era. ax/api-discovery looks for an OpenAPI specification and OAuth discovery documents, which let an agent call your API without a person reading the docs first.

Those five are recommendations and don't affect your score. They report absence so you know about it.

A declared policy that agrees with itself

ax/content-signals parses Content-Signal directives in robots.txt, the per-purpose syntax from contentsignals.org that says search=yes, ai-input=yes, ai-train=no in one line instead of listing bots. It warns on invalid syntax and on a contradiction, such as ai-train=yes above a robots.txt that fully disallows GPTBot. A crawler reading both has to guess which one wins.

ax/rsl-license checks for a License: directive pointing at a valid Really Simple Licensing document. Having none passes. A reference that doesn't fetch, or doesn't parse as RSL, gets flagged so you can repair the pointer.

ax/noai-signals reports declared noai and noimageai opt-outs, plus nosnippet and max-snippet:0. Nothing enforces these. They record intent, and crawler blocks do the enforcing.

Presence in the archives the models were built from

ax/archive-indexing checks whether the site appears in the Wayback Machine and the Common Crawl index, the shared corpora many training sets draw on. Blocking CCBot or the Internet Archive takes you out of every dataset built from them, not just one vendor's. This check runs in the cloud, so it needs a signed-in run.

Signals that tell an engine what the page is

Crawler access gets an engine to the page. Structured data and clean prose decide how confidently it quotes what it finds.

schema/json-ld-valid validates JSON-LD blocks rather than detecting them, because an invalid block is worth the same as no block. schema/organization and schema/article check the two types that carry attribution, publisher and date. content/date-agreement compares the visible date, the schema datePublished and dateModified, and the Last-Modified header. A page that states three different dates gives nobody a reason to trust any of them.

Three more need a model in the loop, so they run on a signed-in audit only. ai/llm-parsability scores how cleanly a model extracts the page's claims. ai/page-type-match checks whether the page is the type it presents itself as. ai/site-metadata resolves the site's identity, type and audience for the site profile in the report.

Run it in CI

Put the check in the pipeline, so an AX score below your bar or any error-level finding fails the build instead of waiting for someone to notice:

name: GEO check
on: [pull_request]
jobs:
  ax:
    runs-on: ubuntu-latest
    steps:
      - run: |
          curl -fsSL https://install.squirrelscan.com | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
      - run: squirrel audit https://staging.example.com --rule-include ax --fail-on 'score:ax<80' --fail-on 'severity>=error'

--fail-on takes thresholds: an overall score, a category score such as score:ax, a count of errors or warnings, or a severity. When any of them trips, the command exits with code 2 and the job fails. Code 1 means the run itself broke, so a network failure doesn't pass for a bad score. It compares against the numbers you set, not against the last run; gating on new issues since a baseline isn't supported yet. Quote each expression, since the shell reads a bare < or > as a redirect. A runner with no API key is signed out, so this audit runs locally and costs nothing. The CI guide has the GitHub Action and GitLab versions.

Getting more out of a run

Two settings change what a run can see. Sign in, because ax/content-without-js, ax/archive-indexing and the three ai/* rules skip on a signed-out run; signed in, every audit spends credits. Then raise the page budget: --level full reads up to 500 pages and --max-pages up to 10,000, with cloud runs clamped to your plan's page ceiling. Page count is what gets you to the templates a homepage check never loads.

Leave --render off for this check. It forces the whole crawl through a browser, which makes the raw HTML and the rendered DOM the same document, so ax/content-without-js has nothing to compare and skips. Without it, the audit fetches raw HTML and renders separately, which is what the comparison needs.

--format llm writes the findings as text a coding agent can read and act on, which is where this gets useful: the agent that reads the report can open the robots.txt or the template and fix it in the same session.

What to fix first

1. Unblock the user-action fetchers. ChatGPT-User, Claude-User and Perplexity-User carry live requests from real people. Check robots.txt and your edge rules both, because ax/agent-blocking and ax/ai-crawlers can disagree, and the edge wins.
2. Get main content into the raw HTML. Nothing further down this list helps a page that renders empty without JavaScript.
3. Unblock the AI-search indexers if you want citations. That is a separate decision from training, with separate user agents.
4. Repair invalid structured data before adding more of it.
5. Cut the token weight on your content templates.
6. Add llms.txt and the other recommendations once the layers underneath work.

The recommendations come last because they describe content a crawler has to reach first. An llms.txt on a site that blocks PerplexityBot changes nothing about what PerplexityBot sees.

For the crawler layer, AI crawlers and robots.txt goes bot by bot, the robots.txt generator writes per-bot rules, and the free llms.txt validator checks the file without installing anything. What generative engine optimization is covers the content side. Once the technical side is clean, AI visibility trackers priced per prompt and the wider list of generative engine optimization tools cover measuring citations, which is the one thing a checker can't do.