v0.0.110

A small release with two fixes worth having. A site that answers with compression bombs no longer eats gigabytes of your memory while squirrel crawls and checks it: squirrel now decodes compressed responses itself, as it reads them, and stops at each read's size limit. And a Quick audit stops telling you your links are fine when it never checked them: those checks now say Not checked, with what to run instead.

Behaviour changes:

• links/broken-external-links and links/broken-links report Not checked instead of a pass when they did not run. A Not checked result is left out of the score, and the JSON report gains a notChecked list when there is one. A Quick audit's Links score can drop a few points where the empty passes used to pad it.
• squirrel now decodes compressed pages and the other responses listed below itself (sitemaps already had their own reader). A response sent as x-gzip or an upper-case GZIP is now decoded, where it used to be read as raw bytes. A gzip body made of several members is read to the end of its first member, as browsers read it, where the previous release read every member. A page body that does not decode is still recorded as a decode failure, and the message now gives the decoder's reason.

Fixed

• A compressed page, root file or checked resource that inflates to gigabytes no longer exhausts memory. The runtime could inflate a Content-Encoding body in full before squirrel read it, even when squirrel read it under its size limit, cancelled it unread or discarded it with a redirect. In our test, an audit of a four-page site whose page, redirect, robots.txt and llms.txt were each a 1 GiB bomb (sent as about 1 MB of compressed data) peaked at 1.8 to 2.8 GB of memory on the previous releases.
  • These requests now turn the runtime's decoding off, and squirrel decodes the body as it reads it, stopping at that read's limit: pages and redirects, robots.txt, llms.txt, the well-known and agent files, RSL licences, Markdown versions of pages, the reachability check, scripts, external links, and the soft-404 and cloaking re-checks. An asset's size is counted as its body streams by, without holding it. Sitemaps already turned the runtime's decoding off and capped what they read.
  • The same audit, run with this release's build, now peaks at about 275 MB, against about 150 MB for the same site without the bombs, and reports the same findings. The bomb page is cut at 10 MB, as before, which is where an uncompressed page over the limit is cut too.
  • These requests send the same Accept-Encoding as before, squirrel's decoder handles gzip, deflate, br and zstd, and bytes a server appends after a complete compressed body are still ignored. The decoding changes are listed under Behaviour changes above.
  • The CLI's script check now stops reading one byte past its size limit instead of downloading the whole script first, and an external link's 403 page is read up to 1 MB for bot-wall detection.
  • Per-request figures are in benchmarks/2026-10-content-encoding-bombs.md. See Compressed responses.
• A Quick audit no longer says your links are fine when it never checked them. Quick, --offline and [external_links] enabled = false do not request external links. Quick also does not follow internal links, unless the sitemaps give it nothing to crawl. Yet links/broken-external-links passed with "No external links found to check", and links/broken-links passed with "No broken internal links detected" when it had seen only the pages the crawl fetched.
  • links/broken-external-links now reports Not checked when external links were not checked. links/broken-links reports Not checked when the audit did not follow links and left internal link targets unfetched. Each comes with the reason and what to run instead.
  • The result appears in the console, text, Markdown and HTML reports, as <not-checked> in the LLM and XML reports, and as notChecked in the JSON report.
  • Not checked results are not scored. A broken target a Quick crawl does fetch, such as a 404 listed in the sitemap, is still reported. A Quick audit that fetched every internal link target and found none broken still passes links/broken-links.
  • See Which audits can find a broken link.