v0.0.114

A pricing correction, and a release about trusting your link results. Version 0.0.113 said a signed-in audit's link checks were free. For the links the cloud checks, that was wrong, and this release puts it right: a signed-in audit now pays for its cloud link checks the same way a cloud audit does. Links squirrel checks on your machine stay free, and so do internal links. Beyond pricing, a link that meets a sign-in or bot wall is listed as unverifiable instead of broken, broken internal links say exactly what each audit covered, an audit of a site on your own network keeps what it finds there on your machine, and squirrel analyze fetches what its rules read the same guarded way squirrel audit does.

Pricing

• A signed-in audit pays for its cloud link checks: 1 credit for each distinct external link the cloud's link checker answers, an unverifiable answer included, at most 1 per page of the audit's page limit, and only at the levels that send external links to the cloud (surface and full, or a custom level with cloud checks and external links on). Version 0.0.113 said these checks were free; they are not.
  • Still free: a link the cloud can't answer in time, which squirrel checks on your machine; internal links; and every link in a signed-out audit. A quick audit with --external-links checks its links on your machine and pays nothing for them.
  • Each page of the audit's page limit now reserves its link check, so the same max_credits_per_audit covers fewer pages, and starting a signed-in surface or full audit needs 1 more credit than before. The pre-audit estimate and the page cap fitted to your balance include the link checks. squirrel credits and the low-balance messages state the price, and the first signed-in audit that checks external links after you update prints a one-time notice. Skip the checks with --no-external-links.
  • An audit that can't register a run, such as one signed in with an organization API key, checks its links on your machine and pays nothing for them.
  • Older versions keep free cloud link checks until 2026-11-12. From then, they check their links on your machine until you run squirrel self update.
  • See Credits and pricing.

Security

• An audit of a site on a private network (localhost, a private address, a .local name) no longer reads or writes the external link cache that every audit on your machine shares. A later audit of a public site could otherwise be answered from that cache about a public name that resolves to a private address. Results cached by earlier versions are not reused, so each link is checked again once after you update.
• No audit sends the cloud's link checker a link whose URL names a private or local host (a private IP address, localhost, a .local or single-label name). squirrel checks those itself. When the cloud declines a link because it, or a redirect it leads to, is on a private address, that is never reported as a broken link.
• squirrel analyze fetches the assets its rules read the way squirrel audit does. Its own copy of that step followed a script's redirects without checking where they led, so a script that redirected to a cloud metadata address led it to request that address. Every asset and link check, and every redirect those follow, is now held to the audit's rules. analyze also checks the internal links the crawl did not fetch, uses the crawl's own scope and page limit rather than the squirrel.toml in your current directory, and sends none of that file's custom request headers. It now also fetches fonts, preloaded scripts and web app manifests, status-checks third-party scripts, stylesheets and fonts while reliability/broken-resources is on, and no longer caches scripts between runs. See analyze.

Changed

• A link checked from your own machine that meets a wall is listed as unverifiable, not broken. That covers a sign-in or bot wall (401, 403, 405, 406, or LinkedIn's 999), a challenge page served as a 503, and a site that never answers in time. Most of these used to be counted as broken, so a signed-out audit's broken link list was mostly links that work in a browser. They are now listed under waf-blocked-external-links in links/broken-external-links, and links/dead-links no longer counts them. A real 404 or 410, a plain 503, a domain that does not exist and a refused connection are still broken, and so are a storage bucket's 403 for a missing file, an HTTP sign-in prompt and a 405 that allows no GET. Walls and timeouts are no longer cached, so the next audit asks again, and one an earlier version cached as broken is checked again. A signed-out audit with unverifiable links says so and suggests squirrel auth login, since the cloud's link checker can usually get a real answer. See links/broken-external-links.
• links/broken-links says what it covered on every row, the failing one included: how many internal link targets the pages link, how many the internal link check got an answer for, how many nothing fetched, and why (past the internal_links.max_targets limit, skipped by design, not trusted, or no answer). Comparing two audits can then tell a link that was fixed from one the later audit did not check. squirrel report --diff leaves these counts out of each broken link, so a change in coverage alone no longer marks every broken link as changed. A link that only a rate-limited page carries is no longer requested, or counted as skipped or past the limit. See links/broken-links.
• Names under the reserved .test, .example and .invalid zones, and under .localdomain, .home, .onion and .alt, count as local hosts, like .local. An audit of a Laravel Valet or Herd site such as myapp.test is treated as an audit of your own machine: no publish, no cloud render, the delivery rules (such as HTTPS and caching) skipped, and its links to other local names checked from your machine. On a public site's audit, a link to one of these names is not requested and is listed as refused-external-links instead of broken. See Local audits.

Fixed

• links/broken-links keeps the crawl's answer for a linked page the crawl fetched but does not grade as a page. A PDF that loaded no longer counts as an unchecked target, so a site that links its own PDFs can read as fully checked, and one that answered 404 is still broken. A linked page that answered the crawl with rate limiting is listed in rate-limited-links, and one that answered with a bot-wall challenge in refused-links. See links/broken-links.
• integrity/orphan-page no longer reports the page the crawl started on as a hidden page. A site whose root redirects to a deeper page, such as /en/ or a sign-in page, had that landing page read as hidden, and as a likely injected page when it also carried a page-level signal. The opt-in cloaking probe no longer picks it either. See integrity/orphan-page.
• crawl/sitemap-valid reads a sitemap whose root uses a prefix bound to the sitemap namespace, such as the ns0:urlset that Python's ElementTree writes by default. It used to report an unknown sitemap format and read none of its URLs. A prefixed sitemap index reads too. See crawl/sitemap-valid.
• The sitemap date checks read every date as the period it names, the same way on every machine. A date with no time is the whole day, a datetime with no zone is read in UTC, and a date that is not a real date is left out instead of guessed at. crawl/sitemap-lastmod-drift no longer warns that a lastmod of 2024-09-10 is 253 days ahead of a page dated 2024, crawl/sitemap-lastmod-churn counts the day a lastmod was written in its own zone and says when most of a sitemap's lastmods can't be read, and the cloaking probe stops treating pages dated only with this year as old two weeks into the year. See crawl/sitemap-lastmod-drift.