v0.0.161

A reliability release: a page with a huge class attribute no longer fails the audit, security/leaked-secrets keeps its memory and its findings bounded on crafted pages and sites, and NAT64, 6to4 and other special-use IPv6 addresses are treated as non-public where policy is decided.

Fixed

• NAT64, 6to4, Teredo and the other special-use addresses are treated as non-public where policy is decided. The shared host classifier named loopback, private, link-local, CGNAT and ULA ranges only, so a start URL such as http://[64:ff9b::a00:1]/ (NAT64 for 10.0.0.1) or http://[2002:a00:1::]/ (6to4) was handed to the cloud, billed for a render, published and given the production delivery rules as if it were public. Cloud handoff, the render charge, publish and register gating, the MCP billing note, the skipOnPrivateTarget rule gating, a smart audit's not-applicable list and the default scheme now use a wider test that also names NAT64, 6to4, Teredo, ISATAP, site-local, multicast and documentation IPv6 literals and the IPv4 documentation, benchmarking, multicast and reserved ranges, whatever IPv4 they embed. The egress guards and the private-network allowance of a start URL's redirects are unchanged. A fixture that wants a public literal must use a real public address: 203.0.113.x and the other documentation ranges are no longer public here. (#1560)
• One page with a huge class attribute no longer fails the whole audit, and the cut cannot be steered by crafted markup. A class attribute of about 640,000 tokens (a little over a megabyte, far under the page cap; duplicates count) made the HTML parse throw RangeError: Maximum call stack size exceeded in linkedom's className setter, and the audit failed with no report. The parse is contained where pages are parsed: the page is read again with that attribute cut to its first 20,000 tokens (everything after it reaches the rules), a page that still cannot be read, or that fails with any other error, is audited empty, and either way crawl/html-size warns html-parse on it, on every audit path (the CLI's own page reader left the warning out). Where to cut comes from htmlparser2's tokenizer, the one the parse uses, run with the parse's options: a first version read the markup with its own scanner, so comments, bogus comments, CDATA, raw-text elements, whitespace written as entities, duplicate class attributes and the like made it cut the wrong attribute, lose elements or miss the huge one. A page that parses the first time is untouched; a class cut to 20,000 tokens loses class-based matches past the cut. (#1557, #1569, #1570, #1571)
• A class value keeps NBSP, U+2003, U+FEFF and vertical tab inside a token. The HTML parser's class list split on every JavaScript whitespace character and rejoined with a space, so class="cover x" read back as two classes where a browser keeps one. A class is now split on tab, line feed, form feed, carriage return and space only. This covers the class list of HTML elements only: an <svg> element's class is still split on every JavaScript whitespace character, CSS selectors (.a, [class~=a]) still match across NBSP, and rules that split a class value themselves on \s+ are unchanged. (#1567)
• security/leaked-secrets no longer fails, or balloons, on a page full of secret-shaped values. A page kept every finding it made, and the rule merged them with a spread (push(...findings)), which throws RangeError: Maximum call stack size exceeded past a few hundred thousand: one generated page took the whole site's result with it. A 10 MB page of key shapes also kept 245 to 277 MB of findings, made a rule result of 107 MB and peaked at 1.25 to 1.41 GB. A page or external script now keeps at most 1,000 distinct values (the report's own per-check item cap; copies of one value are merged the way the rule merges them before they count), critical first, then high, medium and public or informational, each in scan order. A check lists at most 1,000 items and records the rest in details.additional. What was left out is counted: a leaked-secrets-omitted warning gives the number and lists the pages that lost findings. The merge is a loop. The same 10 MB page now keeps well under 1 MB and gives a result under 200 KB. A site with fewer findings than the caps is unchanged. (#1554)
• security/leaked-secrets keeps the scan's own bookkeeping bounded on a crafted page. A page repeating one value, flipping a value between tiers, or listing hundreds of thousands of distinct or public values left residue in the per-page sink (dead entries, sets of left-out values) that grew with the page instead of with the cap. The sink now compacts what it replaces and bounds what it remembers, and the findings of scanPageForSecrets hold copies of their strings again, as the collector's do. (#1554)
• security/leaked-secrets keeps what it found when a page's scan runs past the rule's time budget. A page whose scan overran the 2 s budget used to keep no findings at all, so a critical one found before the stop was lost and the page showed only "not scanned". The findings made before the stop are now kept, and the page is still listed in leaked-secrets-unscanned (now "not fully scanned") because the rest of it was not read. (#1556)
• security/leaked-secrets keeps a bounded total across a whole site, across pages. Many pages that each stayed under the per-page cap could still add up to an unbounded total. A site now keeps at most 10,000 distinct findings (public and informational ones can use 40% of that, medium 60%, high 80%, critical all of it, so low findings never take the room a critical one needs); a value already kept from another page costs no room, and the rest are counted in the leaked-secrets-omitted warning. (#1568, #1554) The cap is one cap for pages and external scripts, a later copy that is more serious than the one kept is never dropped, the omitted count is "at least" that many (a value left out on several pages counts on each), and leaked-secrets-omitted is info, not a warning, when only public or informational findings were left out.