v0.0.151

Technology detection finds Sentry, React, PostHog, Stripe and GA4 past the start of a large bundle, and a Next.js App Router page no longer detects a vendor it only mentions. A start URL at the site root that redirects to a deep page keeps the root-start crawl order. Thanks to William for the tests that pin the [[ignore]] timeout notice in a cut run (#1434).

Fixed

• A Next.js App Router page no longer detects a vendor it only mentions. The flight script (self.__next_f.push) was read as one block of text so the layout's next/script loader was found, which also matched a vendor URL that the page renders as content: a JSON-LD sameAs, a meta description, a CSP <meta>, a link or prose. The chunks of all the page's flight scripts are now decoded together and only the src, inline __html and string children of a script element, or of a next/script component (recognised by its props, since a production build does not name the module), are read; the layout loader is still detected on Next 14 and 15 production builds (afterInteractive, lazyOnload, in the head or the body). A flight script with no decodable chunk is read as an ordinary script. (#1437)
• A start URL at the site root that redirects to a deep page keeps the root-start crawl order. The crawler put the redirect destination ahead of the sitemap whenever the typed URL or its destination had a path, so https://example.com/ redirecting to /en/ changed which entry owns the page and, in quick mode with a sitemap that listed only the destination, switched link discovery back on. The order now follows the URL you typed. A start URL that is also the only sitemap URL still counts as a pending sitemap URL. (#1405)
• Technology detection finds Sentry, React, PostHog, Stripe and GA4 past the start of a large bundle. A bundle bigger than its byte cap is read as its first 16 KB, windows of text around the literals the detectors look for, and the end of the file, in 24 KB. Only a detector with a hand-written literal got a window: the rest (Sentry.init(, the React devtools hook, posthog.init, Stripe(, GA4's gtag() could only match in what the windows left of the budget, which was nothing in a bundle with many hits for the others. A detector without a literal now gets one derived from its pattern (the most selective text every match must contain; a few detectors, such as adobe-analytics, have none and still depend on the end of the file), the windows are shared out one round at a time with the hand-written literals first, and the end of the file keeps up to 6 KB (at most a quarter of the budget). A window counts for a detector that needs code only when its match starts in code, so mentions in strings and comments no longer use up the windows kept for the real call, a single dotted capital I (U+0130, in a Turkish locale table) no longer switches off the case-insensitive literals, and when the rest of a small script fits the room it is sent whole, with a loader the string tracker misread as part of a long string no longer dropped. (#1420)