# Cache the miss paths, or an attacker will

An adaptive L7 attacker finds your uncacheable endpoints by timing edge
responses - cached answers are fast, origin answers are slow - and then
hammers the handful they find: unique-path 404s, 302s, search endpoints. A
small number of them is enough to take an origin down even when most of the
site is static. The technique: give error and redirect paths the same
edge-caching treatment as content (even a few-minute TTL is enough), move
redirect handling to the edge, and rate-limit on request cost rather than
source - count 404s and redirects against browser-fingerprint and ASN buckets
(the RTD team calls it the "penalty box", and their postmortem credits it with
the biggest automated effect) - because IP blocks are useless against a botnet
distributed across millions of addresses and hundreds of ASNs.

Reported in the Read the Docs postmortem of their June 2026 attack (author
David Fischer), and the HN discussion at item 49628614, both read 2026-09-10.
Numbers from the author: 5.5M requests/min at peak (about 100x baseline) for
about 10 days; a plain Nginx rewrite-regex redirect was overwhelmed to dropped
requests despite horizontal scaling; the botnet ramped up to discover
rate-limit windows then backed off to let them expire ("yo-yo"), apparently
optimizing for autoscaling cost rather than full outage.

## Where it fails

**Fingerprint-based limits have a shelf life.** The attack randomized TLS
parameters, so JA3/JA4 fingerprint rules were "not very helpful" - the author
still rates JA4 useful against proxy scraping generally ([49631717]), and is
challenged on that by a commenter noting TLS spoofing is cheap with tools
like cycletls; the author concedes "it will probably be useless one day. In
practice it is still useful today" ([49633486], [49633562]). Keep fingerprint
rules as one signal in a combination, never as the wall.

**The blunt instrument was deliberately not tested.** Several commenters
thought Cloudflare Under Attack Mode should have been flipped ([49629656],
[49631115]); the operator declined because a challenge-everyone interstitial
breaks API integrations and hundreds of thousands of real readers, and assumed
(not observed) it would have cut the attack "to borderline irrelevance"
([49631584]). A third-party operator reports Under-Attack-style measures only
partially mitigated much smaller attacks on their Business-plan setup
([49631369]). The economics of challenge walls - attacker must run real
browsers - is argued in [49634076] but unmeasured in this attack.

**Early confident diagnoses were wrong.** A top comment asserted this "is
probably an AI lab that misconfigured their data scraper" ([49630298]); the
author contradicted it with data - real scrapers pull content, this traffic
was almost entirely 404s/302s pulling near-zero docs and deliberately
targeting CDN-bypassing URL patterns ([49630503]). Another confident premise
("docs are static, you would need way more traffic") was contradicted by the
postmortem itself: the attack specifically bypassed caching ([49630527]). Who
was actually behind it remains unknown in the thread - the author's own guess
is that attackers struck while he was mid-rollout of tighter scraper rate
limits and saw the window closing ([49629625]).

**Offering a considerate path does not get taken.** On the scraping side:
RTD publish per-project archives and markdown exports, yet scrapers still
hammer the site; the same is reported at git.kernel.org where a single
`git clone` would do - bulk-archive availability does not steer automated
crawlers ([49632553], [49633086]).

## Source

HN item 49628614 (comments 49629625, 49630298, 49630503, 49630527, 49629656,
49631584, 49631717, 49633486, 49633562, 49631369, 49634076, 49632553,
49633086) and the RTD postmortem (about.readthedocs.com, Sep 2026), read
2026-09-10. Author-reported incident data plus forum discussion; nothing
independently verified here.