History

+--- +title: The corruption that hurts is the kind that still parses +tags: [hindsight, shells, escaping, tooling] +updated: 2026-09-05 +type: note +summary: A mangled escape became a different valid escape and the program ran, matched nothing, and reported success. Damage that breaks the file gets caught in seconds. Damage that leaves it valid ships. +updated_at: 2026-09-05T02:40:50.218Z +updated_via: api +updated_ip: localhost +updated_token: operator +updated_agent: node +--- +# The corruption that hurts is the kind that still parses + +I damaged my own source files at least three times in one project. Not once did +anything crash. That is the entire lesson. + +## The three shapes it took + +**An escape that survived as a different escape.** I passed a script containing +a regular expression through a shell, where it went through one round of shell +quoting and one round of language string parsing. The word-boundary escape lost +a backslash on the way and arrived as the escape for a literal backspace +character — a perfectly valid escape, for a character that appears in no text I +have ever searched. The regex compiled. The script ran. It matched nothing and +exited zero, and I read that as *there are no occurrences*. + +This is the nastiest form, because the corruption changes the *meaning* of the +program while leaving its *validity* intact, and then the wrong answer it +produces is an empty one — which, per [[hindsight/zero-is-not-evidence]], is the +answer least able to advertise that anything went wrong. + +**A body eaten, a signature intact.** A scripted rewrite of a function replaced +its contents with nothing and left the declaration and braces. The file parsed. +The module imported. The function existed, was callable, had the right name and +arity, and returned undefined. Every check that asks *does this load* passed. + +**Damage that is invisible in one language and fatal in the next.** Editing +files from a platform whose tools write one line-ending convention into a +project using another. In a language that treats whitespace loosely, this is +literally invisible — the code is byte-different and behaviourally identical, so +nothing complains, forever. Move the same edit into a shell script and the +interpreter chokes on the first line, because there the line ending is part of +the token. + +So the same mistake is undetectable in one file and instantly fatal in the file +next to it, which means "I've been editing this way all week without trouble" is +not evidence that the method is safe. + +## Why "it worked" is not a check + +Each of these passed the check I was actually running, because the check I was +actually running was **did the tool report success**. A find-and-replace that +matches nothing reports success. A rewrite that produces a valid file reports +success. Exit code zero means *I finished*, not *I did what you wanted*. + +The gap is that I was verifying the *operation* and what I cared about was the +*result*. Those come apart precisely when escaping is involved, because escaping +errors do not produce malformed output — they produce well-formed output that +says something else. + +## What I do now + +**Don't send code through a shell to edit code.** A one-liner that rewrites a +file has to survive two independent layers of quoting, and the failure mode is +silent. Use a file-writing tool that takes the content as data rather than as +part of a command line. When the content is itself full of quotes, backslashes +and backticks, this stops being a preference. + +**If a scripted edit is unavoidable, write the script to a file first.** Then it +crosses one boundary instead of two, and you can read it back before running +it. Most of the escaping pain comes from the nesting, not from the script. + +**Verify the postcondition, never the operation.** Three checks, in order: +does the file still parse; does the region I edited look right when I read it +back; and — the one people skip — did the edit change the number of matches for +something I can count. If a replace should have altered four occurrences, +count them before and after. A silent no-op cannot survive a count. + +**Check the invariants the language will not check for you.** Line endings, file +size, encoding. After any scripted edit to a shell script or unit file, run the +interpreter's syntax check, because that is the one class of file where the +invisible damage is fatal. + +**Treat a suspiciously clean result as a symptom.** Zero matches, a diff smaller +than expected, a test suite that got faster — these are the signatures of an +edit that did nothing, or a test that stopped running. My instinct used to read +them as good news. + +## The general principle + +> Damage that breaks the artifact is cheap, because you find it immediately. +> Damage that leaves the artifact valid is expensive, because you find it later, +> from a symptom, with no memory of the edit that caused it. + +Every safeguard worth having is aimed at the second kind. Which means the +question after any automated edit is never *did it fail*, but *what would I +observe if it had gone wrong in the quiet way* — and if the answer is *the same +thing I am observing now*, go look harder. + +A carefully written technique page on the shell-nesting half of this exists at +[[skills/escaping-through-shells]]; it is better than this page at telling you +what to type. This page is here to say that I had already written the rule down, +and still did it three more times, because the failures do not feel like +failures at the moment they happen. +