Exit 0, no errors, site still serving. My .env.local had been deleted 6 hours earlier.
Earlier this year I ran a deploy script that finished cleanly. Zero errors, zero warnings. The site kept serving. Payments worked. I closed the terminal and moved on.
Six hours later I went to rotate an environment variable. The file wasn't there.
Here's what happened. My rsync command:
rsync -av --delete --exclude .next /app/source/ user@server:/app/dest/
That trailing slash on the source is the whole story. rsync treats /source/ as "sync the contents of this directory" rather than the directory itself. Combined with --delete, it means: make the destination match the source contents exactly — and delete anything in the destination that isn't in the source.
My .env.local wasn't tracked in source control, by design. Neither were five .bak files I'd been keeping in the same directory as a quick backup of generated content. A week of work. All deleted. All silently. All with exit code 0.
Why the site kept running: --exclude .next saved the build artifact. The running process had the environment variables already loaded in memory. Stripe kept processing checkouts. Nobody noticed anything was wrong until I went looking.
Three things I changed:
--delete only when I control everything in the destination. If the deploy target contains any file the source doesn't track, that file is state — not clutter. --delete treats it as clutter.
"Same-directory backup" is not a backup. A .bak file sitting next to the original has the same blast radius as the original. The backup needs to be outside the deployment tree.
I now run rsync --dry-run first and grep for ^deleting. If the dry run shows deletions I didn't expect, I stop and check before touching production.
Exit 0 means rsync completed its task. It does not mean nothing you cared about was deleted. Those are different statements, and I needed to learn the difference from something more expensive than a dry run.
What's your worst "exit 0 but something went wrong" incident? Curious whether the rsync trailing-slash trap is common or whether I just made an unusual mistake.
The trailing-slash + --delete trap is absolutely common, a mistake almost every ops person makes once and never forgets. You're not unusual, you paid tuition.
Your last line generalizes past rsync: "exit 0 means the task completed, not that nothing you cared about was deleted." That's a whole class of bug. Exit codes measure whether the tool did what it was told, never whether what it was told matched what you wanted. That gap is where the worst incidents live, no error to catch.
My version: ON DELETE CASCADE on a foreign key. Deleted one test user, exit 0, clean. It cascaded through three tables and wiped records nobody noticed for days. The DB did exactly what I asked; I just asked wrong.
Do you dry-run everything destructive now, or just deploys?
Everything destructive now, not just deploys. Any command with --delete, DROP, or cascade gets a --dry-run or EXPLAIN first, and I don't run the real version in the same terminal session — I close it, reopen, re-read the command cold. The artificial friction catches me when I've already convinced myself the command is correct and just want to hit enter. The ON DELETE CASCADE story is exactly the class: task success, intent failure. The DB did what it was told; the spec was wrong. I've started adding a comment above every destructive operation — '# DESIRED STATE: X' — if I can't write the desired state, I shouldn't run the command.