4
8 Comments

I don't write code. I shipped an accounting SaaS. The worst bugs were the invisible ones.

I built EliteBooks over the past few weeks — free bookkeeping for freelancers, with a real double-entry ledger underneath. I direct an AI to write the code; I don't write it myself. That part gets attention, but it isn't the interesting part.

The interesting part is which bugs survived.

The test suite is 78 files and 1,838 assertions. It was green. The product was live. And a feature I'd shipped at launch — "this invoice number is a duplicate, save it anyway" — had never worked once.

The reason: in production, the framework strips the text out of thrown errors before it reaches the browser. Good security default. But it doesn't distinguish between "internal stack trace" and "a deliberate message the user needs to read." So the warning was replaced with a generic error, and the feature was dead. Tested, covered, dead.

Once I understood the shape of it, I went looking. About 150 server actions had the same pattern. I wrote a test that walks the codebase and fails the build if any action returns a refusal the client never checks for. That test — not me reading files — found eight real defects, including one where a permissions role broke an entire settings page.

Same week, a different flavour of the same problem: every rate limit in the app was inert in production. Behind a CDN and a shared proxy the client IP resolved to null, and null went into one shared bucket for the whole platform. Locally it worked perfectly. From the internet, eight wrong passwords in a row returned eight 401s and not a single 429.

And the one that stings most: the site had been live for weeks with no analytics installed at all. No GA4, no Plausible, not even a Search Console tag. Every visitor up to that point is unrecoverable.

The pattern in all three is the same. Not writing the code myself didn't cause any of them. They came from the gap between what the system does and what anyone can observe it doing. Tests assert what you thought to assert. Local behaviour isn't production behaviour. And something that fails silently looks identical to something that works.

What I'd do differently: verify from outside the system, not inside it. Hit production from the internet, not localhost. Check the database, not the console. Install measurement before writing a single word of marketing.

For anyone else building without hand-writing code — what's the failure mode that got you? I'm curious whether it's the same category or something else entirely.

posted toAvatar for product EliteBooks
EliteBooks
  1. 1

    Adding one that sits a rung below silent. It reported success.

    Our integration suite ran green for weeks while skipping all 47 tests. A path change meant the runner matched zero files, and a harness with nothing to run exits 0 and prints a pass. That test you wrote wouldn't have caught it either, because there was no action returning an unchecked refusal. There was no action.

    Yours failed quietly. That one claimed it had worked, which is the version I've had a much harder time building a check for. The only thing that's caught it since is reading the state back afterward instead of trusting what the call returned.

    When the walker found your eight, did it look at actions that returned OK, or only the refusal paths?

    1. 2

      Only the refusal paths — you've put your finger on exactly what it can't see.

      The walker starts from the refusal helper and fails the build when a server action can return one that no client ever reads. An action that returns OK having quietly done nothing walks straight past it.

      And I hit your version of it the same month, in the deploy. The migration container printed "migrations applied successfully" while running a stale image. Nothing had been applied. The build tool had built one service and not the other, and the exit code was 0 either way. What caught it was counting the rows in the migrations table afterwards — reading the state back, same as you.

      So my rule now is that the success message is not evidence for anything irreversible: check the store, not the return value. What I haven't managed is to turn that into a test instead of a habit. If your 47 skipped tests taught you a way to assert "this run actually did something", I'd genuinely take it.

      1. 1

        Three things worked, in increasing order of what they cost to build.

        The cheap one: assert on a count that had to move, never on the return value. Rows before, rows after, fail if equal. Exit 0 and a success string are both compatible with having done nothing, so neither can be the assertion.

        The one that actually fixed it: make the run open a ledger row before it does any work, and close it when it finishes. An entry still open when you read it is a failed run, not a quiet one. That is the difference between a habit and a test, because the check stops depending on someone remembering to look. Mine caught two missed runs this week that I would otherwise have read as quiet days.

        The one I wish I had built first: assert the input existed before you assert anything about the output. My 47 skipped tests were that wearing a disguise. A harness over an empty input set passes forever, and the pass is real, it is just meaningless.

        The rule underneath all three is yours, applied one step earlier. If the success message is not evidence, then neither is the absence of a failure.

  2. 1

    What stood out to me is that the hardest failures weren't incorrect behavior—they were invisible behavior.

    A system can be technically functioning while still hiding the information that tells you whether users are actually experiencing what you intended. That's a very different kind of reliability problem.

    1. 1

      That's a sharper way of putting it than I managed. Incorrect behaviour has a shape you can write a test against. Invisible behaviour has none — it looks identical to working.

      None of the three were found by reasoning about the code. They were found from outside it: hitting production from the internet instead of from my own machine, counting rows in the database, reading what actually reached the browser. Which is why the analytics one still bothers me most. It wasn't a bug at all. It was just a thing nobody was watching.

      1. 1

        That’s helpful context. The distinction between visible failures and invisible failures is an interesting one.

        I’d like to continue the conversation outside the thread. What’s the best email to reach you on?

  3. 1

    Free bookkeeping’ is a powerful hook. What’s the long-term play — monetising through services, or turning this into a daily financial hub for freelancers?

    1. 1

      Neither services nor a data play — the split is on cost, not on leverage.

      Invoicing, quotes, expenses and the full double-entry ledger are free permanently, because they cost me nothing per user to run. The paid plan covers only what bills me every month: bank feed connections, AI receipt scanning, automation. So the free tier isn't a funnel with a timer on it — it's the part of the product that happens to be cheap to serve.

      Long-term it's closer to your second option, the place the money actually passes through. But I'd rather earn that by being the thing they already open in April than by bolting on a services layer I can't staff. I'm one person.

      The honest risk in that model: if free is genuinely enough for most people, conversion is thin and I need volume. I'd rather find that out from real numbers than build a fake limit to hide it.