2
5 Comments

I argued here this morning that a fuzzer can't find your worst bug. Now I'm charging £149 to prove it.

Earlier today I got into a proper disagreement in the comments of a CheCeno post about what AI coding assistants miss. Imris made a good argument: bad input lives in a bounded, enumerable space, so you can just fuzz it - a machine generates weird inputs faster than any human can type them. The unbounded question, "should this thing even exist", is the one that needs a mind.

I think that's half right, and the half that's wrong is the reason I'm building something.

Fuzzing solves generation. Generation was never the bottleneck. The bottleneck is the oracle - knowing that the output is wrong. A fuzzer types 1e400 into your price field, gets HTTP 200 back, and moves on happy. It has no idea the total now reads £Infinity, because nobody ever told it what the total was supposed to say. Crashes are self-labelling, so it finds those. Wrongness isn't, so it doesn't. And most of what actually reaches your users was never a crash - it's a screen rendering something absurd under a perfectly healthy status code.

I've been building and taking software apart since 1993, and that gap is the one thing I've never seen a tool close.

So I'm productising the other side of it instead of writing another tool. Launch Sprint: five human testers hammer your app for three days on real devices. Every bug arrives with numbered repro steps and a screen recording. I triage every report myself - reproduce it from the steps alone, merge duplicates, bin anything out of scope - so you get one ranked document instead of five piles of tester notes.

£149 for the first 10 teams, list price £249. Functional and input QA only: no security testing, and nothing starts without written authorisation from someone who owns the app.

Where this honestly is: the platform doesn't exist. I'm running the first sprints by hand precisely because I want to know whether anyone pays before I write software for it. The kill condition is written down - 3 paying customers or 25 signups in 14 days, or I change the offer once and then park it. Two days in, zero of both.

https://testmob.io/?src=ih

The question I actually want answered: what's the worst thing a real user found that you'd shipped without noticing?

I'll go first. A public API where ?limit=-1 returned the entire twenty-two-thousand-row table in a single unauthenticated request, because the clamp was min(value, cap) with no floor on it. Perfectly valid HTTP 200. Every fuzzer on earth would have shrugged and moved on.

on August 12, 2026
  1. 1

    That distinction between generating edge cases and knowing which ones actually matter is pretty interesting. Curious to see what you learn from the first few sprints.

    1. 1

      Cheers - and credit where it's due, your "a bug has ground truth, worth doesn't" line in the CheCeno thread is what made me sharpen this. It took someone stating the opposite cleanly for me to see where the seam actually was.

      No sprints run yet, so I've got nothing to report but intentions. What I can do is put the question back to you though: what's the worst thing a real user found in something you'd shipped?

      1. 1

        That’s a good question. I’ve definitely had cases where the issue wasn’t the bug itself but what it revealed about what users actually cared about. Happy to continue this outside the thread — what’s the best email to reach you at?

        1. 1

          hello@testmob.io - happy to keep going there.

          Though I'd still take the thread answer if you have one: not what the bug revealed, but the actual bug. The specific thing a user did that nobody on the team had thought to try. That's the bit I'm collecting.

          1. 1

            Thanks! I’ve just sent it over.

            Looking forward to hearing your thoughts whenever you have a chance.