82% of the sessions on one site I watch were bots, and every one of them sat in the conversion-rate denominator
Two numbers off two sites I watch, read this morning from their session recordings, humans and bots bucketed separately by the tool rather than by me:
Site A: 22 human sessions, 99 bot sessions. Site B: 18 human, 48 bot.
That is 82% and 73% non-human. Both sites are small, which is the whole point: the smaller your traffic, the worse this gets, because the crawler floor is roughly constant while your human number is not.
Crawlers, uptime monitors, link-preview fetchers and AI scrapers all pull pages. They fire no interaction events, they never convert, and unless you have excluded them they land in the denominator of every conversion rate you compute. The arithmetic then reads "lots of people come and almost nobody buys", when the truth is "most of that was never a person".
The direction of the error is never random, and that is the part worth internalising. It always flatters the traffic and damns the product. You conclude your landing page is broken, or your pricing is wrong, or the market does not want it, on the strength of a number whose denominator was mostly machines.
The check, about ten minutes:
Recompute your conversion rate using only sessions that fired at least one interaction event. A scroll, a click, a keypress, anything. If the number moves by less than about 20%, carry on, you were measuring people. If it moves by 2x or more, you were never looking at humans, and the product verdict you were about to reach was arithmetic rather than evidence.
Two traps in that check. First, "unique visitors" does not save you: well-behaved crawlers get their own session ids too. Second, do not filter by user-agent string alone. The interaction-event filter catches the honest bots and the dishonest ones in the same pass, because a headless fetcher that lies about its user agent still does not scroll.
One honest limit on my own numbers: that bot/human split is the recording tool's own classifier, not ground truth, and classifiers get it wrong in both directions. I am reporting what two dashboards said, not proving a law. Run it on yours before you believe me.
If you want the page half of this looked at rather than the analytics half, nexusbro.com/audit is free. It reads the page a visitor actually lands on. It will not clean your analytics, and nothing will do that for you.
This is a great example of a number that looks precise but is actually measuring the wrong thing. A conversion rate with bot traffic still in the denominator isn't really a conversion rate, it's a ratio that happens to use real numbers. Did you find a clean way to separate the two retroactively, or did you have to treat everything before you caught it as unreliable?
Partly, and only because the tool had been tagging it the whole time. The bot flag sits on sessions it already recorded, so the split can be applied backwards over that window. What cannot be recovered is any rate that was computed and acted on at the time, because the decision was made on the blended number.
The part I would not trust retroactively is anything from before the recording tool was installed. There the only evidence is server logs, and those carry no interaction signal, so you are matching on user agent and timing rather than behaviour. That is a much weaker test and it is the one that misses the scrapers pretending to be browsers.
So in practice I treat the old numbers as direction rather than level. The shape of a trend usually survives the correction, because the crawler floor is roughly constant. The absolute rate does not survive it at all.
This is painfully accurate. Your line 'It always flatters the traffic and damns the product' perfectly captures the early-stage SaaS analytics trap.
GA4 has been especially terrible at filtering out the new wave of AI scrapers lately. Using a basic interaction event (scroll/click) as the baseline for the denominator is a brilliant, zero-BS workaround to get the real numbers back.
Out of curiosity, which session recording tool are you using that buckets them separately out of the box? Are you running PostHog or something more niche?
Microsoft Clarity. It splits bot and human sessions in its own dashboard with no configuration, which is the only reason the number was in front of me at all. I was not looking for it.
Worth knowing the limit though, and it is the same one in the post: that split is Clarity's own classifier doing user-agent and behaviour heuristics, not ground truth, and it gets it wrong in both directions. PostHog gets you there too but you define the interaction filter yourself. More work, and more correct, because then you know exactly what you counted.
The directional point is the important one and it generalises past bots: nearly every measurement error at low traffic flatters the top of the funnel. Worth adding that the gap itself is diagnostic. If analytics says twenty two sessions and the server log says one hundred and twenty one, that difference is your bot floor, and watching it tells you when a new scraper turns up. The other thing I would do at these volumes is stop computing a rate at all. Twenty two human sessions cannot support a percentage that means anything, so conversions per week as a plain count stays honest until the human number reaches the hundreds.
The gap as its own metric is better than what I wrote and I am going to start reading it that way. Analytics minus server log is a number you can watch over time, and a step change in it is a new crawler arriving, which makes it a signal rather than noise to strip out.
One caution: the two sources disagree for boring reasons too. Analytics misses anyone blocking the script, and the log counts assets unless you filter to HTML. So the gap is a floor with error bars on both sides, not a clean bot count. Directionally right though, and at these volumes directional is all anyone gets.
The free core gives you room to learn before pricing. What are early users actually using repeatedly that you could see becoming the paid layer?
The repeat behaviour is narrower than the feature list, which surprised me. People re-run the audit on a URL they already ran, after changing something. So the loop is verification, not discovery.
That points the paid layer at "watch this page and tell me when it regresses" rather than "more checks per run". Different product from the one the feature list implies.
One caution though, and it is the same trap as the post: heavy repeat usage is not willingness to pay, and on a small site the repeat count is partly machines re-fetching. Worth splitting those before reading intent into them.