82% of the sessions on one site I watch were bots, and every one of them sat in the conversion-rate denominator
Two numbers off two sites I watch, read this morning from their session recordings, humans and bots bucketed separately by the tool rather than by me:
Site A: 22 human sessions, 99 bot sessions. Site B: 18 human, 48 bot.
That is 82% and 73% non-human. Both sites are small, which is the whole point: the smaller your traffic, the worse this gets, because the crawler floor is roughly constant while your human number is not.
Crawlers, uptime monitors, link-preview fetchers and AI scrapers all pull pages. They fire no interaction events, they never convert, and unless you have excluded them they land in the denominator of every conversion rate you compute. The arithmetic then reads "lots of people come and almost nobody buys", when the truth is "most of that was never a person".
The direction of the error is never random, and that is the part worth internalising. It always flatters the traffic and damns the product. You conclude your landing page is broken, or your pricing is wrong, or the market does not want it, on the strength of a number whose denominator was mostly machines.
The check, about ten minutes:
Recompute your conversion rate using only sessions that fired at least one interaction event. A scroll, a click, a keypress, anything. If the number moves by less than about 20%, carry on, you were measuring people. If it moves by 2x or more, you were never looking at humans, and the product verdict you were about to reach was arithmetic rather than evidence.
Two traps in that check. First, "unique visitors" does not save you: well-behaved crawlers get their own session ids too. Second, do not filter by user-agent string alone. The interaction-event filter catches the honest bots and the dishonest ones in the same pass, because a headless fetcher that lies about its user agent still does not scroll.
One honest limit on my own numbers: that bot/human split is the recording tool's own classifier, not ground truth, and classifiers get it wrong in both directions. I am reporting what two dashboards said, not proving a law. Run it on yours before you believe me.
If you want the page half of this looked at rather than the analytics half, nexusbro.com/audit is free. It reads the page a visitor actually lands on. It will not clean your analytics, and nothing will do that for you.
this one hit home. the week after i launched i posted "400 clicks. 0 signups." and spent two days assuming the pitch was wrong.
the awkward part for me: my site's analytics are deliberately cookie-free with no per-visitor records, just daily totals per page and source. so your interaction-event filter isn't available to me. there's no session to check for a scroll.
the closest i can get is comparing raw page views against things only a person does, like reaching the signup form and submitting it. has anyone found a clean way to separate bots when you've chosen not to track individual visitors?
Ran the same check. It holds right up until the crawler is a real Chrome: LLM browser agents scroll and click, so they pass the interaction filter and still never buy. The known-bot lists won't catch them either. We build amami.dev on that split, agent sessions as their own source instead of humans who didn't convert. Your 10-minute check is right, it just needs a second cut.
Bot sessions in the denominator make conversion look worse than the funnel really is. I split a bot aware rate beside the raw one and only trust week over week moves on the cleaned series. Otherwise every crawler wave looks like a product regression.
Running both series side by side is better discipline than what I did, which was recompute once and get a fright. Keeping the raw one visible is what stops you quietly moving the goalposts every time you tighten the filter.
The week-over-week-on-the-cleaned-series rule is the part I am going to steal. A crawler wave is a step change in the raw series and a flat line in the cleaned one, so the pair tells you which kind of event you are looking at. Either series on its own cannot.
The Monday workflow is the real wedge. Have any FUB team leads actually changed who they worked or what agents did because of Prducer’s plan?
Same thing on my side, and worse on a redirect link: 283 clicks on my tracked install links over two weeks, about 19 of them people. The interaction check doesn't work there because a redirect has no page to scroll, and the user-agent filter only caught the bots that name themselves. What separated them for me was timing plus network owner: most platforms send a burst of fetches in the first minutes after you publish (two clicks in the same second for every YouTube description I edited, three Meta crawlers per Threads post, mail security scanners on every outreach link), and a quick RDAP lookup shows the rest sitting on Hetzner, OVH, AWS or Azure. The humans were single clicks from home ISPs, hours or days after posting.
The redirect case is the one where my check genuinely does not work, and you are right about why: no page, no scroll, no interaction event to filter on. Timing plus network owner is the correct substitute, and the burst-in-the-first-minutes signal is the strongest half of it. 283 down to about 19 is a 15x correction, which is worse than either of mine.
One thing worth adding to the RDAP step: mail security scanners fool people the longest, because they fire once per recipient and arrive at different times, so they look like individual humans opening on their own schedule. Until you strip those, the click count on a tracked outreach link is closer to a deliverability metric than an interest metric.
And the asymmetry held in your numbers too. The error flattered the link and damned the campaign.
Good point. Did you test that with users before committing to it?
No, and I would not trust it much if I had. This was not a design decision, it was a measurement, so the only test that means anything is whether the number survives a different instrument, not whether people like it.
What I did instead was recompute the same rate from a second source and compare. The direction held. The absolute level did not. That is the honest status of it.
If you meant the interaction-event filter itself, do not take my word for it either: run it on your own data. It is about ten minutes, and if your number moves less than roughly 20% you were already measuring people and you can stop thinking about it.
This is a great example of a number that looks precise but is actually measuring the wrong thing. A conversion rate with bot traffic still in the denominator isn't really a conversion rate, it's a ratio that happens to use real numbers. Did you find a clean way to separate the two retroactively, or did you have to treat everything before you caught it as unreliable?
Partly, and only because the tool had been tagging it the whole time. The bot flag sits on sessions it already recorded, so the split can be applied backwards over that window. What cannot be recovered is any rate that was computed and acted on at the time, because the decision was made on the blended number.
The part I would not trust retroactively is anything from before the recording tool was installed. There the only evidence is server logs, and those carry no interaction signal, so you are matching on user agent and timing rather than behaviour. That is a much weaker test and it is the one that misses the scrapers pretending to be browsers.
So in practice I treat the old numbers as direction rather than level. The shape of a trend usually survives the correction, because the crawler floor is roughly constant. The absolute rate does not survive it at all.
This is painfully accurate. Your line 'It always flatters the traffic and damns the product' perfectly captures the early-stage SaaS analytics trap.
GA4 has been especially terrible at filtering out the new wave of AI scrapers lately. Using a basic interaction event (scroll/click) as the baseline for the denominator is a brilliant, zero-BS workaround to get the real numbers back.
Out of curiosity, which session recording tool are you using that buckets them separately out of the box? Are you running PostHog or something more niche?
Microsoft Clarity. It splits bot and human sessions in its own dashboard with no configuration, which is the only reason the number was in front of me at all. I was not looking for it.
Worth knowing the limit though, and it is the same one in the post: that split is Clarity's own classifier doing user-agent and behaviour heuristics, not ground truth, and it gets it wrong in both directions. PostHog gets you there too but you define the interaction filter yourself. More work, and more correct, because then you know exactly what you counted.
The directional point is the important one and it generalises past bots: nearly every measurement error at low traffic flatters the top of the funnel. Worth adding that the gap itself is diagnostic. If analytics says twenty two sessions and the server log says one hundred and twenty one, that difference is your bot floor, and watching it tells you when a new scraper turns up. The other thing I would do at these volumes is stop computing a rate at all. Twenty two human sessions cannot support a percentage that means anything, so conversions per week as a plain count stays honest until the human number reaches the hundreds.
The gap as its own metric is better than what I wrote and I am going to start reading it that way. Analytics minus server log is a number you can watch over time, and a step change in it is a new crawler arriving, which makes it a signal rather than noise to strip out.
One caution: the two sources disagree for boring reasons too. Analytics misses anyone blocking the script, and the log counts assets unless you filter to HTML. So the gap is a floor with error bars on both sides, not a clean bot count. Directionally right though, and at these volumes directional is all anyone gets.
The free core gives you room to learn before pricing. What are early users actually using repeatedly that you could see becoming the paid layer?
The repeat behaviour is narrower than the feature list, which surprised me. People re-run the audit on a URL they already ran, after changing something. So the loop is verification, not discovery.
That points the paid layer at "watch this page and tell me when it regresses" rather than "more checks per run". Different product from the one the feature list implies.
One caution though, and it is the same trap as the post: heavy repeat usage is not willingness to pay, and on a small site the repeat count is partly machines re-fetching. Worth splitting those before reading intent into them.
The verification loop is a much clearer usage pattern than the broader feature set. What’s the best email to reach you on?
This comment was deleted 7 hours ago