Last month I posted here with a real question. I had a broad, thin content site, about 110 pages, and I couldn't decide whether to gut it to 20 genuinely great pages or keep all of them and upgrade in place. Most of you said prune. I didn't. Here's the month-two data, because it turns out you were right and I was right, about two different things.
I kept the pages. For search, that was the correct call.
Google had indexed 4 of my pages in June. It's now indexed 115 of 129. The pages I nearly deleted are the pages now pulling impressions. If I'd gutted to 20, I'd have 20 indexed pages instead of 115, and no honest way of knowing in advance which 20 would have been the keepers. Keeping them and being patient worked on the indexing side.
Then the part that stung.
AdSense rejected the site for "low value content." Twice. The exact trait the indexer stopped caring about, thin and templated pages, is the exact trait the ad reviewer rejected me for. Indexing recovered. Monetization did not. The thin-pages problem didn't get solved by keeping the pages. It just moved from one gate to the next.
So why am I still not pruning to 20?
Because I ran the experiment instead of guessing. I rebuilt one page to be genuinely deep: worked example, citations, real FAQ, all of it. Over the next month it earned 8 impressions. My site averages about 8.6 per page. The deep version landed at dead average. My pages don't sit on page 4 because they're shallow. They sit on page 4 because my domain has no authority: DR 0, one real backlink, average position around 40. Pruning to 20 great pages doesn't fix a backlink problem. It just leaves me with fewer results buried on page 4.
The thing that genuinely surprised me
My best-performing traffic source right now is this post's predecessor. The June build-in-public post I wrote here is my #2 traffic source overall in GA4, behind only Google organic, at about 20 sessions in 30 days, up 900% (off a small base, but it's real). For contrast: six weeks of posting prompt images to Pinterest produced 1 session. One honest post to a community that cares beat a month and a half of image distribution by roughly 20x.
That reframed my entire month-two plan. I'm not fighting for crawl budget anymore, I have it now. I'm fighting for authority and revenue, and the channel actually paying out is showing up where people already are and being useful, not firing more pages into the void.
The plan
Stop resubmitting AdSense unchanged. A third identical submission earns a third "low value" reply. I'm going to try Ezoic instead, which is far more forgiving of new, low-traffic sites, and let AdSense sit until I have the traffic and links to come back with.
Keep writing in public here, and keep answering questions where my niche actually lives, because that's the measurable winner so far.
Honest numbers so this isn't a highlight reel: about 69 sessions in 30 days, and a real slice of those are datacenter bots rather than humans (the top "cities" in my analytics are literally AWS and Google server-farm towns). Revenue is still 0.
Two questions I'll throw back to you. For a broad-but-thin library like this, has anyone actually turned it into revenue via the Ezoic-then-grow route, or did the people who won prune hard first and I'm about to learn that the expensive way? And separately: has build-in-public ever converted into real revenue for you, or does it top out as a nice traffic-and-moral-support channel that never pays rent?
How do you currently handle your analytics and would you mind reading my indiehacker pots on how crawlers and bots are ruining analytics data?
https://www.indiehackers.com/post/i-just-discovered-my-analytics-numbers-are-mostly-fake-here-is-why-8197e3ff9d
Read it, and the GA4-versus-server-CPU moment is basically the one I had too, minus the CPU spike to tip me off. I caught mine the lazy way: GA4's top "cities" for my active users were Prineville, Boardman, and Council Bluffs, which aren't cities, they're the AWS and Google datacenter towns. Once you see that, you can't unsee it.
Honest answer to your question: right now I don't handle it well at all. GA4's default "exclude known bots" is on, but that only catches the IAB list and does nothing for AI agents or headless scrapers coming from datacenter IPs. I haven't cross-checked against raw logs the way you did, partly because I'm on Vercel's hobby tier and don't get clean access logs without extra plumbing. So I'm flying on numbers I already know are inflated.
Your "logs are ground truth" point is right, but I'd add that the real number sits between two wrong ones: logs overcount (every bot is in there) and GA undercounts humans (your 20–30% adblocker loss is real). Neither side is clean alone.
On the product: the hard part isn't flagging the obvious scrapers, it's the "verified human" bucket. Cloudflare and GA both swing at that and both miss. A clean three-way split from logs, especially for people who can't easily see their own logs, is a real gap. How are you planning to tell a legit headless fetch (a link preview, say) from an actual scraper?
Man, seeing Prineville and Council Bluffs pop up as top cities in GA4 is such a classic developer moment. You figure it out once and can never unsee it.
You totally nailed the core issue here. GA misses the humans running ad blockers but happily counts datacenter bots, while raw server logs just dump everything together into one giant mess. The actual human number is definitely trapped right in the middle.
Figuring out the clean split between humans, legit link previews, and actual scrapers is the hardest part. To tell a real Slack or Discord link preview apart from a malicious scraper, we look at a few different signals.
The first is request depth. A link preview bot just makes a single quick request to grab the meta tags and leaves within milliseconds. They do not load your CSS, images, or JS. Scrapers usually pull full asset trees or recursively crawl through your links.
We also use reverse DNS checks to verify that a bot claiming to be Twitterbot is actually coming from a verified Twitter IP range. If it says it is Googlebot but comes from a random Hetzner IP, it gets flagged immediately. On top of that, TLS fingerprinting catches the sneaky ones. Even if a Python script spoofs a Chrome user agent, the TLS handshake ciphers it uses are completely different from a real browser.
Your point about Vercel is huge by the way. Nobody wants to build custom log plumbing or mess with SSH just to see basic traffic truth. That is exactly why I am building a native Vercel log drain integration. It lets you stream clean traffic metrics without ever touching raw log files.
I really appreciate you taking the time to write this out. It completely validates the exact gap I am trying to bridge. If you are open to it, I would love to send you a link to test an early Vercel friendly build when it is ready.
The request-depth heuristic is clever, and reverse DNS is the one I'd trust most, since it's literally how Google tells you to verify Googlebot. TLS fingerprinting I hadn't thought to apply here, but you're right that a JA3/JA4 mismatch against a claimed Chrome UA is about as dead a giveaway as it gets. A spoofed Python client can fake the user agent all day and still hand you the wrong cipher order.
One honest wrinkle on the Vercel angle, since it's my exact situation: log drains are Pro-only. On Hobby you can't attach one at all. So the people most likely to be staring at inflated GA4 numbers, the small and side-project sites, are exactly the ones who can't reach a log drain in the first place. That feels like the real product question to me, whether there's an edge-middleware or reverse-proxy path that gets you the same ground truth below Pro.
And yes, send it over when it's ready. I'm the target user with the actual problem, on Hobby, watching datacenter towns show up as my top "cities." Happy to kick the tires and tell you where it breaks.