How accurate are privacy-focused analytics platforms? [30 days experiment]
I got interested in "privacy-focused" (or "privacy-first") analytics platforms a couple of months ago, but I had my doubts about how accurate they actually are.
So I run an experiment.
I signed up to 3 tools (Fathom, Simple Analytics & Metrical) installed their script on one of my websites and collected data for 30 days.
Here are the results 👉 https://manuel.friger.io/blog/privacy-analytics-experiment
Trending on Indie Hackers
Wow, amazing article! I can learn a lot from
I'm "also" building an Analytics (and A/B testing) platform. Mine isn't privacy focused (even tho it's not free & I don't sell the data). I need to use user pinning because of A/B testing.
https://splitbee.io
I'm launching quite soon!
I use fathom presently. I am also shocked it doesn't have domain bypass.
Would be Interesting to know if they think about implementing this
Late, but like it - if you're ever planning on updating. Take a look at https://www.offen.dev please. Thanks!
I'm very late to this so there's not much to add, just wanted to say it's really good @Manu66, and much appreciated.
Great summary @Manu66. I'm building a privacy-focused survey platform called https://blocksurvey.io and I was looking for alternatives this week. Eventually, I found ticksel.com and played with it today. I'm interested in Metrical. Have registered for the beta access. Looking forward to it.
great stuff!
Very cool test and interesting finds! Wondered why you didn’t consider Matomo, which has been around for a long time.
I didn't know about Matomo!
It was previously called Piwik.
I also noticed this when using my own self-hosted analytics platform. Google Analaytics is so popular and has been around for so long that it is not only blocked by many ad-blockers, but also abused. I keep seeing referrer URLs in my dashboard that are just spam, just like ads showing up in my Google Analytics interface. One advantage of a self-hosted solution compared to a hosted analytics platform is that you can change the name/contents of the included scripts so they are no longer blocked by adblockers, but this is just a never-ending race of ad-blockers blocking analytics and analytics trying to not be blocked.
Do you find it strange that ad-blockers block analytics and other scripts that are not actually ads? A lot of users are not even aware of this. I think naming it "adblocker" was a bad decision or it just evolved in something more than it should have.
not very strange. ad-blockers's goal is to block anything that can track user activity
Yes, this is their goal now, but in that case the name "ad-blocker" is misleading. Most users think it only blocks ads from showing, not analytics or anything else not related to ads or something even breaking legit site functionality.
@XCS naming those things "ads" is a misnomer since they're mostly tracking tools.
Interesting, I was running Fathom and Google Analytics in parallel for a few month and the unique visitor count and pageviews were pretty close (< 10% different).
yeah I only run the experiment for 30 days. Maybe in the long term it levels out.
I noticed a few of them get blocked by adblock which is weird.
We just added analytics to our website builder and collect 25%-40% more data than google analytics.
We also got adblocked quickly but we were able to find a solution due to hosting the both the site and analytics.
Ah for some reason I thought you started the test today. Nice write up, learnt a lot :)
"SaaS website or simply a large website that requires accurate data, custom segmentation and advanced event tracking"
What features specifically?
glad you like the article!
features: custom reports, funnels, landing pages, per-page bounce rate, etc, etc, etc, etc
Thanks, custom reports and per-page bounce rates are very interesting.