Share your experience of setting up product analytics
Hey, awesome Indie Hackers!
I’m consulting young startups on their first steps in product analytics. I help with: defining the best data structure and analytics strategy, creating clear and meaningful tracking plans, implementation and setting up dashboards that really answer the questions. In short, my goal is to empower product teams to conquer and evolve their analytics tracking.
As I strive to constantly improve my expertise and understanding in this area, I decided to conduct a small research to better understand different approaches and the most-common pain points here.
Would greatly appreciate if you could share your experience with me on the following topics:
- What tools do you use for your product analytics set up and tracking plans? What do you like and don’t like about them?
- Tell me about your experience of starting with product analytics tracking. Was it easy or difficult to set everything up and running? What problems have you faced (if any)?
Ideally, we could set up a short call, but can also discuss it here. Also feel free to catch up with me on Telegram @khkseniaa
When it comes to the tools, I use Heap.io for quantitative Product Analytics.
I use fullstory or other session recording tools for the semi-quantitative-semi-qualitative, and any survey platform for the fully qualitative.
I love Heap more than Mixpanel or Amplitude for two main reasons:
This makes data analysis much more of a self-served experience for my peers and colleagues, thus sharing the power of data.
As for the data structure -
I help advise on that too :) I find that people don't always understand the implications of structuring the data properly ahead of time.
The way I contribute to that is by supplying examples of questions that hard to answer given the wrong data structure.
I've heard a lot about Heap.io but actually have never worked with it on a real project. Does their no-code events capture work for mobile apps as well?
And I couldn't agree more on the fact that people don't think about data structure in advance. It's actually the reason of 90% of problems with self-serving product analytics. By the way, your approach with questions sounds interesting. Could you give an example of such questions you usually ask?
Re: mobile apps - I think it doesn't. I haven't had it implemented on mobile, only on web.
Re: questions - I use the onboarding sequence as an example.
*Answering the question "How many users, who identified themselves as Product Managers, did action X down the journey?" is going to be harder, had the data been stored as a user property. Harder, and maybe even imprecise. Depends on the system you use to query the data.
Having said that, in this case, it is still important to send the data as events (as well as user properties), in order to look at drop rates throughout the onboarding journey.
This is what I usually recommend in consulting, and also what we practice for our own products:
Our own tool, Query.me, is currently free for small teams and does the job well even though we are still at an early stage.
Hey Ksenia! A little to your post here, but I've been doing some product analytics consulting for 5-10 companies over the last few years. I've been using mostly Mixpanel and Amplitude but I've lately discovered Posthog which looks great as it can be self hosted. (I've also made an overview of all the analytics tools I've found at https://analytics.rip)
As for problems for settings things up, I guess it depends on your technical skills, the team around you and the culture for measuring, analyzing and improving in the startup you are consulting. My best advice is to get others to se the value of measuring and learning from the numbers and insights you collect. No matter how good you are, you have to get the rest of the team on board to some extent for the data to be valuable.
If you are able to achieve one successful case first (an analysis that can contribute to improve the product or set up an AB-test on a new feature) my expirience is that is't much easier to engage the rest of the team to prioritize and contribute to the development of product analytics later.
Thanks for referencing Posthog and for your overview, it's pretty helpful. I've never heard of it, studied their website - looks interesting, it has really wide functionality. Will definitely play with it!
Couldn't agree more with what you are saying. Culture of analyzing data is so important among the teams so they could feel themselves how powerful and valuable it is. From my experience setting things right from the beginning plays big role in that. From what I've seen, a common problem was that a team was tracking something, but their tracking was a mess. Like they had a lot of different data, events they collected but had little idea of what exactly it meant and what exactly they were tracking and everything was very unstructured and hard to maintain. As a result, no one could understand how to get the data they needed. And I noticed that it really demotivated teams from implementing, maintaining and using the analytics because they just couldn't get that value from it. But when we get everything organized and clear, they become engaged with analytics immediately.
I've tried a bunch of analytics tools but realise a few years ago I would be better serve by storing raw data on a relational database and have my own queries to monitor the things I'm interested in. 3 years ago, I made a shitty PHP script that load itself in the website as an image (or a pixel as facebook calls it) and record the referrer, language and other things available from HTTP headers.
Best hours I have ever spent, that script is heavily used and still works fine. That solution has evolve from a few reporting scripts to some nice graph with grafana and it's just great, the solution has been staying for almost 2 years already and I use it every single day. Would never migrate for google analytics or any other solution I've tried in the past, nothing better than custom stuff and the entire thing run in about 20 lines of code (https://gist.github.com/mickael-kerjean/289d3d0be8fab2f90e2ff541bc3415a6) ...
I would second this. In the end, you will always regret not having just sent all your data to an SQL database from the very beginning.
Having all the data in one place - and thus easily analyzing across different sources is how you find channels that work best for you. And that's super important from the very beginning and forever.
It's great to hear you've found your ideal tool and even created it yourself! Can't believe that smth like that can be created with only 20 lines of code.
But why did you decide to go for a custom solution? What didn't you like in the analytics tools you used previously?