Product Analytics for Closed Alpha Launch
Hey everyone!
I was hoping to get some advice on what kind of product analytic and/or user behavior metrics I should be tracking for the first closed alpha launch of my product?
Some background info/context:
My small team and I should be finishing the development of our web app (SaaS) product in the next few weeks. We are planning on having a closed alpha release where we've talked to a number of our target users/customers and will have them use & test our product. Prior to this, we had built a proof of concept that helped us validate some of our early assumptions when we demoed it to our target customers, but it was not very usable to where we could track how it was actually being used. Which is why we decided to build out a minimal version (~1-2 major features) of the product as a web app in order to start tracking how it was being used for the purpose of validating more assumptions and iterating based on these insights. The closed alpha will be free and not open to everyone in the public so we won't be able to collect any monetary or activation based metrics.
So to clarify, my question and what I was hoping to get help on was determining:
-
What kinds of metrics should we be focusing on collecting? (daily active usage, surveys, heatmaps, user sessions recordings, session time, page time, etc)
-
What tools would be best to use in order to collect this data? (Hotjar, Inspectlet, or any others?)
We want to be able to observe how our closed alpha users are utilizing the product in order to prioritize features to further build upon and iterate our product. I have some ideas but would really appreciate some help :)!
Hi @devos,
I joined this platform a little late. I am a founder of fibotalk ( https://fibotalk.com )
I suggest you should track page engagements, API latencies, power user behavior, features adopted during your alpha launch. There are many analytics tools that may help you but would need a lot of effort to implement.
Fibotalk is extremely easy to implement and helps you track your KPIs without being an analytics ninja.
I would recommend against heatmaps or user session recordings as they are too time-consuming with very limited insights.
On Fibotalk, you can easily build multiple dashboards and charts (funnel, trend, and journey charts) for different parameters and events to track product usage. We readily provide login and signup trends, top power users, page engagement, and much more.
This is not all. You can build user cohorts based on feature utilization, API response errors, trial plans, and a multitude of user parameters.
We just don't provide insights but close the loop as well. There is an engagement module to send in-app announcements.
We are open to co-innovate with indiehackers tribe.
Hey @devos, I do analytics for a living. To answer your questions:
You should pick metrics that give meaning to the things you care about most. Common analytics tools will give you a wide variety of metrics, so you don't have to make a choice between Metric X or Metric Y. It sounds like your objective is to understand what your audience is doing in closed alpha, so I would focus on navigation and feature interaction. That can be measured with page views, aka what you get from the basic JavaScript snippet of tools like Google Analytics, Mixpanel, etc.
I have a recommended stack: Google Analytics and Mixpanel, fed by Segment. It's an extremely powerful set of tools, all with a generous free tier. GA will give you aggregate flows of users throughout your app. Mixpanel lets you zoom in on individuals (and also helps you take action like personalize messaging).
By the by, I'm writing a book on all of this.
The goal for this book is to take you from "not sure what to do" to "done" in a half-day, with future-friendly analytics data capture all set up.
The book tells you
-which tools to use, and why (basically an expansion on why I chose that stack listed above)
-what data to capture, and in prioritized order
Why me? I've spent the last 8 years in the analytics industry, building scalable, billion-dollar analytics for PlayStation, Rakuten, and a chunk of the Fortune 500.
Sign up here to be notified, and to receive four quick wins and a quick start cheat sheet as freebies: https://signup.analyticsforindies.com
And/or, follow the product here on IH (but the freebies come with your email signup):
https://www.indiehackers.com/product/analytics-for-indies
Google analytics does a great job overall, but it also collects a bunch of crap that you might not need. Hotjar is pretty good. I would suggest you look into Mixpanel and Fullstory as well, both are amazing tools.
If you want to go the custom route (which I always do but strongly advise against it) you can set up an action tracking system, push the logs into Elasticsearch indexes and create custom dashboards with Kibana.
Hey andreigasper, I really appreciate the advice! I was also looking into Mixpanel but wasn't entirely sure if something like that was necessary at this stage. I'll def look into Fullstory!
Now what kind of metrics would you suggest we focus on collecting at this stage? As this is a small group of end users/customers in a closed alpha. Things like daily active usage, surveys, heatmaps, user sessions recordings, session time, page time, are fine or is there something I am missing? Should I also map out a chain/flow of events that lead to some kind of significant value from the product and track this also for example? Would be great if you could point me in the right direction, am always eager to learn and explore new resources!
Well sure, the more data the better. I haven't heard a single person complain about having too much data about their product.
That being said, all that data is merely noise if you haven't figured out the KPIs for your product. So, I think you should first focus on establishing what your core KPIs are and expand from there. Once you have that you can look at the rest of the data in relationship to your KPIs.