How do you record and analyse your Lean Start-Up Experimentation/Learnings?
Hi Folks,
Wanted to ask for some guidance. We are big fans of the lean startup method and are applying it on a current project. Unfortunately, the aspect we are not so great at is recording, analysing and extracting learnings from our experiments. What tools or approaches do you use? Appreciate your wisdom : )
I try to boil my "hypothesis" down to a set of cells in a spreadsheet. For example, last month, I tested (and rejected) a project based on Google search ads. I started out by producing a "formula" for what the basic funnel would look like. Here's a simplified list of the variables that would affect my funnel:
Using these values, I could estimate a few more things:
I then put up a landing page and directed search traffic to it. I couldn't measure everything, but I could validate a subset of these: impressions, CTR, CPC, and the conversion rates. I plugged in different values for the others to see what the business might look like under different assumptions.
So I'd try to set up my hypotheses in that manner: how many people are subject to it, and how will they behave? Pre-register your beliefs, and compare to the result after you're done.
Thanks Jake, I appreciate your guidance. I really like how you have simplified your approach down to something that is simple and repeatable. Have you used this framework with more of user feedback / NPS focus? Our product is early, we are initially focusing on useable to get it to a v.1 level.
I work for a bigger company, and that's basically our approach to experimentation. I have a little less experimentation doing it "in the small," but from what I've seen, I don't see why it wouldn't work. NPS and quantitative feedback help us pick areas to focus on. We then identify trends/strengths/weaknesses in our funnel, and pair these together to conduct user research for qualitative feedback to narrow down on specific hypotheses.