Actionable Metrics and Hypothesis Testing
I read "The Lean Startup" by Eric Ries a couple of years ago. One of the things I made a note of was the idea of "vanity metrics".
Today I finally grokked how vanity metrics are really just "outcome measurements", whereas strong actionable metrics are "input measurements", and how these two kinds of metrics interact in a bi-directional feedback loop.
Vanity Metrics
For those of you not in the know, a "vanity metric" is something you measure, but can't act on. They are things you measure to "feel good" about your progress, even if it doesn't help you. These measures are often seen as important, like monthly recurring revenue, but they are outside your direct control.
For example, a vanity metric could be "number of units sold last month". You have no control over your customers wallet, so this is in general a "feel good" metric.
Actionable Metrics
Actionable metrics on the other hand are metrics you can act upon directly; you have complete control over the outcome here, as the outcome is your own input.
An example of an actionable metric is "cold calls made last month". Given that leads are not a bottle-neck, this can be improved simply by picking up the phone.
The Feedback Loop
A core idea behind Lean Startup is hypothesis testing. You want to throw away wrong beliefs about how to grow or otherwise improve your business.
The problem with vanity metrics is that they are usually too complicated; for example, everything factors into your total revenue. If your revenue goes up, you probably can't pint-point exactly why.
So we choose proxies like number of sales, but these are still vanity metrics, because you have no control over it; you do not control your customers' wallets.
How can you improve sales then? Well, you hypothesize that making more cold calls will improve sales, so you make the actionable metric of "Cold Calls Made Last Month".
The feedback loop comes from keeping tabs on sales; if your number of cold calls soars, but your sales don't, you should probably stop making cold calls!
Recap
You want to measure the important things, but you can't affect these things directly. Instead you measure a thing you have complete control over, which you believe will correlate possitively with the important stuff.
Later you keep tabs on the "important thing" and see if it improves given improvement to the proxy. If it does, do the proxy more! If not, substitute the proxy!
It might seem obvious in hindsight, but some things just take longer to truly understand that they ought to.
Thanks @undreren - very clear summary!
Keen to know how would you leverage actionable metrics to go further. In fact, I do regular targeted cold outreach that bringing me favorable conversion for my little side hustle. How would you apply other actionable metrics?
Thank you!
From my perspective, you take important metric you do not have complete control over, like conversions, sales or churn. From that metric you choose an action that you believe will move the needle on your metric in the direction you want.
From that concrete action, you create a new metric. For example “cold calls the last 10 business days” or similar. This is your actionable metric.
Thank you for your takeaways! Just to add my two cents:
Yes, vanity metrics in themselves do not provide an opportunity for action. However, they sometimes can provide leading indicators that direct to more actionable metrics.
For example, while an increase in pageviews is not actionable, it may help to find more information taking a look at what users do next. How many of them returning to the site? Who are visitors, and why are they coming to the site?
A low return-rate may indicate a need for more useful content, and an increase in returning users—more engagement, which in turn may lead to more conversions.