Do our metrics help us to make decisions, or do they just make us feel good?
When we track our progress and report our results we need to avoid using vanity metrics.
These are the metrics that make us feel good. They favourably compare our performance to others, or to arbitrary benchmarks. They tell a positive story.
For example, the number of visitors to our website, the number of likes on our social media posts, or the number of times we’re mentioned in the media.
For those of us who are competitive by nature the comparisons baked into vanity metrics can be very motivating. But we need to be careful that we don’t fall into the trap of comparing our insides with others outsides.
We also need to be wary of vanity milestones.1
These are the things that make for good press releases and generate attention. They give us something to publicly celebrate.
For example, winning awards, partnerships with much bigger companies (which rarely, if ever, translate into significant sales for the smaller partner) or attracting small investment amounts from celebrity investors.
We need to remember causation vs. correlation. Lots of successful companies win business awards. But winning business awards is only loosely related to the things we need to do, have and be in order to build a successful business.
Vanity metrics and milestones are invigorating but potentially flattering and misleading.
What’s the alternative?
We should prefer clarity metrics.2
These metrics highlight trends in our performance and ideally help us decide what to do next. They are actionable.
For example, average usage times per user, average time spent in on-boarding, churn/retention rates by cohort, revenue:acquisition cost ratios, etc.
Clarity metrics are humbling but sometimes exhausting.
It’s easy to be dismissive about vanity metrics and milestones, but…
The average finishing time for a marathon on Strava is 3:58.25 - i.e. just under 4 hours.
The technical difference between a 4:01 marathon and a 3:59 marathon is insignificant - just 2.8 seconds per kilometer (perhaps a few strides). But the psychological difference is huge.
So, let’s not pretend that vanity metrics don’t have the potential to positively impact performance.
It can also be useful to have some hero metrics to demonstrate progress to people outside of the company, and to generate buzz that might tempt some of them to get more involved.
The best example I’ve seen of this recently is an environmental restoration project in the Coromandel, where the metric they talk about is “the number of complaints about bird song noise”. Those running the project figured that was a great way to show they were having an impact, and even more, to signal to everybody in the area what the real purpose of the project was (i.e. it’s not just about planting trees and clearing traps).
We just need to always keep in mind which metrics are useful because they make us (and our team) feel good and which are useful because they help us decide what to do next.
Those are likely different metrics!
Source: Chris Dixon. ↩︎
These names are not new or original. Eric Ries talked about vanity vs. actionable metrics in his book “The Lean Startup”. First Round Capital also published a good breakdown of vanity vs. clarity metrics, with specific examples for different business models. ↩︎