Beware of user interviews and metrics.
When you're drawing insights from your metrics and user interviews, keep in mind that the data you're collecting is heavily influenced by the manner in which it is connected. Users (and metrics) are notoriously bad at explaining the true nature of things.
User interviews and metrics are essential, but they can also be extremely misleading. Listen to them, but don’t immediately believe what they say.
Here are 2 reasons why...
First, users are terrible at explaining what they want and need.
In the words of the late advertiser, David Ogilvy:
"The trouble with market research is that people don't think what they feel, they don't say what they think, and they don't do what they say."
As entrepreneurs, we need to read between the lines and understand the nuance of what our customers are telling us. As Henry Ford (apparently never) said:
"If I had asked people what they wanted, they would have said faster horses."
If you’re not even talking to your customers (as many early-stage founders aren’t) the situation is even worse - the foundation of your business model is probably guesswork.
Second, metrics can be both useful and misleading. Be careful how you interpret them.
Take the concept of “average”, for example. If your metrics demonstrate a range of user behaviours, you may be tempted to design something that works for everyone. But say, for example, ten users scores 100 on a specific metric you’re tracking, while ninety other users score 0, the average would be 10. The trouble is, no single user scored 10. You would literally be designing your product for nobody.
I’ll include some examples in the comments that explain this way better than I have, but in the words of TedX superstar, Rory Sutherland:
"Metrics, and especially averages, encourage you to focus on the middle of the market, but innovation happens at the extremes. You are more likely to come up with a good idea focusing on one outlier than ten average users. It is perfectly possible that conventional market research has, over the past fifty years, killed more good ideas than it has spawned, by obsessing with a false idea of representativeness."
Takeaway:
When conducting user interviews and analysing metrics, look for commonalities, not averages, and focus on the extremes. It’s much better to start with 3 users who use your product several-times-a-day than 300 users who log in once-a-month.
Happy hunting!
Image credit
– Featured image by neo tam.
– A good article about averages.
P.S. If you like Rory Sutherland, or have never come across him, this interview shows him in full flow.