Just because something is easy to measure doesn’t mean it’s important.Seth Godin
Who vs. how many
July 3, 2008
Don’t waste time trying to learn anything from your data and analytics until you have a meaningful number of customers and visitors. That’s at least 100 paying customers and 5,000 unique monthly visitors. But even then, your data likely won’t be enough to make great decisions. With smaller numbers, you won’t have any kind of statistical significance. More importantly, nothing you can learn from that data will come close to giving you the kind of insight you’ll get from talking to customers. This early in the game, data and analytics are effectively meaningless.
I’m not advising that you don’t put these tools in place. But don’t get distracted by browsing analytics and looking for insight too soon.
The term “analytics” can mean different things, but I’m referring to data like traffic, referrers, SEO, customer acquisition costs, conversion rates, churn, lifetime value, and all that good stuff. It’s all incredibly valuable at the right time, but in the early days it’s a diversion. None of it matters until you’ve talked to enough customers to know what to look for.
I’m not advocating neglecting the data; just take a measured approach to it. Skim it once a month and then get back to work. The same goes for calculating churn, customer acquisition costs, lifetime value, and all of the usual SaaS metrics. Do what you can to set up your tools and record this data for historical purposes, but until you have hundreds of paying customers, don’t get too caught up in it.
There will, then, be a point at which the data becomes useful. It’s handy to take the time to set up your analytics tools and ensure this data is being tracked accurately. But don’t spend too much time poking around. The same goes for A/B testing. Until you have huge amounts of traffic, A/B testing won’t tell you much. You’ll be better off talking to customers and observing their behavior.
The flip side to data and analytics is that it can be misleading. It’s incredibly easy to make data tell you the things you want to hear or that you already believe. You can hypothesize and interpret the data so that it supports ideas you already have. But data and analytics can only expose what is happening–it can’t tell you why people are doing what they’re doing. For that, you’re going to have to talk to them.
I may sound like a broken record at this point, but the best data for your business in the early days will stem directly from your conversations with real paying customers. Every time you’re tempted to start browsing Google Analytics, reach out to your customers and start scheduling phone calls.
Once you’re ready, focus on purpose-driven analytics. Avoid the temptation to simply browse your data. Use it to inform your decisions and answer questions that arise. Use it in conjunction with talking to customers. Use it as a way to monitor things and check the pulse. Just remember that it’s only a fraction of the big picture. Even once you’ve gathered statistically significant data, don’t forget to talk to your customers. Data is no substitute for conversations.