Analytics: Storytelling not data reporting

So often I get requests like “I need this metric.” or “When can we have this report?” Part of the job is enablement and facilitation but there is also the part where you help people to understand the story that is told with the metrics. If you want to be considered a leader in a particular industry, you have to provide analytics that tells a story. You also have to know who is reading the story so you can frame the context, keep them engaged and wanting more.

It is taboo in the analytics world to use the term “vanity metrics” with a client. Yet, they do exist.

A vanity metric is something that shows scale without anything to compare it to.

It is similar to the reason TV producers put some useless sex scene in a movie or show. It usually doesn’t push the story along but it keeps the users engaged. You see them all the time in social media or website statistics. Things like, “Total Page Views” or “Number of Twitter Followers”. Clients love to see that they had 1.3M page views last month, 250K unique visitors or they now have 350K Twitter followers. But compared to what? Alone, these metrics are useless on a report. If you cross other metrics with these numbers and use metrics over time as a percentage of the whole, things start to get interesting.

For example, if I run an e-commerce site and I track value per user, I can use my unique visitors total, tracked by day/week crossed with my transactional data (goals that target orders, newsletter signups, social shares, etc) to determine an overall value per user. Then that unique visitors total becomes an interesting story to discuss with marketing in regards to the campaigns they ran last month. Especially if you spent 25K on a campaign that netted you 3.5K new unique users or, more importantly, netted you a negligible amount. Unless you are omnipotent and all-knowing, analytics sometimes disproves thoughts and ideas. There is a whole lot of value in that.

In my dealings with Data Science, I work with them to test a hypothesis. It is never an exercise in being right. You should always go into an experiment with “I have this idea, the data infers a relationship between a and b. I believe that if we do something, x, that relationship will be proven correct some percentage of the time. That percentage of accuracy becomes the story. And it varies from experiment to experiment or market to market. If i tell you that batter in baseball gets a hit 3 out of 10 times (.300 average), then that player would probably be an All-Star, year in and year out. But, if I equate that same average to a quarterback in football, a quarterback completes a pass 3 out of 10 times (30%, same .300 average), that player would be cut. The context is so important for the experimentation.

Back to those questions.

“I need this metric.”
This may actually be a valid request. But ask questions and listen to the answer. You should ask, “why?” or more importantly, “what are they trying to say with this specific number or group of numbers?”. As an analyst, product manager or solution provider, you should always ask for context when a request is made. You want to be the subject matter expert and not just a facilitator. A dialogue can sometimes yield valuable insights and lead to solutions that are essential to organizational success. If 40% of your clients are asking for the same type of data, there might be a common industry story waiting to be told. Don’t discount or immediately facilitate the request, capture, question, qualify and act on it. Over time, you will find gaps in reporting and you will just have to provide simple, baseline metrics without this process. It is important to know the difference. If someone says “I need page views” and you don’t have that basic metric, make it a priority. Vanity metrics are usually the numbers from which all valuable data is derived. All your worthy competitors will have it.

“I need this report.”
A report is just an organized collection of metrics and visualizations meant to validate and lead to new business decisions. When creating a report, always ask “what does this tell the audience about the source of the data?”. Does each part of the report give insight into a target business metric meant to validate previous actions or cause new actions? If it doesn’t, drop it. So often I hear “Well, this report only has 2 things on it so can we add a couple more items?” Short answer, “can we?”, yes. “Should we?”, no. Useless data drowns and invalidates reports and clients pick up on that.A report is perfectly fine have 1 item, provided that item has context within itself. If its a visualization of page views over time it isn’t ok but if it is page views over time benchmarked with month over month data showing an increase or decrease from the previous month than it may have value. A secondary metric over the same time period showing number of campaigns launched by day would tell an even bigger story because the temporal context is there to validate dollars spent on campaigns to new page views.

Storytelling in analytics is important. If you sell products, it creates a genuine dialogue with your clients that helps you understand their needs. It also shows that you are invested in providing solutions to real business problems across an industry. It can also help you refine your own products in ways you never would have thought. Very often, the questions we ask ourselves or assumptions we make are proven false because of patterns in data. Those patterns, however, reveal the truth. We can act on that with more determination since we know the correlations will lead to a positive impact on our business. Do not get caught up in the trap of not validating your business decisions with testing. If you have the means to test a number of ways via A/B testing, do not pass that up. It is just as important to tell the story of what does not work so you don’t repeat history and waste money.

After all, the point of all of this is to make more money, right?.