Strategic Analytics – Validation, Accessibility, Innovation, Repeat

I’ve written posts before on the four stages of product analytics. The four stages are Collection, Validation, Enhancement and Accessibility. To be strategic with your analytics programs, you need to be able to do a couple of things really well. You need to first have confidence in your data and you need to leverage it quickly. Before you dismiss the attention to quality because I used the word quickly, I am not saying this process happens overnight. I’m referring to the ability to build services that give you access to the data quickly. In order to innovate, you need to create and solve problems faster than the market requests them. Sometimes, if you are good enough, your products create that market demand.

In order to innovate, you need to create and solve problems faster than the market requests them.

Many times I see companies who use Analytics as a means to an end. There could be a general assumption that data is accurate and therefore validation is an unneeded resource drain. Huge mistake. I take the bookends approach to strategic Analytics. It is always involved at the beginning (planning and design) and is the last thing in the process (reporting). If those two do not line up, then what stakeholders usually see is an inconsistent, frankensteined page of data that holds little value and tells no story.

A Novel Approach
To use that analogy a bit more, think about your product as a novel. You create an outline, writing out ideas about what the characters will do, where they will go and how they will interact. You create the table of contents, then you set out on your journey to write your novel. The goal is to get them to the end and be entertained. Think of Analytics as the outline and the table of contents. Reporting would be the last chapter where all the hard work you put in pays off and giving the user the information they need to validate the story. Think about if you were reading a book and in the final chapter nothing was wrapped up? You do not find out what happens to the characters and the story doesn’t match the outline you initially wrote. Not a good story.

I challenge you to have a series of questions created during the planning of your application. What do you want the user to know when this is done? List them out and validate those questions when you have data. You will get a sense of what changes you need to make or be able to see how you made decisions during development that did not align with your original goals for the product.

Setting Goals
Set goals for what you need out of your Analytics platform. Try to do things that move you closer to those goals. You may be missing validation, set a short term goal to do 3 things that help validate data. Once you get those done, you may want to move on to setting a goal to creating a simple platform for delivering the data. I have recently started thinking in a different way about setting goals. My goals are thematic and not tactical deliveries. I start there and then push to include specific items within the themes I created. The themes could even be just the pillars of Analytics (examples):

• Collection: Enhance collection methods to streamline how users send data
• Validation: Provide methods to catalog, tag and verify information collected
• Enhancement: Using collected data, create data sets that help provide insights to users
• Accessibility: Make organizational data available without developer support

Establish Context
If I told you that 74% of people in the US are voting for Donald Trump in the 2016 election, what would you say? Break that down with some questions you should always ask everytime you see a statistic.

• Where did the data come from? – A republican poll? Democratic poll? Trump campaign? Hilary campaign?
• When was the data point taken? – last year, yesterday?
• How was the data collected? – At a Trump rally? Which state? What time of day?
• What else can you tell me about the people who were polled? – Gender, Race, Socio-economic Status?

Context is key to understanding Analytics. A great Analytics product manager questions all statistics, even the ones he/she creates. A good set of Analytics and good dashboards always create more questions than they answer. That is the purpose, to build insights that move you to make better and more innovative business decisions. KPIs (Key Performance Indicators) are beneficial even if they are positive or negative. They are telling a story about a process or workflow. Understand the context of the data, you will build better insights.

Innovation and Influencing Behavior
Once you have a good foundation, it gets really fun to be able to slice and dice data to create new paths of innovation. The goal of Analytics is usually just being able to react to previous patterns of behavior and influencing future behavior. Successful companies do this well. Being able to read data points and take action or delivering data so that others can take action is the fundamental purpose of product Analytics. Reporting is not analytics, it is one component of a greater ecosystem that has to work together.

Repeat the process over and over and you will see that your ability to innovate is limited by the validity of your data the speed at which that data is accessible.

Be a leader, question everything and build from a good foundation. Consistency is worth more than accuracy. You can be consistent and inaccurate, that is ok. You cannot, however, be inconsistent and inaccurate. Consistency is the ability to give the same answer over and over no matter the temporal variance of that question (If I ask a question on Monday will I get the same answer if I ask the exact same question on Friday). Stakeholders will understand if your equations are wrong, that can be explained. If your processes are wrong, you have lost all credibility no matter if you get the right answer eventually.