Web Analytics KPIs = BS

I’m starting to prepare for the tutorial I’m giving at the JBoye Conference in Philadelphia on May 3. The class is called Beyond Visits and Page Views…How to Develop Actionable Web Metrics, Reports and Analysis.

One of the areas that I’ll be going into in some depth is how you can build metrics that drive decision making. This would seem to be a clear goal for any Web site measurement initiative, but it is a huge struggle for many organizations. There are multiple reasons, but I think there is one in particular that causes considerable distraction from meeting this goal:

The popularized understanding of Web Analytics KPIs has led us down the wrong path.

Key Performance Indicators (KPIs) are a term that you hear a lot in Web Analytics practice. KPIs are commonly considered to be the metrics that you use for measuring your Web site with Web Analytics. For many, visits, visitors, page views, referrals trended month over month and year over year are the Web Analytics KPIs. This is not what KPIs are about; in fact KPIs are only a part of an equation that creates a complete and useful measurement.

Let’s get some historical perspective. In the 1960s, D. Ronald Daniel of McKinsey and Company came up with a methodology to help organizations define and measure their effectiveness against business objectives. In the 1980s, this methodology was clarified and evangelized by John F. Rockart of the Sloan School of Management. The result was a hierarchal system of evaluation that was based on:

• Definition of organizational mission
  • Identification of strategic goals that map to the mission
    • Determination of concrete and measurable objectives
      • Definition of Critical Success Factors (CSFs) that are used to evaluate the objectives
        • Developing a KPI measurement and target that determine whether the CSF is being met

In Web Analytics, the model could look something like this:

• Organizational mission – Be the industry leader
  • Strategic goal – Provide the most reliable and authoritative information
    • Objective – Publish a weekly email newsletter
      • Critical Success Factor (CSF) – Increase number of subscribers to the weekly newsletter
        • KPI- # of newsletter subscribers per month
          • Target – increase # of newsletter subscribers by 10% each month

The idea here is that you can have a number of CSFs per objective, and a few KPIs and Targets associated with each CSF.

As you can see, KPIs are only one part of a bigger picture and you can do a lot more to make your current Web Analytics numbers relevant, compelling and useful. It doesn’t necessarily have to take a lot of work or sophisticated tools…it just takes a different and more contextual way to look at the numbers.

So, I hope you’ll join me in Philadelphia for the tutorial. I’ll show you how you can use the real KPI methodology to improve your metrics and talk about other techniques that will really help you get value from Web Analytics.  I also suggest you consider the rest of the JBoye Conference. As always there is a great program and I’m pleased to see a big emphasis on Web Analytics this year, as well as online strategy, user experience, content management and lots more.

One last note: There's still time to apply for the X Change Web Analytics Challenge to Benefit Non-Profits...deadline is April 30.

Comments

[Jay Tkachuk]

Completely agree - KPI by themselves are just useless sets of numbers. Analytical decision making must be an established process and an accepted, if not integral, part of the corporate culture to be successful.

[Stephane Hamel]

Who wouldn't agree with this approach?

I think some of the issues we face in web analytics are related to "ignorance" - not knowing that everything we do is ultimately the combination or slight variations on things that have been studied, proved and demonstrated for decades.

This approach is something I'm talking about when I do web analytics training & workshops.

[Matthias]

I am sure that everybody agrees.
As long it is theoretical.

But sometimes this logical rationale is not implemented in a website/e-initiative because an originally sharp strategy is diluted during the (content) development phase. Reasons can be to over-discuss technical issues (out of IT), focusing on design before having content and functionality defined, or by having too many stakeholders.

Worst possible outcome: When many things are intended to be met in parallel, nothing is really met. If a website is meant for anyone, its actually for no one. (Show me a single user who believes to be just "anyone")

Having too many KPIs (by trying to serve too many CSFs, which may be due to multiple goals) may lead to a similar situation than by not having any methodology in place.

Did anyone(!) experienced something like this, too?

[cleve]

Thanks for the post. I really like the hierarchical way that you've broken down the route from mission to target. Very informative.

Talking to Matthias point above, I experiencing a similar kind of thing. Too many KPIs have in the past de-valued them.

In this post I'm reading KPIs as performance metrics. I need a KPI to inform me how well the company is doing against it goals because those are where the key decisions are made. Maybe there are different levels, but KPIs bring hard numbers to goals, rather than SMART objectives.

Does that make sense?

[Cleve Young]

The same concept Matthias mentions of 'a website is meant for anyone' too often applies to KPI's - they are meant for everyone. Yet what is important to a Marketing Manager is not what's important to Product Category manager. Each of these folks cannot make a business focused decision by looking at the same metrics, yet they often try.

Usually when I get asked to create KPI's or other reports the requester asks for specific metrics. Yet they usually don't have the in-depth knowledge of web metrics to be picking the 'right' things. As much as possible I will go back to them and ask "What is your business question?" From there I can work on giving them the numbers and analysis which will best answer their need, which often is not the same metrics as their original request.