Web Analytics Management

Web Analytics KPIs = BS

I’m starting to prepare for the tutorial I’m giving at the JBoye Conference in Philadelphia on May 3. The class is called Beyond Visits and Page Views…How to Develop Actionable Web Metrics, Reports and Analysis.

One of the areas that I’ll be going into in some depth is how you can build metrics that drive decision making. This would seem to be a clear goal for any Web site measurement initiative, but it is a huge struggle for many organizations. There are multiple reasons, but I think there is one in particular that causes considerable distraction from meeting this goal:

The popularized understanding of Web Analytics KPIs has led us down the wrong path.

Key Performance Indicators (KPIs) are a term that you hear a lot in Web Analytics practice. KPIs are commonly considered to be the metrics that you use for measuring your Web site with Web Analytics. For many, visits, visitors, page views, referrals trended month over month and year over year are the Web Analytics KPIs. This is not what KPIs are about; in fact KPIs are only a part of an equation that creates a complete and useful measurement.

Let’s get some historical perspective. In the 1960s, D. Ronald Daniel of McKinsey and Company came up with a methodology to help organizations define and measure their effectiveness against business objectives. In the 1980s, this methodology was clarified and evangelized by John F. Rockart of the Sloan School of Management. The result was a hierarchal system of evaluation that was based on:

• Definition of organizational mission
  • Identification of strategic goals that map to the mission
    • Determination of concrete and measurable objectives
      • Definition of Critical Success Factors (CSFs) that are used to evaluate the objectives
        • Developing a KPI measurement and target that determine whether the CSF is being met

In Web Analytics, the model could look something like this:

• Organizational mission – Be the industry leader
  • Strategic goal – Provide the most reliable and authoritative information
    • Objective – Publish a weekly email newsletter
      • Critical Success Factor (CSF) – Increase number of subscribers to the weekly newsletter
        • KPI- # of newsletter subscribers per month
          • Target – increase # of newsletter subscribers by 10% each month

The idea here is that you can have a number of CSFs per objective, and a few KPIs and Targets associated with each CSF.

As you can see, KPIs are only one part of a bigger picture and you can do a lot more to make your current Web Analytics numbers relevant, compelling and useful. It doesn’t necessarily have to take a lot of work or sophisticated tools…it just takes a different and more contextual way to look at the numbers.

So, I hope you’ll join me in Philadelphia for the tutorial. I’ll show you how you can use the real KPI methodology to improve your metrics and talk about other techniques that will really help you get value from Web Analytics.  I also suggest you consider the rest of the JBoye Conference. As always there is a great program and I’m pleased to see a big emphasis on Web Analytics this year, as well as online strategy, user experience, content management and lots more.

One last note: There's still time to apply for the X Change Web Analytics Challenge to Benefit Non-Profits...deadline is April 30.

An “Under the Hood” Look at SiteCatalyst 15

On the fly segmentation, additional variables and custom events, display of multiple metrics on a single graph and presentation of unique visitors, visits and page views on all reports are the upgrades that got most folks excited about last week’s preview of SiteCatalyst 15 at the Omniture Summit. My colleague Gary Angel has a nice summary of these features in his recent post.

I thought one of the more interesting upgrades is about what you don’t readily see…how data in SiteCatalyst is processed. In the new version, you will be able to have what appears to be significant more control over how your data is processed as it goes into SiteCatalyst than you do currently, and will likely need to depend less on s_code customization and VISTA rule implementation to achieve this flexibility. This data processing rules control will be available from the Admin interface. What this means in “on the ground” terms is still a bit murky based on the conversations I’ve had with Omniture team, but I like the fact that this is being introduced. It’s definitely a step in the right direction.

SiteCatalyst Data Processing Today

Today, to really control how data is processed in SiteCatalyst, you need to either do a lot of customization on the s_code, as well as pay for expensive VISTA rules. If you’re a SiteCatalyst customer you know that there are roadblock issues associated with both methods.

For s_code modifications, not only do you need to unit test your code, but if you’re in a large enterprise or one that has established IT change control processes, you may need to have the new code go through additional QA as it moves through test and production environments. If you live in this world, you know that this can often take a lot of time and impact your ability to collect data in a timely manner.

VISTA rules allow you to populate multiple variables based on a keyed value in another variable. Internal IP filtering, for example, is a good candidate for a separate VISTA rule. Sites with complicated page hierarchy or intense usage of URL parameters often will use VISTA rules to decipher information in the page URL and populate more user-friendly or analytically useful variables. VISTA can also be used for visitor segmentation purposes, populating visitor information based on source or page URL. Not only are VISTA rules expensive, but you need to contact your Account Manager to set them up.

First Look at SiteCatalyst Admin Based Processing

On first look, the new Admin based processing control will mitigate reliance on both VISTA rules and s_code customization. It isn’t completely clear at this point how much, but it is certainly a step in the right direction for enabling easier implementation and maintenance. This addresses a key sore spot in SiteCatalyst and has certainly been a contributing factor for some organizations to just give up on using it.  

If you are a bit concerned about having so much control over your data processing at basically the flip of a switch, you should. If you implement a rule incorrectly you could seriously and irrevocably compromise your data collection and accuracy.

Omniture has considered this as well. Prior to allowing any clients to use the data processing rules dashboard, you will need to be certified by Omniture as an administrator. This will be a new certification. Sure, it’s another money making opportunity for Omniture, but I think this training and certification makes sense. 

I think there’s a bit of irony here as well if you have an historical perspective on web analytics. In the “olden” days of enterprise log file analyzers, you really needed a systems administrator to manage the tool for tasks such as log file importing, stitching, and processing (and of course, those that use systems like Webtrends, Urchin and Unica still do). As web analytics migrated away from IT and into Web and online marketing teams (due in large part through analytics vendor changes in sales direction) there has been an emphasis on not having IT involved. Now, we have another example of analytics coming back to having a resource involved who understands how data works and what occurs under the hood with data. Perhaps it isn’t someone in IT, but it must certainly be someone who is more in the weeds than a typical online marketing or web content manager or even a web analyst.

This will be something new to those who have relied on a software as a service model for web analytics. I think it speaks to the need to view digital analytics as a program with its own leadership, governance, and process. If you’d like to read more about how to develop this in your organization, I invite you to take a look at my white paper on managing digital analytics.

Related Posts:

Omniture Summit

Is Omniture Stepping into a Privacy Morass with Demdex Acquisition?

Getting a Grip on Digital Analytics

P.S - I'll be giving a tutorial on developing business-focused web metrics at the JBoye 11 conference in Philadelphia on Tuesday, May 3 and a conference presentation about the work we've done with The Nature Conservancy. Hope to see you there.

P.P.S - If you're a non-profit, I invite you to take the X Change Web Analytics Challenge.

Is Omniture Stepping into a Privacy Morass with Demdex Acquisition?

The opening session at an Omniture Summit is a mix of pep rally, vision and celebration. Today’s event did not disappoint.  To start the show, VP and GM of Omniture, Brad Rencher, tossed out some interesting stats about the audience.  The one that struck me was that 40% of total online ad spend was represented by the companies in the room. That’s a pretty big number. So, it was not surprising that a big part of the “vision” aspect of the session was all about online advertising and the potential value offered to online advertisers by the January, 2011 acquisition of Demdex.

Online advertising is where the money is…$109 billion by 2013 according to IDC… and Adobe Omniture is placing itself squarely in the middle of trying to enable both advertisers and publishers do a better job of targeting ads. Demdex is a platform that allows you to aggregate all information you can get your hands on about online visitors. It’s like a CRM system, but allows you to put data in from external sources, such as Demdex’s data partners Acxiom, Datalogix, Exelate and TargusInfo.  It’s a pretty powerful concept, no doubt. Vivek Shah, CEO of Ziff Davis, went into a surprisingly detailed description of how they’ve created BuyerBase, a targeted platform focused on 40 million tech buyers, with the Demdex technology.

Well, as one of the opening session speakers quipped, “the Omniture Summit is like Woodstock for geeks”…Or something to that effect. So if all of this talk about how to use data for online advertising gets folks feeling as heady as the contraband passed around at Woodstock, is there also a parallel to the “brown acid” warning in the form of privacy?  Demdex seems like just the type of company that the FTC was talking about in its Privacy Report.

Demdex claims that it is a good citizen when it comes to providing consumers with a view into all the data that is collected about them and be able to opt out. This is accomplished through the Digital Advertising Alliance (DAA) opt out program.  According to Politico, FTC Chairman Jon Leibowitz was questioning this effort, "We commend industry's effort to get a broad group of industry leaders on board. However, the effectiveness of this effort will depend on how, and the extent to which, the opt-out is actually implemented and enforced - all of which is yet to be seen. We also urge industry to make sure that the opt-out is easy for consumers to find, use, and understand."

If visitors don’t accept third party cookies, they won’t be tracked by Demdex.  However, the issue for Adobe Omniture is to both make sure they provide clear and effective options into opt out so it provides Demdex customers with both powerful marketing and something that isn’t going to get regulated out of existence. Maybe this will be covered in next year’s Summit opening session…probably not as exciting as today, but certainly something that will make all potentially interested purchasers be able to rest more easily.