Web Analytics Management

An “Under the Hood” Look at SiteCatalyst 15

On the fly segmentation, additional variables and custom events, display of multiple metrics on a single graph and presentation of unique visitors, visits and page views on all reports are the upgrades that got most folks excited about last week’s preview of SiteCatalyst 15 at the Omniture Summit. My colleague Gary Angel has a nice summary of these features in his recent post.

I thought one of the more interesting upgrades is about what you don’t readily see…how data in SiteCatalyst is processed. In the new version, you will be able to have what appears to be significant more control over how your data is processed as it goes into SiteCatalyst than you do currently, and will likely need to depend less on s_code customization and VISTA rule implementation to achieve this flexibility. This data processing rules control will be available from the Admin interface. What this means in “on the ground” terms is still a bit murky based on the conversations I’ve had with Omniture team, but I like the fact that this is being introduced. It’s definitely a step in the right direction.

SiteCatalyst Data Processing Today

Today, to really control how data is processed in SiteCatalyst, you need to either do a lot of customization on the s_code, as well as pay for expensive VISTA rules. If you’re a SiteCatalyst customer you know that there are roadblock issues associated with both methods.

For s_code modifications, not only do you need to unit test your code, but if you’re in a large enterprise or one that has established IT change control processes, you may need to have the new code go through additional QA as it moves through test and production environments. If you live in this world, you know that this can often take a lot of time and impact your ability to collect data in a timely manner.

VISTA rules allow you to populate multiple variables based on a keyed value in another variable. Internal IP filtering, for example, is a good candidate for a separate VISTA rule. Sites with complicated page hierarchy or intense usage of URL parameters often will use VISTA rules to decipher information in the page URL and populate more user-friendly or analytically useful variables. VISTA can also be used for visitor segmentation purposes, populating visitor information based on source or page URL. Not only are VISTA rules expensive, but you need to contact your Account Manager to set them up.

First Look at SiteCatalyst Admin Based Processing

On first look, the new Admin based processing control will mitigate reliance on both VISTA rules and s_code customization. It isn’t completely clear at this point how much, but it is certainly a step in the right direction for enabling easier implementation and maintenance. This addresses a key sore spot in SiteCatalyst and has certainly been a contributing factor for some organizations to just give up on using it.  

If you are a bit concerned about having so much control over your data processing at basically the flip of a switch, you should. If you implement a rule incorrectly you could seriously and irrevocably compromise your data collection and accuracy.

Omniture has considered this as well. Prior to allowing any clients to use the data processing rules dashboard, you will need to be certified by Omniture as an administrator. This will be a new certification. Sure, it’s another money making opportunity for Omniture, but I think this training and certification makes sense. 

I think there’s a bit of irony here as well if you have an historical perspective on web analytics. In the “olden” days of enterprise log file analyzers, you really needed a systems administrator to manage the tool for tasks such as log file importing, stitching, and processing (and of course, those that use systems like Webtrends, Urchin and Unica still do). As web analytics migrated away from IT and into Web and online marketing teams (due in large part through analytics vendor changes in sales direction) there has been an emphasis on not having IT involved. Now, we have another example of analytics coming back to having a resource involved who understands how data works and what occurs under the hood with data. Perhaps it isn’t someone in IT, but it must certainly be someone who is more in the weeds than a typical online marketing or web content manager or even a web analyst.

This will be something new to those who have relied on a software as a service model for web analytics. I think it speaks to the need to view digital analytics as a program with its own leadership, governance, and process. If you’d like to read more about how to develop this in your organization, I invite you to take a look at my white paper on managing digital analytics.

Related Posts:

Omniture Summit

Is Omniture Stepping into a Privacy Morass with Demdex Acquisition?

Getting a Grip on Digital Analytics

P.S - I'll be giving a tutorial on developing business-focused web metrics at the JBoye 11 conference in Philadelphia on Tuesday, May 3 and a conference presentation about the work we've done with The Nature Conservancy. Hope to see you there.

P.P.S - If you're a non-profit, I invite you to take the X Change Web Analytics Challenge.

Who’s Web Analytics Data is it Anyway…Part 3

Since I wrote Whose data is it anyway? and Whose data is it anyway? Part 2 I've started to dive a bit deeper into the subject  of web analytics data ownership as part of my vendor research for the just released The 2009 Web Analytics Report and because I'm managing a vendor selection process for a Federal government entity.

It’s a complex issue and one that often gets overlooked during web analytics vendor selection and contract negotiations. Many analysts and web managers whom I speak to at large organizations either don’t read or don’t have access to their vendor contracts…and generally don’t ask about data ownership during the vendor evaluation process.  Many people just assume that they own their web analytics data.

So what does data ownership really mean? Is it simply a “concept”, or does it translate into anything of business value? Well, it depends on your vendor…what they say, what they don’t say and what they put in writing.

Let’s look at the four aspects of "data ownership":

Use of Your Data

As we’ve already written, Google Analytics and Yahoo! Web Analytics state in their contracts that may use your data for a variety of commercial purposes. Not all, but a few of the fee-based vendors state that they’ll use the data for benchmarking or industry average information. This seems to be a more palatable pill to swallow for large, security-concerned enterprises, such as those in government and financial services.

Data Security

All of the largest web analytics vendors have gone to great lengths to ensure the safety of your data from the perspective of storage, access, and redundancy and server failover. Yet if there’s a data loss, you’re on your own. All agreements absolve the vendors of any liability. So, in this case, what does data ownership really do for you? Not much.

Data Retention

Being able to run reports on historical data and having access to unaggregated data as far back as your contract runs may be considered to be a sign of data ownership, and one that many take for granted, but it is nothing of the sort. There are basically two types of web analytics data you can retain: data that is used in creating summary tables used for the reports you run in the core solution, and unaggregated data that makes up your web analytics database. The length of time that this data is kept may or may not be in the contract. We find this to be part of your service agreement, and can be completely negotiable. Your vendor will have a default length of time for both types of data…typically 12-36 months. You want more than the default, you can pay for it. We know of one fee-based vendor that breaks out default retention on an hourly, daily, weekly and monthly basis…just a bit more complicated and just another way to add to your cost of ownership.   The upshot of this: Data retention has real meaning for what “data ownership” is all about…or does it?

Data access- API or file extraction of core data tables for export to another system.

Before you get all excited about making a deal for keeping your data for 36 months, you have to think about how you’re going to export the data out of the web analytics solution. While it may be easy to pull old reports, what if you want to extract the unaggregated data to export to another database, or what if you want to use it with a business intelligence tool? How do you get it? Vendors that provide a data warehouse capability, are happy to sell you this capability and therefore your underlying data is available and extractable…typically through APIs. But if you don’t purchase a data warehouse capability or your vendor doesn’t offer one, you could get your unaggregated data in a state that is really hard to deal with…it could be unprocessed, meaning that the data is in a “log file” state, or it could be in a CSV format. In either case, you’ll have a lot of work to do to get the data in a state that is usable.

So, what’s “data ownership”? I think you have to figure out what elements of data ownership make the most sense. From my perspective, making sure that you can get your hands on your web data when you need it and with as little interference as possible are prime considerations. Security and privacy are paramount, but if you choose to go with an on-demand solution, you’ll need to be assured that the vendor meets your criteria, knowing that if there’s a data loss, you’ll have little recourse.

California Dreamin' about Web Analytics

Ok, so I am a little wistful about Semphonic's first X Change conference in Napa last week. It seemed special...the conversation, the people, the location, the wine... To me it felt more like a gathering of the best and the brightest in web analytics and site measurement; more like a retreat than a conference. And as Gary explains so well in his blog, that was the vision of the event. The atmosphere was relaxed, the ideas were high charged and during the huddles there were lots of great takeaways...not just new knowledge, but inspirations around the process and practice of web analytics. It was the only conference I've attended where after a full day, I actually felt recharged instead of tired. 

I led an X Change huddle on the topic Focusing Senior Management on the Business Value of Web Analytics.

My group included analytics managers from American Express, O'Reilly Publishing, JP Morgan Chase, Intuit, Xilinx, HIMSS and WebMD. While many challenges were expressed, I'd say that the they could be grouped in 2 primary categories:

1. Obtaining funding and resources support
2. Creating compelling metrics for senior management

We discussed a number of ways to address the issues. Not surprisingly, web analytics practice takes a back seat to project management, communication and organizational political savvy as the way to best tackle the situation.

Developing your strategy for dealing with senior management is based on researching your organization...talking to people, asking questions, and understanding the culture.

Let's break this down into 7 key success actions:

1. Know your audience – understand what’s important to senior management. In most cases, you'll find that its all about the bottom line. That boils down to the following:
   • Making money
   • Saving money
   • Not losing money (i.e., avoid law suits and fines, fraud and theft detection, security)
   This is a pretty simple formula. Use this as a benchmark against all you do, and you'll be successful in making your case most of the time. I learned this in the early days of the Internet (1993), when convincing the company president that a dial up account to AOL was a good investment!
2. Know your environment- understand how information travels; is it presentations, documents, conversations, or the grapevine? The fact is even though you want to reach senior management, you need to conduct an information campaign directed to them directly, as well as at the grassroots level. Get a poster child success story that can generate grassroots buzz, as well as percolate to the top. While it's important to build processes, the success of any process is in getting cooperation and buy in from participants. With this in mind, do what you can to build your internal networks and relationships. It pays to make friends throughout the organization who you can rely on for favors and information. As one of my former managers told me, "To get things done, make sure you have friends in low places."
3. Get their requirements and buy in - find out what senior managers want to know; or do your research and make a proposal on what to analyze; get on the schedule to present in person. If you can't, then refer to Point #1.
4. Know your competition – what other information sources compete with analytics for senior management attention – it's best to cooperate and combine with other business intelligence sources within your organization. You can also discredit other data if it doesn't address business goals. For example, if senior management is fixated on page views, you'll need to educate them on the real value of analytics to the bottom line metrics.
5. Know how to tell a story – know company objectives and how they translate into operational decisions at your level. One of the key stumbling blocks in making analytics relevant is the disconnect that occurs between the lofty objectives for the organization and how these objectives get interpreted outside the executive suite. You need to demonstrate outcomes that relate to the company objectives; tell the story in language the managers understand, i.e., dollars and cents – money made and saved.
6. Present the data formally – create opportunities to give formal presentations; work through established channels such as advisory boards, steering committees, and web councils.
7. Present the data through grassroots methods – develop web analytics newsletters, memos, and user groups; conduct training for senior level managers to introduce them to the business value of analytics and why it is important – tell them how much they’re spending on the web; make sure that there are channels going from the bottom up, and that web analytics is credited as the source for data moving to senior managment.

If you manage analytics, you know that it's not simply an issue of getting key metrics understood by senior staff. Your success in communicating the importance of what you do will also yield bigger budgets, and more resources in good times...and help protect you against downsizing in bad times. 

Managing Web Analytics

I'm real pleased to start blogging for Semphonic. I'll be using this space to write about Web Analytics Management.

What is Web Analytics Management (WAM)?

Let's start with this definition:

Conducting strategic and tactical activities related to planning, coordinating and implementing the web analytics initiative.

OK, but what does that mean exactly?

In my experience, organizations that succeed in implementing web analytics initiatives adhere to a 6 point framework where the following items are addressed strategically and managed tactically:

1. Commitment - from senior management: financial and mission importance/from middle management: resource allocation and program management/from subject matter experts, content, marketing and application owners: using data for driving web channel tactics.
2. Organization - resources and roles that include a senior management champion, web analytics project manager, IT resources (dependent on license/contract and data collection models, as well as data integration sophistication, may include developer, system administrator, database administrator, application administrator), web data analyst and web metrics consumer/stakeholder (content, marketing, finance, operations,IT and so on)
3. Metrics - focus on analysis of site activity associated with revenue, cost savings, avoided cost and user experience.
4. Process - implementing work flows, policies, governance and tools that support metrics development, data collection, quality assurance, data archiving and back up, system administration, user access, report and analysis distribution, and analysis recommendations.
5. Solutions - coordinating integration and use of all web site measurement and testing tools and methods, i.e., web analytics, site performance (i.e., server response time and errors), site compliance (i.e., Section 508), usability testing, focus groups, surveys, panel testing, call center customer feedback, competitive intelligence and so on.
6. Communications - All of the preceding elements are dependent on clear communication...top down, bottom up and across the board.

I expect that you may have a slightly different take on this framework. If you'd like to contact me directly with your thoughts, I'd love to hear from you. Just email me at [email protected]

PS - Some of you may be familiar with my work on the CMS Watch Web Analytics Report. I'll continue to cover vendor selection issues and analysis on CMS Watch.