Web Analytics Management

What the New OMB Guidance on Web Measurement Means for Federal Agencies?

I was overseas when the guidelines on the use of web analytics and measurement on Federal government web site was published on June 25 and then spent a few days tromping in the woods with my son, so I haven't had a lot of time to digest the OMB's final memos. Yet after reading through the memos pertaining to web site measurement (M-10-22) and use of third party websites and applications (M-10-23), like Facebook and YouTube, I feel optimistic about options and opportunities if you're running a Federal agency web site or using web analytics on a Federal web site. When I last wrote about this on May 4, I expressed doubts as to how friendly the policy would be for Federal agency web managers and web analysts. 

Here are the biggest wins for conducting web analytics and measurement on Federal agency web sites:

1. Using persistent cookies is ok. This had previously been a major sticking point for Federal web managers and web analysts who had only been able to identify visitors at a session level. Remember, unique visitors in web analytics means only that you are identifying the computer from where the visit originated...there is no Personal Identifiable Information (PII) being made available. Now you have the opportunity to use your web analytics tools to create segments based on unique visitors, such as first time and repeat visitors. You can apply these segments to referral sources such as search engines, search terms, geographical origination and domains. Now, you'll be able to observe how your content, applications and marketing are used by groups of visitors. With this information, you'll be able to understand more clearly the requirements and usage that visitor groups have based on behavior and be able to develop new content and applications that are more focused on specific user groups. For example, if you can now tell that first time site visitors are most apt to exit the home page, you could consider providing a web site tour specifically for first time visitors and promoting this on the home page; or if you find that you get multiple visits to your search engine from visitors who have been to your site more than 2 times within a month and are using the same terms, you could perhaps use this information to provide more visibility to the desired content. What you need to do: You can not start using persistent cookies today or tomorrow, or next month for that matter. You'll need first to alter your Privacy Policy (see discussion below) to enable visitors to opt out.
2. Using free software that depend on persistent cookies is ok. You can now consider using Google Analytics and Yahoo Analytics! to be viable site measurement options. Use of these tools has been considered off limits or in a grey area at best because of their reliance on persistent cookies. While both provide methods for disabling or minimizing the use of persistent cookies, neither gained much traction within the Federal government.  What you need to do: The same rules apply as described above with regard to Privacy Policy. If you do want to go with the free software approach, please, please, please, spend time considering how you are going to enable access to the web analytics, who is going to use analytics, how are they going to use it, how to train and educate folks on the use of GA or YA, as well as how to interpret analytics data. I strongly suggest too that you determine the important metrics to track and how you'll use this to support decision making.   I can assure you that if you don't plan your deployment of GA or YA with any sort of plan you'll create more headaches for yourself than you have now without any web analytics in place and quite possible damage your management's and peer's buy-in to the value of web analytics for a very long time.
3. You'll be fine as long as you don't collect PII.The guidelines are relatively easy to accommodate as long as you don't want to collect PII, such as names, email addresses, postal addresses. This is classified as Tier3 - multi-session with PII data. Most of you will be concentrating only on Tier 2 data - multi-session without PII. What you need to do: If it is only Tier 1 and Tier 2 data you are collecting, you need to be in compliance with what is outlined in M-10-22. If you do want to collect Tier 3 data, you'll need to work with your Senior Agency  Official for Privacy (SAOP)and then put out for public comment as to why you need to collect such data and then a review by the CIO. This will likely to be a long process.The only exception is getting written permission from your agency CIO to this process if you can make a case for a delay causing serious public harm...which in most cases is not going to happen.
4. You need to evaluate the data collection practice of third party sites, such as YouTube and Facebook.This is one of the most interesting aspects of the guidance...as long as a third party site does not share the PII that they collect with your agency there appears to be no issue, and as long as the third party site does not use their data on behalf of a Federal site, there also appears to be no issue. It also appears that you can use baked in analytics tools, such as YouTube Insight, or blog packages such as WordPress, as these collect Tier2 data. What you need to do:If you are using third party social media sites, you'll need to consider risks as outlined in OMB’s memorandum providing Guidance for Agency Use of Third-Party Websites and Applications that can be found in M-10-23 and prepare a Privacy Impact Assessment (PIA). You will need to work with your agency Senior Agency  Official for Privacy (SAOP) for approval of the PIA, so find out the process and expectations of documentation you will be required to provide.
5. Data retention is flexible.There are guidelines for retention of web data, but it is not an explicit mandate. According to the policy, agencies should limit the retention of data to one year or less. However, it would seem that there can be reasonable arguments made for keeping data for longer, such as wanting to understand year over year trends.What you need to do: Consider your current data retention time. Is it necessary to keep analytics data beyond a year? While many agencies keep data for years, it may not have business value. I'd suggest that  you start planning your data archiving and expiration strategy, if you don't already have one.
6. Web site measurement opt-out options focus on Privacy Policy communication. You need to make it easy for site visitors to opt out of web site measurement. Fortunately, there are a number of options at your disposable, and they do not require you to work with your web analytics software vendor to make them happen. The good news is that as long as you're in compliance with what is outlined in M-10-22 and make your privacy policies very clear and easy to find, you will have fulfilled the OMB requirements. Attachment 3 in the M-10-22 document provides a check list of items that need to be addressed in the privacy policy. The caveat here is that it may take multiple rounds of revisions and approvals to get privacy policies approved. What you need to do: You should understand the process for revising your agency privacy policy and who or whom provides final approval. While the OMB policy sounds relatively straightforward from a content perspective, the actual process in your agency may not be quick nor easily deciphered. On the content side, here's what you can do to provide the opt-out measures:
   

• Use web analytics software that enables visitors to opt out, such as Google Analytics, Omniture, Yahoo! Web Analytics and Webtrends. The caution here is that if you allow the GA or YWA opt-out, you'll be providing visitors with the option to opt-out of Web wide measurement for these solutions. Read my post on CMSWatch for a more detailed description of this issue.
• Provide clear instructions to visitors on how they can opt out of web site measurement on your site. From my perspective, this is clearly the better option to take. USA.gov provides a nice central location with instructions on opting out of measurement on popular desktop and mobile browsers. This is a great reference.

So with all of this seemingly good news are there any downsides to the policy. Well, it may depend how you read it. I think the biggest grey area is around the Privacy Policy discussion in Attachment 2. The memo states that you can use web measurement and customization technologies for Tier 1 and 2  as long as the agency is in compliance with the Memo, there is clear and obvious notice in the Privacy Policy citing the use of technologies and there is compliance with internal policies governing the use of such technologies. Some may see this as needing to get permission or needing to amend the Privacy Policy to maintain even their current web measurement. I don't think this is the intention of the policy and I certainly wouldn't recommend approaching implementation of the policy in this way. However, a quick review of about a half dozen Federal agency web site Privacy Policy statements tells me that there are no policies currently in full compliance with the new additions described in Attachment 3 of the memo. What you need to do: From a practical perspective, it wouldn't make sense to shut down web analytics across the government, however, to really beat home this message, if you are in charge of web analytics at your agency or are using web analytics within your agency, you need to make it a priority to determine how your Privacy Policy needs to change, the process for enabling the change, and who you will need to coordinate with in order to make this change possible.

Final Take: While the policy itself looks like it is fair and provides Federal web site managers and web analysts with long awaited flexibility, we know that implementation of the Memorandum, may not be smooth sailing. As the cornerstone to much of the flexibility is derived from rewriting web site Privacy Policies, swift realization of the Memorandum's benefits could get hung up in long rewrite and review processes. I suggest taking a pro-active approach to determining how your site Privacy Policy needs to be revised and understanding who is involved in approval and the process for obtaining approval. These will be the most challenging aspects to making the Memorandum guidance real.

Getting a Grip on Digital Analytics

One of the themes we talked about at the 2010 Vision webinar was how a strategy around analytics is going to move from being a "nice to have" to "need to have" component to online marketing strategies.

With the proliferation of new digital channels such as social media, video and mobile, the management of data and then enabling analysis and action from that data is going to become a much bigger challenge.In a few years, we'll look back on this period as the end of the slow days of online...when all you needed to do is concentrate on building your web site and bring people to the site to conduct business.

We're now entering the age of The Data Tsunami. If managing the Tsunami is something that you grapple with day to day, and/or think about late at night, you might like to take a look at the white paper I just published:

Getting a Grip on Digital Analytics
A Plan for Driving Online, Social Media, Mobile and Video Strategy

I'd like to hear what you think of the centralized and cooperative plan I've laid out, especially the need to appoint  an "analytics czar" to bring governance and vision to the management of digital analytics.

 

The idea of creating a centralized office or C-level person for analytics may sound somewhat "pie in the sky" especially if your organization still has a silo style method of dealing with web analytics, or user experience testing, for example. Or, your organization may consider that mass distribution of Google Analytics is in of itself a strategy.

The availability of free Web analytics tools has not necessarily led to an increased use of analytics data to make business decisions. Even though Web analytics is available for free, the mistake that many make is that they believe that having the software is enough to provide answers. It’s checking the box off: Yes, we have Web analytics; now let’s move on.

As we move forward in 2010,I think it's clear that organizations that have not grasped the concept of analytics centralization and governance will be at a significant competitive disadvantage due to their inability to move digital data out of silos and into the hands of senior business executives, as well as managers who run the day to day marketing, commerce and content operations.

I hope the whitepaper will give you something to think about and hopefully help you plan for managing the Data Tsunami. Let me know what you think.

By the way...we'd love to see you at the Semphonic Think Tank in Wash DC on May 12-13. Featuring in depth Omniture training, Voice of Customer and analytics integration, and lots more.

Let’s Party Like its 1999!! Web Analytics 2.0... the new Web Analytics 1.0?

In Eric Peterson's Web Analytics 2.0 keynote at X Change, he references “The Good Old Days” of analytics in a Web 1.0 world. In light of the fact that I started doing web analytics in 1995 and was an early evangelist for the use of Internet technology, I found this reference very interesting and somewhat telling about the state of web analytics practice.

In the Web Analytics 1.0 world, when log file tools ruled the web world, let’s say 1995-2000 (this is arguable of course, but lets not get distracted), it was clear that you needed more than tool driven analytics to understand whether your web site was accomplishing its business objectives. In my first web site management role, I relied on WebTrends and NetAnalysis plus usability testing, surveys, focus groups, and interviews, as well as cost s to determine whether the web sites were making money or saving money - I had to in order to justify the existence of the sites.  I considered this to be part of a web site measurement and testing methodology of which web analytics was a part, but not the sole component.

So, when Eric suggested that Web Analytics 2.0 demands these methodologies, in addition to only looking at the numbers, I was rather surprised that this should be considered groundbreaking.

On one hand, there is already activity in the combination of quantitative and qualitative data. Interestingly, Federal agencies have been doing this for some time by combining survey results from companies such as Foresee and Optimost, panel data from Hitwise and web analytics. It could be argued that Federal agencies are in a Web Analytics 1.0 world, but as I learned from my own experiences that might make for potentially more of a complete analysis if managed effectively. At the DC Web Managers Roundtable in December, we had a panel discussing how to use qualitative and quantitative measurement. You might find the podcast an interesting listen to get a perspective on how folks are approaching this.

On the other hand, in many organizations, the practice of web analytics is simply driven by the purchase and use of tools, rather than the use of analysis. Just look at the job reqs that come out of organizations today…consider how many focus on x years of experience with a tool. How many reqs ask for someone with experience in quantitative and qualitative measurement and analysis? How many look at wholistic web site measurement and testing as part of a comprehensive web channel strategy? So with all of the tool sophistication that we now have when compared to 10 years ago, is it possible that web site measurement has gotten more narrow? Have the tools driven the analysis rather than analysis driving the tools? I'd have to say yes, the focus on numbers and stats has become somewhat of a distraction to seeing the whole picture.

If you manage a web channel, quantitative and qualitative testing should always be part of your web site measurement and testing strategy. While this is not new in theory (I discussed this in my book, The Executive’s Guide to Web Site Measurement and Testing), if you haven’t yet put it into practice, you’ll find that it’s a method that will stand the test of time from Web 1.0 going forward.