Web Analytics Management

Semphonic’s X Change and Web Analytics Conference ROI

Getting the budget to attend conferences is not what it was a year ago…or even 6 months ago. So, if you want to come to this year's X Change Conference in San Francisco on September 8-11, you likely need to make a business case to attend. I've already spoken to folks who said they'd like to come, and couldn't get the budget…but perhaps they really weren't making the right pitch…or perhaps they were willing to just take "no" for an answer.

Just like anything else related to analytics, I think there needs to be a return on investment (ROI) story to tell. If you can't tell that story, then I think you'll ultimately fail in your mission.

So, let's do an ROI analysis for your attendance at X Change this year:

• Cost for X Change if you sign up by July 31 with Early bird discount: $1,800
• Cost for Think Tank Sessions: $599
• Hotel if you sign up by August 10: $299/night
• Airfare: Deals to be had - check out Southwest - be creative
• Food: Breakfast, lunch and one dinner…all high quality. It's the freakin' St. Regis after all!!

Ok…so your costs: Probably somewhere between $3,000 to $3,500

Now, let's take a look at what you'll come away with on the knowledge side, and how this will end up in either revenue generation or cost savings for your web analytics initiative.

First, there are the "huddles"…

The conference is built around small group discussions called "huddles" that are facilitated by practicing web analysts from top organizations. There are some immediate ROI benefits from this:

We don't sit in a room and listen to "talking heads," or consultants, and we don't feature any vendors or products: Calculate your hourly rate by the amount of time you space out in most other conferences, or spend time emailing or texting.

Hear firsthand what your peers, even your competitors, are experiencing in using analytics to run their web channel: Calculate based on hourly fees charged by consultants.

Test and discuss your own ideas on how to improve the use of analytics within your organization: Calculate based on trying to pull a group of seasoned web analysts like yourself into a room for a serious discussion with no interruptions.

Compare the cost and takeaways to other training: Calculate the training classes you've attended over the past year, and associated measurable benefit. If you're at a management or experienced web analyst level, we would suggest that most of what you've attended you either already knew, or it was not exactly what you were seeking.

Or, take another approach...start by taking a look at the huddles:

http://www.semphonic.com/xchangehuddles.aspx

Read through the synopses with an eye towards how you'll apply the learnings of the session to your web analytics goals and objectives. For example, if you attend the Measuring Mobile Internet huddle facilitated by Nokia's Greg Dowling, you will likely come away with a wealth of knowledge on how to choose the right measurement solution, how to develop valid metrics and how to monetize them. Or, you can attend the session on how to build a web measurement capability from Gil Roeder at Barclay's IShares, to really understand how to successfully integrate analytics into the mainstream of your organization…so that both senior and line managers use data to run the business.

I think you could you make the case that the takeaways from these huddles could translate into tens of thousands of dollars in more productivity, or savings on a solution. If you find it a stretch, then get in touch with me and let's talk about it. I'll be happy to help you make the case…pkemelor@semphonic.com

These are just some of the reasons we hear that X Change is one of the best values from a cost/learning perspective. To get your own sense for X Change, you can read about the 2008 conference from a few attendees:

http://www.semphonic.com/xchangerecap08.aspx

And then there is the Think Tank…the real ROI opportunity

While the emphasis on X Change is on peer learning and dialogue, we are trying something new…running a series of advanced training seminars, called the Think Tank. It's a day of classes, each are 1 ½ hours and will cover advanced topics in data analysis, behavioral segmentation, Excel reporting and implementation, how to advance the influence of analytics on a shoestring, and a lot more.

Coming back to my previous point about web analytics training, I think you'll agree that there just really isn't a lot of educational opportunity for people like yourselves who are at a management or advanced analyst level. The Think Tank sessions will address these…3 master level classes for only $599. I don't want to sound like a car salesman, but this is a screamin' deal!

I think the ROI story for the classes alone is quite strong. The class I'm teaching…is all about how to build out a web analytics presence with limited resources and budget; how to build cross-team alliances, how to create an effective business case…all while expanding the impact of web analytics on your organization. I'll be focusing on a range of strategy and tactics that I've used as a web analytics program manager and with our clients.

The rest of the classes will be taught by the Semphonic team including Gary Angel, Paul Legutko, Jesse Gross, June Dershewitz, as well as by Eric Petersen. Find out more about the Think Tank sessions at: http://www.semphonic.com/xchangethinktank.aspx

By now, you should have your conference ROI calculations all mapped out and ready to go. Just make the case.

We want to see you in San Francisco in September for both X Change and the Think Tanks. Just register at http://www.semphonic.com/xchangeregister.aspx.

Only 63 days left til X Change.

Web Analytics Insights from JBoye 09

While the rest of the web analytics folks attended eMetrics in San Jose last week, I had the pleasure of participating in JBoye’s Philadelphia 09 conference. This was the first US conference for JBoye; Janus started these conferences in Aarhus, Denmark in 2005 as annual user group meetings for his firm’s Communities of Practice…a really great concept in which practitioners with similar interests but in different verticals exchange experiences and learnings on a monthly basis.  I’ve had the good fortune of presenting and teaching at the Danish conferences, and had a great time doing a joint session at last week’s event with Susan Fariss, Web Analytics Manager from the American Chemical Society on taking a strategic approach to web analytics implementations.

One of the reasons I like the JBoye conferences is because most of the folks are content managers, information architects and user experience professionals and there is a spirit of conversation, questioning and inquisitiveness that gets me thinking outside of the web analytics box. It also gives me a good perspective on how non-web analysts deal (or not) with web analytics.

Some of the takeaways this time:

1.      We (fill in the blank) get no respect

Recurrent themes of a lack of appreciation, respect or influence articulated by content managers, user experience folks and information architects within their organizations would resonate with web analytics people.

A thought: Build more cross-discipline support and help each other out in developing business cases that get joint initiatives funded and recognized by senior management. For more ideas, check out: http://wam.typepad.com/wam/2007/09/california-drea.html and http://wam.typepad.com/wam/2008/05/its-been-awhile.html

2.       Soup or Art?

Have you seen Lily Tomlin’s “Search for Signs of Intelligent Life in the Universe”? Great show…one of the memorable bits was her holding up a can of tomato soup and asking the crowd if it was just soup or art (a la Warhol).

I saw something similar in Erik Hartman’s and Bob Boiko’s presentation about an Information Management Framework  and trying to get content managers to think of themselves not only as managers of Content Management Systems, but Information Managers who are responsible for providing quality information to their stakeholders.

In the same vein, I thought of how many of us in the web analytics space spend a lot of time talking about tools and techniques, rather than the information and analysis we provide. Something to think about...  

 3.     What am I missing?

I got into the concept of web analytics and qualitative techniques at roughly the same time I started managing web sites…1996. I used WebTrends, surveys, interviews and usability testing to change web site content, navigation and interface design…this just seemed like the thing to do. So I guess I’ve never understood why analytics seems like a foreign concept to those on the content side of the house.

 I was heartened that this seems to be changing...most notably in Erik and Bob’s presentation as they discussed the importance of content monetization through modeling in non-commerce scenarios and in Lou Rosenfeld’s keynote on the need for User Experience and Web Analytics to be used together. If you’d like to get some ideas on how to combine qualitative and quantitative measurement methods, and how to develop monetization models for content, you might want to take a look at my book The Executive’s Guide to Web Site Measurement and Testing. I think you’ll find some useful stuff there. There was even some talk of the need for one role to bring all site measurement disciplines together…a concept that I think makes a lot of sense as organizations seek to get out of site evaluation and testing silos.

Check out my post http://www.cmswatch.com/Trends/1592-Web-Analytics-Vs-Design about the discussion over Douglas Bowman's leaving Google over a tiff about web analytics.

The next JBoye conference is scheduled in Aarhus at the beginning of November. I suggest if you’re looking for a conference with a bit of personality, you should check it out….speaking of conferences with personality, remember to put X Change on your calendar for September.

 

 

 

 

 

Compare web analytics vendor compliance with the Web Analytics Association Analytics Definitions

As part of the research that went into the 2009 CMS Watch Web Analytics Report, I asked the vendors about their compliance with the WAA Web Analytics Definitions. If you'd like to see how the vendors stacked up you can download this for free, as well as read my post on the findings at:  http://www.cmswatch.com/Trends/1575-WAA-Standards-Compliance

I'd be interested in hearing from anyone to see how they've used the Definitions in their work...whether it be for vendor evaluation, stakeholder or partner education, or for other purposes.

An Enlightened Federal CIO Meets Web Analytics

Last week I had a great conversation about web analytics with the CIO of a federal agency.

Why was it a great conversation?

 

He understood the importance of web analytics to his organization, and made the commitment to support web analytics with his resources and authority.

 

He stated that web analytics is a discipline that all content managers need to use to plan new content and remove old content. He views web analytics as a “value add” service that his office can offer to internal stakeholders. He committed to having his development team learn how to tag web sites, and wants to encourage site owners to initiate web analytics on their sites. He understood the business benefits of his IT staff moving out of the log file processing business and into a SAAS solution that enabled better access to analytics data by potential analytics consumers throughout the enterprise.

 

Yes, I found this conversation to be somewhat of a revelation because the Federal government is by and large at a very early stage in using web analytics. This is an environment where older log file solutions are commonplace and entrenched, where IT departments still “own” analytics, where reams of page views and visit reports define web metrics. I speak with Federal web analysts who have literally spent years trying to institutionalize web analytics at their agencies and have met extreme resistance and/or apathy.  The common denominator…no senior management support.  True, lack of senior management commitment hampers many web analytics programs in the private sector, but the impact within the Federal sector seems to be more extreme. Historically, there has been no revenue-based goals driving an interest in web analytics, so a lack of senior management focus tends to leave web site managers in a place where they need to determine priorities for themselves. Fortunately, this seems to be changing due to tighter budgets, and zero based budgets (ZBB). According to Forbes Investopedia:

 

ZBB allows top-level strategic goals to be implemented into the budgeting process by tying them to specific functional areas of the organization, where costs can be first grouped, then measured against previous results and current expectations.  

 

If Federal CIOs needed a business case for institutionalizing web analytics as a program office, the ability to tie all web initiatives to a ZBB model and initiate accountability through meaningful web metrics seems to make a lot of sense.

 

Will Federal CIOs take a leadership role in developing web analytics programs at their agencies? I hope so. While there has been a lot of attention about cookies and privacy, I think that in the scheme of things, this is a somewhat overstated issue when it comes to making web analytics useful within the Federal government. The issue of importance as I see it is in establishing viable and relevant web analytics programs within Federal agencies. After all, if you resolve the cookie or no cookie issue, and there’s no senior management commitment to support a web analytics organization, no one to implement the web analytics solution, develop meaningful metrics, create reports and provide analysis, then what’s the point whether persistent cookies are used or not?

 

75 of 2009

It’s been a busy first few months of 2009 and I wanted to tell you about some of the things I’ve been working on…also wanted to pass along a discount to eMetrics, a reminder about the WAA Public Sector Survey and a heads up about an upcoming presentation I’ll be making in May.

• In mid-February, I finished up the CMS Watch 2009 Web Analytics Report. This is the third edition of the report, and I added coverage of video and mobile specific vendors. Also, in response to the WAA’s review of the 2008 report, I asked all vendors to provide their take on WAA Metrics Standards compliance. Out of the 20 vendors that were covered, 10 complied. It was heartening to see that both Google Analytics and Coremetrics went public with their compliance soon after I made the request…and disappointing that more vendors did not. You may have your opinions about the relative value of the WAA Standards, but I think the effort that vendors do or do not put into transparency speaks volumes.
  

• If you’ve been reading my blog posts since the beginning of the year, you’ve noticed I’ve become quite interested in the question of web analytics data privacy and access. To expand on this often overlooked topic, I’ve teamed with CMS Watch founder Tony Byrne to write Do You Really Own Your Web Analytics Data? I’d love to hear what you think about this relative to your vendor selection decisions.
  

•  I mentioned in my post last week about the WAA Public Sector Survey. If you haven’t taken it, there’s still time…in fact you have until Monday, March 23. So, if you’re a .gov, .org, .edu or .mil, take 10 minutes and contribute to the knowledge-base…I’ve been tracking the results and seeing some very interesting trends and findings. This leads me to…

•  eMetrics in San Jose. We’re hoping to have preliminary findings in time for the big, big show May 4-7. And if you use this promo code WAMANAGE12 when you register, you’ll get 12% off your registration. If you register by this Friday, March 20, you’ll get an even better deal with Early Bird pricing! Go to: http://www.emetrics.org/sanjose/2009/register.php

Finally, if you can’t make it to eMetrics, join me on May 7 in beautiful, downtown Philadelphia, where I’ll be speaking at the JBoye conference.  My presentation, Strategic Techniques for Improving Your Web Analytics will show you how a well executed web analytics strategy leads to successful analytics solution implementations. The conference runs from May 5-7.  To register and find out more, go to: http://www.jboye.com/conferences/philadelphia09/schedule.

Web Analytics Association Takes on the Public Sector

Do you practice web analytics for a government agency, a non-profit organization, an educational institution or a branch of the military? If you do, what types of metrics and reports do you run? Are most of your reports generated in an ad hoc manner for spot analysis for senior management? Or do you focus on creating a self-service environment in which content managers and marketers act as “power users” who can take care of their own analytics requirements?

 

The Public Sector Special Interest Group of the Web Analytics Association (WAA) started asking these and other questions  both to find out whether web analytics in the “non-private” sector is really that different than the private sector, and to help web site managers, marketers and analysts develop benchmark metrics, usage and presentation methods that are common to this corner of the web universe.

 

The result is the Web Analytics Association (WAA) Public Sector Web Analytics Survey.  I like to think that this is the first step in developing a greater presence and recognition for this segment, which often seems to be regarded as somewhat of a “backwater” by the larger web analytics community. I’m hopeful that the survey results and communication of the results will help foster a greater dialogue between the public and private sector web analytics communities.  And wouldn’t it be cool if there was enough participation among public sector organizations to ultimately foster the type of index we’ve seen in the retail area, such as the Coremetrics Benchmark?

 

If you work for a public sector (including military), non-profit or educational organization, take the survey now; you don’t need to be a member of the WAA.

http://www.surveymonkey.com/s.aspx?sm=QWRWnODN1hbqLUaNyy52Lg_3d_3d

 

Who’s Web Analytics Data is it Anyway…Part 3

Since I wrote Whose data is it anyway? and Whose data is it anyway? Part 2 I've started to dive a bit deeper into the subject  of web analytics data ownership as part of my vendor research for the just released The 2009 Web Analytics Report and because I'm managing a vendor selection process for a Federal government entity.

It’s a complex issue and one that often gets overlooked during web analytics vendor selection and contract negotiations. Many analysts and web managers whom I speak to at large organizations either don’t read or don’t have access to their vendor contracts…and generally don’t ask about data ownership during the vendor evaluation process.  Many people just assume that they own their web analytics data.

So what does data ownership really mean? Is it simply a “concept”, or does it translate into anything of business value? Well, it depends on your vendor…what they say, what they don’t say and what they put in writing.

Let’s look at the four aspects of "data ownership":

Use of Your Data

As we’ve already written, Google Analytics and Yahoo! Web Analytics state in their contracts that may use your data for a variety of commercial purposes. Not all, but a few of the fee-based vendors state that they’ll use the data for benchmarking or industry average information. This seems to be a more palatable pill to swallow for large, security-concerned enterprises, such as those in government and financial services.

Data Security

All of the largest web analytics vendors have gone to great lengths to ensure the safety of your data from the perspective of storage, access, and redundancy and server failover. Yet if there’s a data loss, you’re on your own. All agreements absolve the vendors of any liability. So, in this case, what does data ownership really do for you? Not much.

Data Retention

Being able to run reports on historical data and having access to unaggregated data as far back as your contract runs may be considered to be a sign of data ownership, and one that many take for granted, but it is nothing of the sort. There are basically two types of web analytics data you can retain: data that is used in creating summary tables used for the reports you run in the core solution, and unaggregated data that makes up your web analytics database. The length of time that this data is kept may or may not be in the contract. We find this to be part of your service agreement, and can be completely negotiable. Your vendor will have a default length of time for both types of data…typically 12-36 months. You want more than the default, you can pay for it. We know of one fee-based vendor that breaks out default retention on an hourly, daily, weekly and monthly basis…just a bit more complicated and just another way to add to your cost of ownership.   The upshot of this: Data retention has real meaning for what “data ownership” is all about…or does it?

Data access- API or file extraction of core data tables for export to another system.

Before you get all excited about making a deal for keeping your data for 36 months, you have to think about how you’re going to export the data out of the web analytics solution. While it may be easy to pull old reports, what if you want to extract the unaggregated data to export to another database, or what if you want to use it with a business intelligence tool? How do you get it? Vendors that provide a data warehouse capability, are happy to sell you this capability and therefore your underlying data is available and extractable…typically through APIs. But if you don’t purchase a data warehouse capability or your vendor doesn’t offer one, you could get your unaggregated data in a state that is really hard to deal with…it could be unprocessed, meaning that the data is in a “log file” state, or it could be in a CSV format. In either case, you’ll have a lot of work to do to get the data in a state that is usable.

So, what’s “data ownership”? I think you have to figure out what elements of data ownership make the most sense. From my perspective, making sure that you can get your hands on your web data when you need it and with as little interference as possible are prime considerations. Security and privacy are paramount, but if you choose to go with an on-demand solution, you’ll need to be assured that the vendor meets your criteria, knowing that if there’s a data loss, you’ll have little recourse.

Whose data is it anyway? (Part 2)

I wanted to spend a bit more time on the subject of web analytics data usage per my post last week, or rather spend some time discussing the comments I received from Stephane Hamel and Jacques Warren.

Jacques makes the point about lack of transparency into “free tools”:

The fact that I can't have access to the raw data, the logs, has bothered me everytime I came across a problem. I just cannot accept that the only validation criterium of the numbers I'm seeing is... the brand! “It's right, because Google says so.”

Now, it is true that Jacques, who is one of the smartest web analysts out there, probably understands the intricacies and nuances of web data more than most people, but what I find ironic is that issues around data trust are common to many organizations, and it is often articulated by managers who have no background in web analysis. They simply don't understand how web analytics works. So, if you use Google Analytics, how do you address core data accuracy questions? Is it easier or more difficult to do than with a tool that has more apparent data transparency?

I also find it interesting that many organizations use Google Analytics to “audit” their fee based tool. I don't quite get the logic in this. How about doing an audit of the data collection for the fee-based tool that has already received investment?

In an odd turn of events, I have to wonder if companies' dual deployment of Google in addition to the fee-based solution has actually helped perpetuate poor implementations of fee-based solutions. Perhaps it is coincidence that it used to be that when organizations got frustrated with their analytics solutions, they'd be more prone to switch solutions, but it now seems that they simply use Google Analytics in tandem. I've got nothing other than anecdotal evidence to prove this, as well as using Stephane Hamel's WASP, but I'd be curious to hear what you have to say about this.

Stephane's comment makes a good point regarding the concept of web analytics maturity within an organization and use of the “right” solution. This is something we've sought to educate readers of the Web Analytics Report through the use of business and resource commitment scenarios. I also share Stephane's concern that “as the power of Google increase, the options are becoming scarcer.”

Then again, while this economy may impact companies well recognized in the analytics space, I think that there will be a growing number of organizations and analysts who will mature into appreciating the value of fee over free.

Jim Sterne...it's not what he says; it's how he says it

A few weeks ago I had the pleasure of seeing Jim Sterne speak at the Web Managers Roundtable meeting here in DC.

 

Now, I’ve seen Jim speak on a number of occasions. The first time was during the famous Web Metrics road show with Matt Cutler. That was back when I was with NetGenesis. Since then,  I've seen Jim's presentations a number of times. But I never tire of watching Jim speak, even though I’m familiar with his material.

 

Why?

 

Because Jim is a great communicator who is a master at talking about web analytics in plain language and business driven simplicity…In other words, exactly how to talk to business people, especially senior managers, about web analytics. This is exactly what I strive for in my presentations. So I always learn from Jim no matter what he's talking about.

In my mind, being able to convey the value of web analytics as Jim does, is in some respects, more crucial than the data itself. If you can do that, you'll convince a lot of people that web analysis is the key to driving web site strategy.  And as my colleague, Gary Angel points out, this extends into developing an accepted language around web measurement, which ultimately leads to raising adoption rates for web analytics throughout the organization.

 

So, when you come to eMetrics in Washington, DC from October 20-23, make sure you catch Jim’s keynote…not just for what he says, but how he says it.

Is Google Analytics the solution for non-use of web analytics?

What would you do in this situation?

 

You manage an international deployment of Omniture SiteCatalyst with about 2000 stakeholders, but a lot of these folks don’t use the reports.  In fact only about 200 are really active SC users.  Not only is this frustrating, but it is costly from a resource overhead perspective to develop and maintain report sets that aren’t being used. So you think, maybe you should deploy Google Analytics for the non-active stakeholders. They’d be able to get the basic set of reports with no intervention from you, and you could concentrate on managing requirements for the 200 power users.

 

This was the scenario someone presented to me at one of the “one on one” sessions at the recent X Change.

 

I asked whether there was much in the way of current interpretation or education provided to the report stakeholders. There wasn’t. Based on this I said, “I don’t think that deploying Google Analytics is going to resolve the issue; I’d focus on stakeholder education.  Besides, deploying Google Analytics won’t necessarily cut down on your support issues if you need to customize the Google Analytics tracking code to do more than basics. You’ll likely be needing to answer lots of questions, or dealing with confusion about what people are seeing, and so forth.“

 

We continued to chat…even if Google Analytics was deployed, the need for stakeholder education would still exist.  Thinking about this now, a 2000 stakeholder deployment is quite a staggering figure, and I’ll admit to never having faced anything of that scale, but I’d approach it with an emphasis on communications and education, and take a phased approach. This is how I have deployed web analytics, content management, email, web and other software throughout my career, and it seems to help ensure acceptance and result in value for the people using it.

 

With that in mind, I wanted to share the third book of my Summer Reading series…the others being Mental Models and Multichannel Marketing. This last book, Words That Work was written by Frank Lutz, known for his political pollstering work. I listened to an interview with him some months ago, and was pretty fascinated by his take on communication, and speech, and how people relate to what they hear.  It’s interesting to me that while we talk about Voice of Customer being so important for our external audiences, we sometimes forget about our internal customers…web analytics stakeholders and what makes sense to them. Pick up the book and read the first chapter…you’ll get most of what you need there. The rest of the book is ok…sometimes a bit repetitive and heavily emphasizing the nature of political communication, but the first chapter will be a good foundation for how to communicate web analytics to people in your organization.

 

So, what would you? Throw the masses Google Analytics? Spend time on education and training? Both? Neither?