Web Analytics Management

So, will Web governance make your digital analytics program run faster, jump higher and make you smarter?

To conclude this year's Profiles in Analytics - Does Web Governance Drive Analytics, I wanted to leave you with some general takeaways on how to move towards a better Return On Investment on your program...regardless of whether your organization is driven by a solid governance model or not.

So, will Web governance make your digital analytics program run faster, jump higher and make you smarter?

Like any programmer will tell you “it depends.”

The Profiles in Analytics research indicates that enterprise-wide Web governance is generally a good thing. The operational processes, the roles and responsibilities, and the overall standards that come out of governance help get everyone in an organization on board with implementing strategy. This appears to help the institutionalization of digital analytics throughout the organization.

There does appear to be a gap between where enterprise-wide governance ends, and where digital analytics governance begins. Whereas Web governance creates a space for digital analytics to grow, the depth of influence and making real change is only beginning.  The dichotomy between the considered priorities of digital analytics teams and the time spent on tasks shows that there are constraints that detract from analytics reaching its full potential as a way for organizations to learn from their experience in the digital channel.

How to clear the constraints?

• Invest in a Web governance process that provides insight on where digital analytics should sit in the organization. The level of influence that digital analytics has within the organization is tied to the office or department in which it resides and the influence of an executive champion in fostering cooperation.
• Conduct an audit of digital analytics operations to understand priorities, roles, responsibilities, tasks and processes. Senior management and digital analytics management need to be on the same page. There needs to be frank and open discussions about constraints and roadblocks that prevent digital analytics in being an effective voice in providing insights throughout the organization.
• Facilitate cooperation between digital analytics team and potential partners within the organization where there are opportunities to complement and leverage resources. Digital analytics programs that succeed depend on a network of resource support beyond the direct team. IT, Procurement, Marketing, Content Management, Business Intelligence, Finance and Operations are among the groups that a Digital Analytics team must interface with and work with to operate effectively. It helps to have these relationships facilitated by senior leadership so it is clear to all that analytics are recognized as a management priority.

This year’s Profiles in Analytics research suggests that it isn’t enough for organizations to pronounce that they are “data driven.” There is a lot more work to do around overall Web governance, development of analytics management expertise, and figuring out the best place in the organization for digital analytics so that it can have real influence.

Interested in continuing the conversation? Email me at [email protected].

 

Has Digital Analytics Made it into the Digital Production Lifecycle?

Continuing on with Part 5 of this year's Profiles in Analytics - Does Web Governance Drive Analytics...or get the whole analysis, survey data and digital analytics profiles at Semphonic.

Has Digital Analytics Made it into the Digital Production Lifecycle?

For all of the aspirations on how to create digital cultures and data driven organizations, when it gets down to basics, simply making sure that data collection occurs is a back burner issue. While Web governance oriented organizations may have processes in place and top down commitment to ensure that analytics is part of the development cycle, that it is only at around 60 percent says that there is room for improvement.

While non-governance organizations have roughly the same compliance, that there is such a high level of dissatisfaction with the process of implementing code changes, tells you something about an underlying issue that can sabotage whether accurate data is available when senior leadership wants to use it for appraising the role of the digital channel.

The Details

It stands to reason that organizations with established Web governance have more formalized processes around production of new content and applications for the digital channel, than organizations that don’t have strong governance.

Web development process. Nearly twice the number of organizations with governance includes digital analytics in the process “all the time” versus those that don’t, as seen in the chart below. Still, at 25 percent, there is clearly more work that needs to be done in this area. Interestingly, once you combine “all” and “most” of the time responses, you get roughly 60 percent in all organizations.

The research shows that there is still a great deal of work that needs to be done in the area of integrating analytics to the digital development process. While there is an increasing amount of project management best practices aligned with enabling analytics, analytics itself is often implemented after a project has launched. With the advent of mobile applications and social media initiatives, there are new opportunities for using analytics, and at the same time a repeat of the poor practices that have left analytics out of the fixed Web development process.

Implementing data collection code changes. The research showed the biggest difference between governance and non-governance when dealing with the process for implementing changes to data collection code, as seen in the chart below.

Anecdotal research tell us that the application of data collection code, considered to be the least glamorous element of analytics, to be a universal and significant hold up in the analytics process.  Because there are often other projects in the queue ahead of analytics, or analytics is considered expendable if content or applications need to meet a deadline for launch, data collection coding may be held up for weeks and even months in certain organizations.  

The “not acceptable” level of satisfaction among organizations without governance is a situation that if addressed, could show immediate improvements to the availability of analytics and potential value of metrics created with the data.

Next week: So, will Web governance make your digital analytics program run faster, jump higher and make you smarter?

Get the entire research at Semphonic.

Are Web Governance Oriented Organizations “Better” at Using Digital Analytics?

Continuing on with Part 4 of this year's Profiles in Analytics - Does Web Governance Drive Analytics...or get the whole analysis, survey data and digital analytics profiles at Semphonic.

Are Web Governance Oriented Organizations "Better" at Using Digital Analytics?

Enterprise Web governance may contribute to high quality digital analytics for the enterprise, but it does not guarantee success. At the same time, lack of enterprise Web governance doesn’t necessarily doom an analytics initiative to failure. However, programs seem to stop short on delivering all that they can deliver. 

In both cases, analytics programs appear to be primarily centralized entities that focus on dashboard development, reporting, providing analysis and recommendations.  While there is service to multiple stakeholders, the lack of time dedicated to training, encouraging self service and creating analytics standards that may be used throughout the organization fosters a somewhat isolated digital analytics culture. So, while multiple stakeholders are potentially served through distribution of reports, the value of what’s being conveyed in the reports may or may not be understood.

The Details

If you serve more stakeholders more efficiently does this mean that you are providing a high quality analytics product? Perhaps, but not necessarily. This is like saying that a Super Size meal is better than a 4 oz filet mignon. Quantity doesn’t mean quality.

To determine the level of quality, the research focused on three elements:

• Digital analytics program priorities of digital analytics managers
• Whether the program priorities were in line with senior management
• Priority tasks of the digital analytics teams

Top priorities.  In looking at alignment in priorities between digital analytics managers and their management, it appears there is actually more alignment among those in organizations without Web governance councils.

 

 

It is altogether positive that there is fairly strong agreement between digital analytics managers and senior management that the primary mission is delivering recommendations and analysis, even if it is not completely equivalent.

There is a fairly strong sentiment among senior leadership around the importance of developing dashboards and reports in organizations that have Web governance. This suggests that there are a number of senior leaders who believe that success in analytics is based on getting reports into as many peoples’ hands as possible. 

But there is more to this story…

Top tasks. When asked about the tasks the analytics team spends the most time on, there is an interesting disconnect from the goals and perceptions described previously…the prominence of implementation and maintenance of the analytics tools, as seen below.

 

Neither Web analytics managers nor senior leadership wants to consider that this is a top priority. However, lack of recognition of the importance of this task is akin to driving a car and not wanting to get it serviced regularly. Negligence can catch up in a number of ways. In the scenario of mutual denial, analytics teams may not have sufficient resources to provide more valuable insight from analytics.  However, the question is whether they make the business case to request these resources. Conversely, senior leadership may not be aware of the amount of time and effort required to maintain and improve the Web analytics implementation.

Finishing among the top three tasks analytics teams spend their time on, and what also doesn’t show up as a priority is “conducting ad hoc analysis.”

Like implementation and maintenance tasks, there would be agreement that this would not be a priority; however, because so much time is spent on this task, it takes away from potentially other more valuable tasks. It is worth noting that organizations without Web governance indicated that their team spends more time on this than any other task. However, this activity actually occurs more often as a second and third task than non-Web governance organizations.  Reasons for this can range from positive to negative: a focus on conducting deep dive analyses, “drive by” requests, weak or non-existent request management and evaluation process, or a need to improve quality and distribution of dashboards.

Regardless of whether the organization is driven by Web governance or no not, the tasks that could potentially drive high value throughout the enterprise are given the lowest priority among both Web analytics teams and senior leadership. These tasks also get the least amount of analytics team attention:

• Train and educate on the use of analytics
• Provide strategic direction on the use of analytics
• Create organizational analytics standards
• Developing culture of self-service analytics

Next week: Has Digital Analytics Made It into the Mainstream of the Digital Production Lifecycle?

Get the entire research at Semphonic.

Does Web Governance Encourage More Efficient Operation of Digital Analytics?

Continuing on with Part 3 of this year's Profiles in Analytics - Does Web Governance Drive Analytics...or get the whole analysis, survey data and digital analytics profiles at Semphonic.

Does Web Governance Encourage More Efficient Operation of Digital Analytics?

Digital analytics teams in organizations with Web governance appear to be more efficient in serving a larger stakeholder base than those that are in organizations without Web governance. While staff size appears to be somewhat smaller in Web governance oriented organizations, there appears to be slightly more management expertise.  The experience of digital analytics team management coupled with overall organization Web governance could contribute to a more process oriented approach to serving stakeholders. This would enable more stakeholders to benefit from the use of digital analytics services.

In addition, stronger management of digital analytics teams could also lead to a tighter definition of roles and responsibilities within the team. This could foster greater specialization among team members that could be manifest in more efficient completion of tasks.

The Details

Digital analytics teams in Web governance-oriented enterprises appear to serve more stakeholders than those organizations without Web governance.  As indicated in the chart below, digital analytics teams in organizations with governance appear to serve more stakeholders in all categories except one.

Of course, some of the difference in stakeholders served can be attributed to organization size…larger organizations have more stakeholders. However, the research suggests that the influence of Web governance on staff size and management experience also plays a role in how many stakeholders are served.

Analytics staff size. Organizations with Web governance had twice the percentage of large staffs with 10 or more people. However, these enterprises actually had smaller staffs than did organizations without Web governance in the mid-range (2-10 staff size).

Digital analytics management experience. From a title perspective, managing a digital analytics team is largely a middle management function, with Director and Manager the most frequently indicated titles.

However, it is in the area of both analytics and management experience where there are potentially significant differences between teams in organizations with and without Web governance.

Organizations with Web governance tend to have managers with more management experience than analytics experience. Organizations without Web governance tend to have managers with more analytics experience than management experience.  A significant plurality of organizations, 30 percent, has managers with both ten and more years of analytics and management experience.

This overweighting of technical and management experience appears logical in context with the characteristics of the types of digital analytics teams one finds in large and Web governance-oriented organizations vs. those that are smaller and not inclined to Web governance. Managers in large organizations appear to be more involved with  strategic and operational functions; in smaller organizations with less staff, managers are in a more practitioner/manager role.

Is there an approach that works better? If there is an organizational strategy to scale analytics across the enterprise, than having more experienced managers who know how to manage programs and staff and are comfortable interfacing with senior leadership makes sense.

 

Next week: Are Web Governance Oriented Organizations "Better" at Using Digital Analytics?

Get the entire research at Semphonic.

Does Web Governance Drive Digital Analytics?

Continuing on with Part 2 of this year's Profiles in Analytics - Does Web Governance Drive Analytics...or get the whole analysis, survey data and digital analytics profiles at Semphonic.

Larger Organizations Invest in Web Governance

By their nature, large organizations recognize the need to invest in operations governance as a means to manage processes and people efficiently.  Web governance is a natural extension of a governance culture that already exists.  Smaller organizations tend to have less process and a flatter management structure. The concept of Web governance may run counter to how the organization approaches overall operational processes. In addition, as smaller organizations tend to run leaner from a staff perspective, they may not have the resources to dedicate specifically to Web governance.

The Details

There is a relationship between investment in Web governance and an organization’s size and annual revenues. Larger organizations tend to be more inclined to have Web governance councils than smaller organizations, as seen in the charts below.

51 percent of the organizations with over $501 million in annual revenue have Web governance councils compared to 17 percent that do not. On the other end of the spectrum, 77 percent of the organizations with less than $500 million in revenue do not have Web governance councils compared to 24 percent that do.

The trend towards Web governance is also borne out based on number of employees. 66 percent of the organizations with over 1,000 employees have Web governance councils compared to 25 percent that do not and 44 percent of the organizations with 100 or fewer employees do not have Web governance compared to 11 percent that do.

 

Next week: Does Web Governance Encourage More Efficient Operation of Digital Analytics?

Get the entire research at Semphonic.

 

What Does Digital Analytics Have in Common with Content Strategy, Information Architecture and User Experience?

I was in Palm Springs last week giving a workshop at The Rockley Group’s Intelligent Content 2012, a conference for Web content strategists, user experience specialists, technical writers, and information architects. Yes, I was likely the only “official” digital analytics person in the house.

Like web analytics folks, content strategists talk about many of the same issues when it comes to being part of the larger digital or web program: not in the loop, not given enough respect, business value of the work is not understood, and so on

Kristina Halvorson, founder of content strategy firm, Brain Traffic and one of the thought leaders in the content strategy space took this issue on…and I liked her perspective. Among the points she made, and one that digital analytics professionals could also learn from comes down to this:

Be open, interested and seek cooperation with others in the organization who are working on digital initiatives.

I agree wholeheartedly with this.

Could you do anything to smash the silos in your organization? Is there anything you could do to build bridges for cooperation? I think this comes down to learning from each other and appreciating that there is much common ground.

So, with this in mind, I’ve decided to do a time-release of my Profiles in Analytics research over the course of the next 5 weeks. In this year’s study, I looked at the influence of overall Web governance on the use of and integration of analytics throughout the organization.

How does this relate to what I described above?

I think web specialties like analytics and content strategy are left out because there is no overall process governing web operations. When you have governance, there is simply a better chance for everyone to be included in the “big tent”; less of a chance of being forgotten about.(If you want to learn more about Web governance, I recommend that you get to know Lisa Welchman at WelchmanPierpoint)

I believe governance begets good process and with good process there’s a place for all of the essential elements of development to be included… at least that’s what I see among my clients…especially in enterprise level organizations.

However, if there is no top down governance from senior leadership, or if it isn’t strong, then folks who are doing the work in content, IA, user experience, and analytics, ought to do more to organize on the grassroots level and work as allies.

 So, today, I’ll kick off with the research premise:

What Does Web Governance Mean to Data Driven Organizations?

Not many would dispute that organizations need a Web strategy to be successful. When it comes to execution, operational governance is considered the key to getting the organization to act on the strategy. Governance takes the strategy and makes it real through alignment of roles, responsibilities, management policies and budget decisions. 

It follows that if an organization has both strategy and governance in place, then it would have a firm grasp around how to integrate the support of and use of digital analytics into their business process.

From a best practice perspective, organization-wide Web governance provides the digital analytics function with a presence in the overall decision making structure. It also provides clarity in defining the role of Web analytics within the organization.

But does this actually happen?

55 percent of the organizations who took part in this year’s Profile in Analytics survey indicated that they have Web strategies. Yet only 30 percent indicated that they have Web governance councils.

There certainly seems to be a relationship between governance and strategy.

Of those that have a Web governance council, 60 percent have a Web strategy in place.

Of those that have no Web governance council, 46 percent have a Web strategy in place.

In the following analysis, we compared how “governance” and “non-governance” organizations managed and used digital analytics.

The question is: Are the 30 percent with governance getting more out of analytics?

To find out, you can read next week’s installment of Profiles in Analytics, or download the entire research.

Profiles in Analytics - Does Web Governance Drive Digital Analytics?

I’ve just released this year’s Profile in Analytics research. Focusing on the relationship between Web governance and digital analytics, I drilled down on whether Web governance-oriented organizations really are “better” at analytics...Do they serve more stakeholders? Do they do higher value analytics work? Are they in alignment with senior leadership?

In doing the research, it was clear that analytics teams are now more common place and growing in size. Nearly 75 percent of those surveyed are part of an analytics team.  Teams of 5 and more now outnumber single person departments, 34 percent to nearly 27 percent. Most teams have between two to four people. Add to this that most digital analytics teams are working with between 1 to 3 third party agencies, and we can see that digital analytics is requiring management of many elements, as well as people who can carry out the tasks.

This raised a few questions:

• Is digital analytics growing in importance because it is recognized as having real benefits? Or is it simply viewed as a central location in the organization to manage ever increasing sources of digital data.
• Are those who work in digital analytics equipped with the management experience and leadership to promote the value of analytics at senior levels?
• Are analytics insights used within organizations?
• Are the analytics insights of high enough quality to be used?

One of my secondary agendas in looking at the influence of Web governance on analytics comes from my general skepticism whenever I hear an organization bill themselves as “data driven.” I’m always wondering: What does that really mean?   

I know it does not mean cranking out dozens of reports to dozens of people and not hearing a word back. I also know it does not mean preparing ad hoc analysis based on a senior manager’s request for a unique visitor report to present to the board of directors.

Does a Web governance structure ensure that analytics is part of the organization’s “mainstream” Web operations and therefore potentially used with more purpose, and therefore actually “data-driven”?

I invite you to draw your own conclusions after reading the findings:

• Research Analysis: Does Web Governance Drive Digital Analytics?
• Research Results:  Survey Response Summaries
• Profiles in Analytics: Collection of Individual Responses, Shared Wisdom and Experiences

All the papers are available at:

http://www.semphonic.com/content/white-papers/profiles-in-analytics.aspx

Upcoming Events: Planning Your Migration to SiteCatalyst 15 and Profiles in Analytics in Washington DC

I’m just getting back from a nice end of the summer vacation spent in the woods of northern Pennsylvania. No electronics, clear night skies and great post-Irene weather. A really nice way to re-charge for what is going to be a busy first quarter at Semphonic.

Of course there’s XChange coming up in San Diego from Sept 12-15. Like last year, this is going to be a sold out gathering. I’ll be teaching two ThinkTank sessions on 9/12: Managing a Digital Analytics Program and Beyond Visits and Page Views…How to Develop Actionable, Web Metrics, Reports and Analysis. Look forward to seeing you there.

Right before XChange, we’re going to be offering a free webinar on migrating to SiteCatalyst 15. Join me, Paul Legutko and Chris Meares this coming Thursday, September 8^th at 1pm EST/10amPST as we take a look at what’s new, what’s missing, and what to prepare for during your transition. As Adobe starts to pick up the pace on migrations, make sure that you know what to expect.

Once you register, you’ll receive your free copy of our new SiteCatalyst 15 Transition Guide. Don’t forget to submit any questions that you have when you register. This way we’ll make sure that your questions get top priority.  To register now, just go to: https://www1.gotomeeting.com/register/938012200

And finally, on October 4^th, we’re kicking off our Profile in Web Analytics series in Washington DC with Nina Mishkin of American Express. Nina will be providing some great case histories, tips and techniques for how she measures success for one of AMEX’s most heavily trafficked global web sites.

This will be the first in an ongoing quarterly series that we’re putting on with the Web Managers Roundtable.  Comcast and The Economist will be first up in 2012. If you’re in the Washington DC area and you want to get inspiring and practical advice from folks who are doing great analytics work, you’ll want to check these out. More details to come.  

To register for the October event, go to: http://bit.ly/n9tsM4.  Seating is limited so sign up soon to reserve your spot.

I hope you’ll be able to join us for these events as you get back into the post-summer swing of things.

Improving Federal Websites and Measuring Effectiveness of Your Customer Service Plan

How far along is your agency in implementing the Customer Service Plan mandated in President Obama’s Executive Order 13571 – Streamlining Service Delivery and Improving Customer Service?

How are you addressing the activities outlined in the Appendix of Memo M-11-24 – Implementing Executive Order 13571? The Appendix provides the roadmap for how to create the Customer Service Plan with specific focus on:

• The need for establishing ways to receive customer feedback
• Developing a process for evaluating the entire customer experience
• Analyzing customer preferences for interactions from less preferred and more costly channels (print) to preferred, less costly channels (online, mobile)
• Reporting progress in meeting customer service standards

If you have an existing measurement strategy in place, you should be able to create the metrics to evaluate and analyze the services being addressed through your existing set of tools and methods.

However, what if you don’t have an existing measurement strategy in place, or tools to support a measurement strategy?

You can potentially develop something quickly to meet the Plan delivery deadline by the end of October. You can then develop a cohesive measurement strategy that takes a longer term perspective. This is something to think about if you want to use data to drive ongoing decisions about customer service, eliminate outdated and duplicative web sites and derive operational efficiencies on web site development…all objectives that the Executive Order is encouraging agencies to act on.

Short Term Measurement Plan

So what does a short term measurement plan look like? Let’s look at a minimal and inexpensive tool set that will measure three key aspects of online customer service that is addressed by the Plan.

 1. Web Analytics – Tool: Google Analytics – free; easy to implement, easy to derive value

To use Google Analytics, you need to be in compliance with OMB Memo M-10-22 especially on being clear about opt-out options on web measurement.  See my blog post from last year for what you need to do to be in compliance:

Uses: For measuring the number of visitors using the site, the number of starts and completion (such as for a form, or multi-step process), geographic location of visitors, where visitors originate.  Segment these basic metrics by first time and repeat visitors, domain name, content interest (professional, consumer, and researcher).

Sample Metrics

Task Completion Effectiveness: Indicates visitor interest in and successful completion of tasks

Definition: Measures completion of site tasks associated with NIH or IC mission.

Calculation: Expressed as a percentage based on the total number of task completion page views divided by total number of site page views.  A higher percentage indicates more successful task completion.

Return on Investment – Provides quantifiable cost/benefit analysis of budget and time expended on Web site operations.

Definition: Measures return on investment in web site operations, content, applications, and labor in context with budget spent on these initiatives.

Calculation: Task Completion ROI: Number of Task Completions multiplied by $ value given to each Task Completion type compared to total expenditure on creation of Task application or in Avoided Cost alternative to manual labor.

2. Post-Transaction Customer Survey  - Tool: SurveyMonkey, Zoomerang – inexpensive; easy to implement, easy to derive value

Uses: For understanding whether the service altered behavior after use, obtaining feedback on how to improve the service, as well as why service was improvement over offline alternatives.

3. Customer Satisfaction Survey – Tool: QuestionPro – inexpensive; easy to implement, easy to derive value

Uses: Provide insight into whether visitors found what they were seeking and whether they were satisfied with the quality of the content they viewed or the transaction that was completed. 

To use surveys, and a host of other voluntary collection mechanisms, you’ll need to participate in the new Fast Track Process for Collecting Service Delivery Feedback under the Paperwork Reduction Act. Your agency is likely participating in this initiative already. You can refer to OMB Memo M-11-26 for the details of the process and http://www.howto.gov/customer-service for supporting information.

Whether you have an existing web measurement strategy or need to come up with a short term plan, the key to getting value from the data that has been collected is to analyze each set in context with its purpose in evaluating the service. The data should be complementary.

In my next post, I’ll discuss how to “triangulate” data analysis from the different measurement methods at your disposal.

Time to Contribute Your Profiles in Analytics

I've just posted up the new Profiles in Analytics survey. Like last year, I'm interested in collecting and sharing collective wisdom about how people are managing their digital analytics programs...what's working, what's challenging, how to meet challenges.

I published everyone's  responses from last year and plan to do the same this year as well. There were tons of great ideas and insights.

I'd appreciate it if you could fill out the survey and pass the link to others who you think would be interested.

Link to survey: http://svy.mk/pz0IYp

Link to last year's Profile in Analytics results: http://bit.ly/acMePk

On another note, I'm looking forward to seeing you at X Change in September. I'll be teaching pre-conference ThinkTank on Managing a Digital Analytics Program, a class I introduced last year and also teaching a class I introduced at the recent JBoye conference: Beyond Visits and Page Views...How to Develop Actionable Web Metrics, Reports and Analysis. Hope you'll be joining me.