We just completed a site optimization project based on the Functionalism methodology that Gary has developed and refined over the years.
It was an interesting engagement for me because it was the first time I'd seen Functionalism applied. It was also interesting because we worked closely with the client's interactive agency. Why was that so interesting? Primarily because they were most eager to see if there were any big holes in the site that they had designed and built. We didn't find any...mostly navigational tweaks here and there, but nothing major.
If you're a web site manager or product manager that uses the web as a major marketing and education channel, do you use analytics to audit your interactive or web design agencies?
I've spoken to some web site managers whose agencies don't want to do web analytics. They're concerned that it will show that the site really isn't "working."
If you'd like to measure agency performance quantitatively, consider building metrics into your contracts as a way to measure performance. For example, you could base this on the number of or percentage increase in conversion events, such as purchases, dollar volume, registrations, or downloads. If you want to use qualitative analysis, you could use online surveys from a company like ForeSee, and base your metrics on the American Consumer Satisfaction Index (ACSI).
In any event, remember that you can use your metrics not only to gauge the health of your web channel, but the effectiveness of those you've hired to build it.