When auditing I would suggest preparing your approach, data capture templates and methods well in advance to assist the speed and coverage of the audit. A simple test of your audit templates can be done – Ask yourself what are trying to achieve with the audit, i.e. what questions are trying to answer – Fill in your template with dummy data – See if you can answer your questions set in step 1.

I like to run the heuristic audit in a way follows a natural browsing pace as far as possible, stopping to notate every couple of minutes or at the end of natural paths (feel the user experience).

The quantitiative content audit is very useful for working out content migration plans or content focusing as part of on going or new developments.

Having started on a long term auditing plan I also ran a couple of additional exercises to get a complete view of the site, when applicable auditing the content administration tools side – often an area of low quality user experience, caused by a lack of focus on the system that will provide the actual engine to the site.

I think the heuristic term of Accessibility that you use may be better termed as Findability, courtesy of Peter Morville, as Accessibility infers that you may have been evaluating the sites Accessibility compliance, which from your discussion definately appears to be more in keeping with Findability.

Thanks for the article.

Richard Marsh
http://www.creative-resonance.com

The company I work for has dealt with clients who are very paranoid that we are ripping them off, or trying to “upsell” things that people don’t need (would you like fries with that?). It is quite possible that we could say, for Appropriate Structure, “This strongly deviates from the heuristic, so you need to totally restructure the site”. They are likely to think “why”, and we would have to explain to them the reason why their site does not have the right structure for their content and intended audience. Quite a few of our clients are non-technical people, and even after explaining all the factors involved, as one customer recently put it after my boss diligently explained the items that would be invoiced, “This is bull****!! I don’t understand a word f-ing word of this! How do I know you are not just making it up!”

I find people understand things like “Your site is getting 20% more hits this month, you are ranked number 3 on google”, but try to explain anything more subtle and their faces turn blank, and they mutter something about just wanting a website, “I’m paying you to work that out”. Ultimately if we DO work it out for them and they don’t like it, we have to do it all again. So any tool that can reduce this likelihood would be a godsend.

I think the ideas presented here are great, anyone with an ounce of common sense can see that the points you mentioned are essential factors in the process of analysis and development, and there will always be people skeptical of anything that smells of subjectivity. I personally believe there is a lot of validity in a qualatitive study which should be used to complement quantative results. Communicating such results to the client is another issue entirely, and one which I am still trying to work out.

Thankyou Fred.

Sorry for my rambling post… First day here.

Thanks for your kind words.

I’m glad you can see the value in how the heuristics will help you when discussing your work with clients. Anything that we can do to increase client’s understanding of our work is always important.

I can say that it was precisely in trying to define what the difficulties with a client’s content were that led me to the development of the heuristics.

I realized that in the rush of preparation for the IA Summit, I never answered your query. I don’t that that increasing the number of people working on content analysis will increase the perception of its objectivity. Rather, what is important is providing concrete examples to the client. Just saying, “Your site doesn’t meet the heuristic of collocation.” isn’t enough. Rather we have to point out the five different places where information on the same subject is spread and then point out that because there are not links among the sections that users will most likely not be able to find all of the information that is relevant to them.

Certainly one must be careful of bias or preconceived notions in any work that we do, so having someone else to validate or verify our results can also be helpful.

How would a quantitative evaluation differ? Would I be correct in assuming its more of a hard statistical expression of the content?

Thanks

Tom

Thanks for your kind remarks about the article. Glad it is of help to you.

A quantitative analysis/evaluation really focuses on specific numbers: How many different page templates are currently used? How many document types do we have and what are they? What is our most-used content? How current is content? How many content authors do we have, etc? Essentially items that can be directly measured or counted.

Obviously there is a place for quantitative analysis also, in addition to the types of qualitative analyses I discuss here.

Fred