Customer Experience Meets Online Marketing at Brand Central Station

by:   |  Posted on

“Marketing” covers a broad range of activities that together represent how easy you can make it for people to do business with you. Customers need to discover your offering, learn about it, compare it, and form a desire for it that is strong enough to compel them to shell out hard-earned cash to acquire it.

A brand is not a name. A brand is not a positioning statement. It is not a marketing message, a jingle, or a logo. A brand is the culmination of all of the interactions that all the people in a marketplace have with the firm.“Customer Service” activities revolve around how easy you make it for people to buy and use your offerings. Once they’ve shelled out the cash for the product, can they figure out how to use it? Can you and do you help them?

“Customer Experience” is all about how your prospective and current customers perceive your company, based on the effort they had to expend accomplishing the above tasks. If the word “brand” pops into your head, you may go to the head of the class.

The sum total of an individual’s experiences with your company will color his perception of it and build a brand image in his mind. A brand is not a name. A brand is not a positioning statement. It is not a marketing message, a jingle, or a logo.

A brand is something that lives apart from what the company plans, because it is the culmination of all of the interactions that all the people in a marketplace have with the firm:

  • A person sees an ad and forms an impression.
  • She looks up information on the web, and her impression changes.
  • She calls the firm and talks to the receptionist, and her impression changes.
  • She is put on hold and hears the music and “Your call is important to us.”
  • She talks to a sales rep.
  • She waits for the materials to arrive.
  • She reads the materials.
  • She talks to her colleagues about the product.
  • She reads about the firm in the financial pages.
  • She reads product reviews.
  • She makes the purchase.
  • She sees and feels the product packing.
  • She tries to use the product.
  • She calls customer service.
  • She talks to her friends about her experience.

There are only a few touch points where a company can exercise any serious control over that brand-building series of customer interactions: the advertisement, the marketing materials, the packaging, the product itself, and the web.

When a prospective or current customer calls your firm, the receptionist may not be having a great day, and that fact may make itself heard across the phone line. Your sales rep may be more worried about a commission and a house payment than listening closely to a potential client. Once a product is launched and on the shelves, you can only hope for great product reviews.

To influence our customers’ opinions and impressions about our goods and services, we want to use the most consistent, trustworthy, high-impact means we can to build a brand. Since we can’t control our employees’ moods, a company’s website is one of the most powerful tools available.

Your website represents your company in a very visceral way. Like advertising, you can use colors, pictures, copy, voice, music, and more to communicate your desired brand. Many companies stop at this cosmetic level and overload their sites with animations, a practice that has given rise to a new addition to our common lexicon: Skip Intro.

Your website also paints a picture of whether your firm is open and generous with information, or reticent and secretive. It shows how willing you are to inform, how hard you are willing to work to educate, and how freehanded you are with information about your abilities and goals. But a much more subtle message than all of these is how well your website actually works.

Think about the last time you praised a company to a friend or associate. You
probably used words like, “professional,” “easy to work with,” “capable,” and “on the ball.” Wouldn’t we like all the firms we do business with to have those attributes? Are people who visit your website left with the impression that your firm is on the ball?

Most companies that bother to interview or survey site visitors often ask questions that don’t quite dig beneath the surface of customer experience. They ask questions such as:

  • Did you like the visual appearance and layout?
  • Did you like the content?
  • Did you like the style, tone and character of the site?

Questions about customers’ responses to site characteristics are valuable, to be sure, but they are missing the customer experience side of the equation: what is it like to use the site? Customer experience questions are more along these lines:

  • Why did you visit our site?
  • Did you find what you were after?
  • Did you run into any trouble?
  • Was it fast enough?
  • What did you like best?
  • What did you like least?
  • Are there any features you like on competitor sites?
  • What would you like to change on ours?
  • What one button would you add?

These are akin to asking a customer if they felt the company was easy to work with, professional, and on the ball.

Customer experience, the building of a brand image in the mind of the customer, is the culmination of all touch points, be they outbound corporate communications or a conversation over the backyard fence with a neighbor who just found a series of 404s on your site.

Because your website is so interactive, and because it is such a strong indicator of how much you devote to potential customers, it becomes one of the most important tools you can use to impress and delight customers and convince them that your firm is the most qualified and the most deserving of their business.

Your website is also the single most measurable means of communicating with the world. In my consulting work with large corporations, I have only discovered a few that understand the brand-building power of the web well enough to bother measuring their success there in a serious way. These few understand that there are quantitative ways to calculate the answers to questions like:

  • How good are we at attracting attention to our site?
  • How easy is it for people to find what they’re looking for?
  • How quickly are we improving the conversion process?
  • What is the value of adding additional content?
  • Want impact does online customer service have on lowering costs and increasing customer satisfaction?

Customer experience is no longer an entirely soft science.

The capacity to measure your customers’ experience on your site is not simple, nor is it inexpensive. But the power of assigning numerical values to customer experiences is being used by a handful of forward-thinking companies, and it will allow them to make the most of the web.

At Hewlett-Packard, for example, the team responsible for web metrics tracks what they call the “Seven Recipes”:

Optimize lead generation
Which attention-getting methods are bringing in the most—and the best—traffic to your website? You can get millions of people to show up by promising free money, but if they’re not qualified buyers, you’re wasting time and resources and giving your brand a black eye. HP tracks leads from the banner ad to the purchase page to make sure their ads are making sales, not just building traffic.

Tune your pages
Are your pages helping people find what they’re after or sending them away? HP measures the number of people who click through each page. They also measure how quickly pages load, and whether customers have good things to say when surveyed about specific pages, rethinking pages that fall below the average threshold.

Optimize navigation
How hard is it for a visitor to get from a landing page to a conversion (shopping cart) page? How many clicks? How much reading? Perhaps even more importantly, are people taking the path you laid out for them? Or, in their confusion, are they wandering through other pages to get there? Is there a lot of “pogo-sticking” going on, where people click back and forth endlessly from a main navigational page to various content pages?

Enhance search
Do people find what they’re looking for when they use your internal search engine? Does it point them to a press release about your product instead of the product home page? HP hardwires some product page links to specific search terms to improve findability.

Optimize merchandising
HP constantly watches what gets put into their shopping carts. Which items sell together? What’s the best way to notify a customer who is interested in Product A that they might also be interested in Product B?

Tune entry points
Landing pages are the bridge between your advertising and your persuasion pages. HP looks at click density maps of each landing page to see if the dispersion ratio is in line with the rest of the site. If people arrive at a landing page and click through (disperse) to different sorts of content than what you intended, your promotion is missing the mark.

Analyze sales funnel
Given a three-, four-, or ten-step process to find, choose, configure, and purchase something on your site, where are the trap doors that people fall through on their way to check out? Where do most people abandon the process?

Throughout their analyses, HP has a very straightforward methodology. They review the reported numbers, surmise the reasons for those numbers, decide what to change in order to affect those numbers, and then closely measure the results of those changes.

Can you measure whether your customers are impressed with your brand because they are having a good experience on your website? There’s no spreadsheet you can download that will answer all your questions. But we’re definitely making progress.

Jim Sterne is an internationally-known speaker and a consultant to Fortune 500 companies, with 20 years experience in sales and marketing, and the author of “Web Metrics: Proven Methods for Measuring Web Site Success.”

What You Should Know About Prototypes for User Testing

by:   |  Posted on

There are several important factors to consider when you are planning to do prototyping for user testing. You will want to make careful choices about fidelity, level of interactivity and the medium of your prototype.

Degree of fidelity “An information architecture wireframe is NOT graphic design. I swear, it’s really not!!!”Fidelity is the degree of closeness to the “depth, breadth and finish of the intended product” (Hakim & Spitzer). Opinions vary a great deal on how much a prototype should resemble the final version of your design. Usability practitioners like Barbara Datz-Kauffold and Shawn Lawton Henry are champions for low fidelity —the sketchier the better! Meanwhile, Jack Hakim and Tom Spitzer advocate a medium- to high-fidelity approach that gives users a closer approximation of a finished version. You’ll want to make a decision about the right approach for you based on the needs of your project.

Low fidelity
You can use hand-drawn sketches to create a paper prototype. If you go this route, you may also want to help your users get into the spirit of things during the test by creating a complete low-fidelity, paper environment. This could include a cardboard box made to look like a computer and an object to hold to point and click with. These techniques help users to suspend their disbelief and get their imaginations involved so that they can better visualize the interface. The advantage of using rough sketches is that users will have an easier time suggesting changes. They may even grab a pen and start making their own changes (Datz-Kauffold and Henry).

In theory, low-fidelity sketches are also a time-saver, but this really depends on your point of view. Personally, I like to draw diagrams and wireframes in Visio where I can revise and move things around without erasing and redrawing. If you prefer to work this way too, and if time allows, you can always have those Visio drawings hand-traced or use them as a roadmap for making sketches to test with. You might even find a graphics tool with a filter that will convert a Visio-generated graphic into a hand-drawn sketch with wavy lines.

High fidelity
This approach takes you as close as possible to a true representation of the user interface —screen-quality graphics. All of the blanks on the page are filled in, and it looks good. However, you might not have all of the technical or backend problems worked out yet, or you might have only a small part of the entire site rendered. That’s why it’s still considered a prototype. For example, it might consist of a small series of Photoshop images or HTML pages with just enough functional links to convey the feel of the site’s flow. You may need to enlist the help of a graphic designer or web developer to build these in a reasonable amount of time. Advocates for high-fidelity prototypes argue that they are easier for users to understand just by looking at them. There is no disbelief to overcome, and it is easier to determine when they really do not understand the design. If you choose a high-fidelity prototype, make sure the you have enough of the design fleshed out so that users can complete several tasks. Decide on these tasks early, so you know which areas of the design need to be represented for your tests. Otherwise, you will be in for a great deal of preparation work.

Medium fidelity
In the grand tradition of Goldilocks, I find myself drawn to the middle approach. A medium-fidelity approach tends to include some visual design and a level of detail somewhere between high and low fidelity. Does this sound familiar? As an information architect, I’m accustomed to creating wireframes I can hand off to decision-makers, graphic designers, web developers and programmers. An information architecture wireframe is NOT graphic design. I swear, it’s really not!!! But… I’ll admit that it has enough visual design to convey a rough version of the user interface. Because I create these with drawing tool software, they tend to have more polish than hand-drawn diagrams. Hakim and Spencer are champions for medium-fidelity prototypes because they fit more seamlessly into the design process while providing more realism for users. I found this to be true during a project to design a search interface for Egreetings with my colleagues at Argus. I created rough draft wireframes for the prototype, and after testing I revised them for use in my deliverables.

Interactivity describes how your prototype behaves. Does your prototype react to user inputs with feedback? Can they “click” on something to go to another page or fill in a form? Will buttons appear to depress and drop-down menus work?

Static prototypes
Prototypes used for testing are static if they are pages or page elements shown to users, which don’t provide any feedback. It can sometimes work well to show a page to a user and ask them to explain it to you or to guess where they can go from here. In this kind of test, the user interprets the prototype rather than interacts with it. This is a good way to validate your design by checking to make sure users understand it. It’s also easy to score this sort of test when you have a standard list of questions to ask about each page.

Automated prototypes allow users to make choices that cause changes. The testing prototype provides the user with feedback. Elements are “clickable” and forms can be filled out. The interface reacts to the user while the tester observes. One way to do this is to create the prototype in HTML or some application that allows interactive elements such as Flash, Visual Basic or even PowerPoint.

Another way to achieve a kind of pseudo-automated interactivity when you have chosen a paper prototype is to pretend (Datz-Kauffold and Henry). Have you ever seen children at play pretend that they are driving a car by setting up chairs for the front and back seats, drawing a dashboard on a cardboard box, and using a Frisbee for the steering wheel? If you have set up the right environment for your users, you can ask them to pretend scraps of paper on a table are their computer screen. When they “click” on a drop-down menu by touching the element with a pointer, a tester assigned to the role of the computer provides feedback by swapping the closed menu for an open one that shows choices. The “computer” may need to write on some elements before showing them to the user, i.e., “Your search retrieved 523,621 hits.” It takes a few minutes to get test participants used to the idea, but if you encourage them to have fun with it you will learn a great deal. You can also easily try out different possible reactions to user input.

This method worked well during the Egreetings project. We especially emphasized the technique of asking the users to click and then provide feedback. We found it useful to laminate the screen components so we didn’t need to produce a clean copy of the test for every subject. The users could write on the laminated pieces with thin whiteboard markers when making selections and entering search criteria. Of course, this meant that we needed to take careful notes because of the need to erase between each test subject.

Here are some other tips to try for low-fidelity testing with simulated interactivity:

  • Bring extra paper so you or the respondent can sketch out an idea if the opportunity arises.
  • As with any user test, it really helps to ask the respondent to think aloud.
  • If you have the luxury, bring a team of three to the test: someone to take notes, someone to play the “computer” and another to facilitate.
  • Use a piece of posterboard as your “screen.”
  • Cut your design into separate pieces or zones as appropriate and ask the user to rearrange them in the order they prefer.
  • Attach the folder tabs that come with hanging files to components so they are easier to grab.
  • Invite users to throw away or cross out components that they don’t think are important.
  • Number the pieces so that you can easily refer to them in your notes and keep them organized.
  • If you do decide to bring separate copies of the test materials for each session, tape down the components to a larger piece of paper as arranged by each user so you have these artifacts to analyze later.

Prepare a kit for yourself containing:

  • Scissors and tape,
  • Different sizes and varieties of sticky notes (which make great drop-down menus),
  • Markers and pens in various colors and sizes,
  • Paper clips and binder clips for keeping slips of paper organized, and
  • Objects that the user can pretend are the mouse pointer, such as a feather or a small toy.

There are many possible combinations to choose from for building your prototype. One of the first choices to make is whether you want to have your prototype viewed on an actual computer screen or if you’ll be working on a tabletop with a paper prototype. Believe it or not, fidelity and interactivity are independent of the medium you choose. It’s probably most natural to think of the extreme cases. An automated HTML prototype is often high-fidelity and, of course, the medium is a computer screen. Likewise, a natural medium for a low-fidelity automated interactive prototype is hand-drawn sketches on paper. However, you can also have the following:

  • Low to medium-fidelity wireframes built in PowerPoint that show only lines and boxes with text;
  • animation features provide automated interactivity,
  • Static Photoshop prototype pages shown to users on a computer screen, or
  • Same as above, but printed out in color on paper.

Mixing the variables
You can mix these three variables (fidelity, interactivity and medium) in many different combinations. The exact combination you choose should match the goals you determine for your testing. Possible goals for an IA prototype include:

  • Testing the effectiveness of labels and icons.
  • Finding out the right balance of depth and breadth of a topical hierarchy.
  • Determining the right options to offer for narrowing a search.
  • Choosing the most important metadata elements to show on a search results screen.
  • Settling the question of whether your target audience accomplishes tasks better with a task-oriented organization scheme or with a topical organization scheme.

If you live and breathe NetObjects Fusion and don’t have much time, your preference might be to create a medium-fidelity prototype. That way you could test that sitemap you are working on using some rough placeholder graphics or text instead of the finished graphic design. How you mix the variables depends on the time and budget you have available, as well as your work style. Try experimenting with different approaches to learn how prototyping will work best with your design process.

For more information

  • Evaluating Information Architecture,” Steve Toub (2000).
  • UPA 2000 Proceedings:
    #28 – “Waving Magic Wands: Interaction Techniques to Improve Usability Testing Low-Fidelity Protoypes,” Barb Datz-Kauffold & Shawn Lawton Henry.
    #32 – “Prototyping for Usability,” Jack Hakim & Tom Spencer.
  • “Prototyping for Tiny Fingers,” Marc Rettig, Communications of the ACM, Vol.37, No.4 (April 1994). (ACM Membership required)
  • Using Paper Prototypes to Manage Risk,” User Interface Engineering.
Chris Farnum is an information architect with over four years’ experience, and is currently with Compuware Corporation. Three of those years were spent at Argus Associates working with Lou Rosenfeld and Peter Morville, the authors of Information Architecture for the World Wide Web.

Slate: Calculated Refinement or simple inertia

by:   |  Posted on

Before we get started, I just wanted to note that my comments are intended to supplement the diagram, rather than vice versa. So be sure to download the PDF version of the diagram to get a full understanding. That said…

No matter how you look at it, publishing content on the web daily is a lot of work. From an information architecture perspective, a daily web publication presents challenges and possibilities no newspaper editor ever had to face. As one of the longest-running daily publications on the web, Slate has dealt
with these issues for years. But it is unclear whether the site’s
current architecture is the result of calculated refinement or
simple inertia.

The architectural decisions here demonstrate one key assumption about the site’s content: the ‘shelf life’ of any given article is about seven days. Navigating to a piece during those first seven days is fairly easy; after that, it becomes very hard.

At a glance, the high-level architecture seems fairly straightforward. But a closer look reveals that the five primary ‘sections’ exist only in the tables of contents. These categories appear nowhere else on the site—not even on the articles themselves. Furthermore, the classification of articles into these
categories only persists for seven days from the date of publication. After that, the section to which a piece belonged is forgotten.

Note the absence of an ‘archive’ area. The only access to articles more than seven days old is through the advanced search page. In place of a browsable archive, Slate offers canned searches by “department” and by author. The author list page works well enough, though such a feature would only be useful in the event that a user already knew the name of the author of a desired piece; but if that were so, the search interface would be sufficient.

The department list page has a greater burden to bear. As the only persistent classification scheme employed on the site, the department list is the only element that can provide the reader with a sense of the range of content and subject matter covered on the site. But the page currently falls far short of this goal. What the user faces here is nothing more than a very long list
that makes no distinction between limited-run features like “Campaign ’98”; occasional, semi-regular features like Michael Kinsley’s “Readme”; and ongoing staples like “Today’s Papers.”

This problem is only exacerbated by the fact that, by and large, the department titles are too clever by half. Even the savviest user could be forgiven for having trouble remembering whether Slate’s roundup of opinions from movie critics was filed under “Critical Mass” or “Summary Judgment.” The cute titles would be fine if the site provided some sort of context for what was to be found inside; as it is, providing a plain list of titles like “Flame Posies”, “Varnish Remover”, and “In the Soup” does little to help readers find specific items or even get a general sense of what the site has to offer.

Letter-sized diagram ( PDF, 41K)

Note: The date on the diagram indicates when the snapshot of the system was taken. Slate may be substantially different now.

Finally, I wanted to find out what sites you’d like to see me diagram in the future. You can post your suggestions here.

Jesse James Garrett is one of the founders of Adaptive Path, a user experience consultancy based in San Francisco. His book “The Elements of User Experience” is forthcoming from New Riders.

Got Usability? Talking with Jakob Nielsen

by:   |  Posted on
Photo of Jakob Nielsen

Jakob Nielsen is the usability guru who hardly needs an introduction. But for the sake of completeness we’ll mention he’s the co-founder of the California-based consultancy, Nielsen Norman Group, and has been crusading against bad web design for years through his biweekly column, The Alertbox, and his numerous books. He’s brought usability to the attention of the general public, but within the user experience community he’s been criticized by those who say he emphasizes a puritanical view of utilitarianism that excludes other dimensions of user experience. Oh, and did we mention he’s the man who launched a thousand parody sites?

So is Nielsen the defender of ease-of-use or the enemy of creativity? We talked to the controversial Dane, and you might be surprised…

B&A: What are some of the toughest design challenges on the web today?

Nielsen: I think to get a really big jump in usability, because I think we can make a website that can show a few things quite well, if you have a few products. We can also do a huge database and you can search it, and it works reasonably well.

But I don’t think we really have a handle on getting the average person through the vast number of things that a website can offer. If you narrow it down and show a few things, yes, if you assume that they are capable doing a lot of data manipulation. But I think there’s a large number of cases that do not fall into one of those two categories. You can go to CNN and see the five big headlines of the day, and that works fairly well. You can go to Amazon and you can buy my book, for example, if you know the name of the book. But in the intermediate case of having a website with 10,000 articles and finding the one that’s right for you, which is quite often the case on a tech support website … basically doesn’t work at all.

B&A: What types of research interest you the most?

Nielsen: How to get usability out to the masses. When I say masses, I mean web designers, not users. Right now we have about 30 million websites, and we will have up to 100 million in three to five years. That’s a large number of design projects. How many usability people are there in the world who are in any way qualified? At the most, maybe 10,000 or so.

Therefore, we know that we’re not going to have this number of web projects done according to the recommended old methodology. So, even what I’ve been pushing in the past—more efficient, quick usability methodologies—is not good enough when you have that number of design projects. We need to have several orders of magnitude improvement in the efficiency of usability to really impact that number of design projects. Can we do things like encapsulate usability knowledge in guidelines such that an average designer can actually apply them?

B&A: What do you feel is the relationship between a usability professional and a designer?

Nielsen: I think they could play two different roles: either that of an editor and a writer, or a professor and a student.

In the more integrated projects, which is the preferred way to do it, I think it’s more like the editor and the writer, where the designer will come up with things just as the writer would write the article, and the editor will make it better, will know what the readers need and how to present it in a good way and help the writer improve their article. I have never met a professional writer who didn’t like to have a good editor. There often seems to be a conflict between designers and usability people, but I think that once you conceptualize it as the usability person helping to improve the design, then I think it goes away.

But you’re going to have a lot of designers who don’t have a usability professional in their team. So the vast majority of them just have to learn what the principles are that work well with users from usability professionals, and then it becomes more of an educational mission. So the relationship is more like that of the professor and the student. The student is the one who has to go do it at the end of the day, but the professor is the one who has the knowledge, having had done all the research in the past and can tell the student what works well.

B&A: How do you react to designers who have strong feelings about usability in one way or another?

Nielsen: I think that designers that don’t want usability are misguided because it’s really just a way of helping them achieve a better design. Some of them just reject the goal of having a design that’s easy to use. If you have the goal of a design as actually trying to accomplish something, then you’re more in the art world, and if the project doesn’t have a goal, then maybe it’s appropriate—design for design’s sake. But if you do design to actually accomplish something, then I’d argue that it has to be easy to use, so I don’t think that it’s appropriate to reject the goal of usability if your project has to accomplish something. Design is creating something that has a purpose in life; art is creating for the sake of creating — that’s my distinction between those two terms.

Whether they want to get usability from someone who knows about it, or whether they want to find it out themselves … can be debatable. How did any of us become usability specialists in the first place? Only by doing a lot of the research and studies. Any designer could do that as well if they bothered. They don’t have to get it from us, but then I would argue that they would need to do it themselves.

B&A: Is there a particular reason you advocate for using guidelines? I’ve heard people say that it comes off as overly dogmatic to simply have a huge list of guidelines.

Nielsen: Experience says that usually these work — usually, but not always. Usability guidelines always need to be applied with a certain amount of understanding as to when they apply and when they don’t apply. If a set of guidelines is written well, then usually they will apply, and it will be the exception when they don’t apply. You have to acknowledge that on one hand it may be that only 90 percent of the guidelines apply … so you can’t violate all guidelines, you can only violate some if you have a good reason to do so.

Some people may not understand the difference between a guideline and a standard. A standard is something that is 100 percent firm, and a guideline is something that is usually right — that’s why it’s called a guideline.

B&A: What’s the difference between a standard, a guideline, and a heuristic?

Nielsen: You get even more vague when you get into the area of heuristics. Heuristics are things that are rules of thumb, so they are very vague and very broad. At the same time, they are very powerful, because they can explain a lot of different phenomena, but that explanation has to be done with a lot of insight, and that is what’s more difficult. One of the lessons from a lot of my research is that heuristic evaluations indicate how to adjust an interface relative to these general principles of good usability. It’s fairly difficult to do well. Anybody could do it to some extent, but they couldn’t necessarily do it very well, and you have to have a large amount of experience to do it well.

On the average design project today, they don’t have that amount of usability expertise on their team, and therefore we’ve got to give them something more complete that it’s easier for them to deal with. It’s a matter of the usability of the usability principles, really. If we make them more specific, they become more concrete, they’re easier to interpret, and … easier for the designers to judge when they do not apply.

B&A: What’s the difference between someone doing a heuristic evaluation solo versus doing it in a team?

Nielsen: The way I developed heuristic evaluations back in the 1980s was meant to be an interaction between solo and the team, because you first do it individually, and then you combine a few people who have done the heuristic evaluation. That’s done very rarely, because it’s rare that a project team will have that many people on board who really know about usability.

“(I)t’s not a matter of intuition. It’s a matter of being very good at pattern matching, being able to spot small things, and hold together the big picture of what that really means.”

A common mistake about heuristics is thinking that it’s just a list of complaints. It’s not a list of complaints, it’s a list of issues relating back to the underlying fundamental principles. When you say that this button is wrong or this flows wrong, you say it’s wrong because it violates this well-known usability principle. And then, of course, people can argue. They can say, “no, it does not violate this principle,” and then you would have a discussion about that, which is a great method of illuminating and getting insight into the design.

B&A: What are the most important skills for a usability specialist to have?

Nielsen: I would say experience. It’s an unfortunate thing to say, because you can’t acquire experience other than by doing it. This is a discipline where you will always start off being bad and you end up being good. You only get to be good by slogging through several initial projects where you didn’t do that well, and then you get better and better. I think that being a truly great usability specialist comes from having 10 years of experience and having seen a very large number of different designs, different technologies, different types of users — a very broad variety of experience.

The benefit of usability, though, is that it is such a powerful method, and the return on investment is so huge that even if you don’t do that great a job at it —maybe you don’t get a return of 100-to-1 and you only get a return of 20-to-1 — that’s still a huge return investment. Even the very first usability project someone does, and they mess up everything, it’s still going to be positive, and it’s going to be a great learning experience for them personally, and their team is going to get value out of the investment as well. Just keep doing it and doing it and doing it.

It’s very much of an analytical and interpretive discipline as well. Intuition is completely the wrong word to use — it’s not a matter of intuition. It’s a matter of being very good at pattern matching, being able to spot small things, and hold together the big picture of what that really means. That’s where experience helps you — it helps you to do pattern matching and match patterns you’ve seen before, and the more things you’ve seen before, the better you can do that.

There’s definitely a big evangelizing and propaganda component as well, so having good communication skills is very important too.

B&A: Are there any usability specialists you particularly admire or whom you took guidance from?

Nielsen: I did actually. I’ll say that two of them are actually colleagues at my company, Don Norman and Bruce Tognazzini. They are two incredibly great people. Another one I’d like to mention who’s now retired is John Gould. He worked at IBM in the 1980s. He developed a lot of the early approaches and for any question you could come up with he’d say, “OK, you can do a study of that.” He was just such an empirical guy that it was incredible.

Another person is Tom Landauer, who worked at Bell for many, many years. I was privileged to work with him for four years when I worked there as well. He was very much on the measurement side: “We can quantify this. We can estimate these things.”

I’d like to mention one more person … I never worked with, Ted Nelson, who was the guy who kind of invented hypertext. He got me into this feeling that we shouldn’t accept computers being difficult, that computers can be a personal empowerment tool. I read a lot of his writings when I was in grad school. His writing is really what got me going in this area in the first place back in the 1970s.

B&A: How many users do you yourself observe in the average month?

Nielsen: I probably sit with too few users, actually. Probably less than 10. It ought to be many more. In my own defense, I’ll say that I’ve done it for many years, and the learning is cumulative. I run a lot of projects where someone else will sit with the user, but I’ll still monitor very closely what goes on. I would still say that it’s very important to sit with the user as well. People should continue to do that forever — you never get enough of that. In particular, for someone who’s starting out in usability, I would say 20 or 30 a month would be a good goal to have, so that you can try to run a study every week.

B&A: Will there be new methodologies for user research in the future, or will we keep refining the ones we have right now?

Nielsen: I think mainly we will keep refining the ones we have. Of course, you never know if some completely new thing will come up, but I think it’s not likely. The classic methodology was developed in the 1970s and early 1980s. John Gould was one of the big people doing that and I learned a lot from him. That was pretty much established by then: how to do measurement studies and all that.

“Usability has very much seemed like a black art … Many things are testable, but at the same time we have to broaden the scope to make it even cheaper, even more accessible, get even more people doing it.”

Then, in the late 1980s, I reacted a bit against my own mentors and said, “These are all great methods, but they take too long, and a lot of projects won’t do them if they’re not at a big, rich company like IBM.” So, we developed discount usability methodologies, which was a faster way of doing these things.

Since 1990 there hasn’t been that much change. I think it’s pretty slow-moving because it doesn’t relate to technology, which changes all the time. It relates to humans and the process of accommodating human needs, which doesn’t change very much.

B&A: Do you ever feel like discount usability methods can be misused?

Nielsen: I think there could be cases where someone does a heuristic without truly understanding the principles. Or you might have someone who tests one user and says, “Let’s go with that.” But in general I think that the methods are so powerful that they actually hold up pretty well even if they’re abused.

I read recently somebody who had criticized the idea of doing studies with a small number of users with the argument that you cannot judge the severity of the usability problems because you don’t have enough instances of observation to know the frequency with which it occurs. This is a circular argument, a self-fulfilling prophecy because you are accepting in their argument that the only way you can judge the severity of a problem is by having a statistically accurate assessment of it’s frequency. I’m arguing that after having had observed it a few times, you can, with the insight that comes from experience, estimate the severity pretty well — good enough anyway. The real issue in severity ratings is that you’ve got to do a cost-benefit analysis.

B&A: What’s your take on information architecture?

Nielsen: The first question I have is what it really even is. I tend to operate under the definition that it’s the structuring of an information space. I view that as being different from information design, which has to deal with how you present the information once you’ve found it, or interaction design, which is a matter of flow through a transaction or task. I know that some people like to use the words information architecture to apply to everything, which is what I would tend to call user experience. That’s purely a matter of what terminology you feel like using. I tend to think that user experience is built of these components: how are things structured, how it is presented, how do you flow through it, and other things like how is it advertised.

B&A: What’s next for you and the Nielsen Norman Group?

Nielsen: Trying to drive usability more broadly toward that larger set of design firms, really trying to encapsulate it to make it more portable. Usability has very much seemed like a black art. I myself have often said, “Well, you can just test that.” Well, that is true. Many things are testable, but at the same time we have to broaden the scope to make it even cheaper, even more accessible, get even more people doing it.

There’s another trend as well which is tackling deeper issues that have been neglected in the past that need to be more in the forefront. Things like users with disabilities, international users, much more focus on task analysis and field studies — those are some of the other things we’re pushing now.

Recently I’ve been pushing the notion of doing discount field studies. Field studies don’t need to consist of five anthropologists taking a year to do a project. We’ve had a seminar at our conference on simplified field studies, which I personally think is a good seminar. But, empirical data shows that people don’t want to do this. You can go to the conference and see people crammed into sessions on everything else, but then you go into the field studies seminar and there’s only 30 people or so. We are pushing it, but we’re not getting enough acceptance of this idea of the simplified field study.

B&A: Who do you think does a good job dealing with content online?

Nielsen: Very few actually. I can’t come up with any great examples — it’s still so print-oriented. My own articles aren’t that great either, actually. I’m very verbose in my writing style. It needs to be very punchy and very short, and it’s very hard to write that way.

There’s more linking happening today with all of the weblogs, which is kind of nice, but I think the commentary is often not that great. The reason is that I think weblogs tend to emphasize this stream of consciousness posting style, which I don’t think is good—that’s not respectful of the readers’ time. What’s good about weblogs is that they’ve broadened the number of authors, but at the same time they’ve removed that feeling that the writing is really being edited.

B&A: If you weren’t doing usability, what do you think you’d be doing?

Nielsen: I would probably be a university professor of something or other. When I think back to when I was a kid, I had a lot of different interests and things I was good at, which I think was one of the reasons I ended up in usability. You have be good at communicating, you have to know about technology, you have to understand interaction and human behavior. There’s all these different angles that pull together very nicely in usability. It’s good for a person who’s broad in the types of things they’re good at.

I might have ended up as a historian, I might have been a mathematician, I don’t know. I think that being a professor is the most likely. The reason I got into usability is that it’s a discipline that gets interesting when you go into the actual practice of it. There’s actually not that much theory, and it’s not that exciting actually.

Chad Thornton works as a Usability Specialist in the User Experience Group at Intuit. He has done similar work at Achieva, the American Museum of Natural History, and Pomona College, where he received his degree in Biology.

Yahoo! Mail: Simplicity Holds Up Over Time

by:   |  Posted on

Email was one of the first applications to move to the web, and the first to find widespread popularity among users.

In many respects, email is the ideal web application: it’s an application that people often need access to when they’re away from their “home” environment, and the core user tasks (reading and writing) are easily accommodated with standard HTML interface elements.

As a result, it should come as little surprise that the basic flow of Yahoo! Mail has hardly changed at all since the portal first acquired the RocketMail service in 1997. But rather than offering an outdated solution to the web-based email problem, Yahoo! Mail demonstrates the lasting effectiveness of a simple approach.

The application is extremely conservative with page designs. Almost all user interaction takes place across only three pages: the “message list” folder view, the “message display” page, and the “compose” page.

Another demonstration of this conservative approach is in the site’s error handling. The entire application contains only one standalone error page (the “no account found” page in the login flow), and this seems more likely to be the result of a back-end limitation than a deliberate design choice.

A few awkward spots do appear in the flow. An empty search result set returns a search result page with a “no messages found” message, rather than bringing the user directly back to the query interface to retry the search.

Downloading attachments is a two-step process, which seems like one step too many. The dichotomy between viewing and editing contact information in the address book seems like an artificial distinction whose purpose is unclear. But these are really minor quibbles; overall, Yahoo! Mail is a model of streamlined interaction design.

Yahoo! Mail diagram
Poster-sized diagram ( PDF, 37K) | Letter-sized diagram ( PDF, 100K)

Note: The date on the diagram indicates when the snapshot of the system was taken. Yahoo! Mail may be substantially different now.

Jesse James Garrett has been working in the Internet industry since 1995. In the information architecture community, Jesse is recognized as a leading contributor to the development of this rapidly evolving discipline.

The Evolving Homepage: The Growth of Three Booksellers

by:   |  Posted on

Web design is expensive. Web designers earn upwards of $50,000 a year1, information architects earn even more.2 During the heyday of web design—the late 1990s—designing a large commercial website could cost as much as designing a medium-sized building. During this period, commercial websites were created and then often completely replaced with redesigned versions a short time later. Today the redesigning continues, albeit at a slower pace. What is the return on this design investment? A report on online ROI from Forrester finds that many commercial sites fail to even try to measure the effectiveness of design changes.3

What lessons have we learned about how design improves the interface between customers and companies?

The web has been with us for about a decade now. We’ve seen some obvious trends, such as greater use of multimedia, search engines, and increasingly sophisticated markup techniques. But these trends were facilitated by changes in technology. What lessons have we learned about how design improves the interface between customers and companies? Perhaps we can start by asking how websites have actually changed over time, and from that we can learn how websites should change in the future.

To start working toward an answer, I compared three eCommerce sites:,, and Much of the media’s coverage of these websites, especially coverage of, discusses the business models, corporate cultures, and finances of the companies. Since the medium of interaction with these companies is the website, it’s ironic that the media rarely critiques the site design and its effect on business performance.

Because it is the homepage that carries the most responsibility for guiding customers, I examined the homepages of all three sites from a number of years, using screenshots from the Web Archive4. Presumably these large retailers had a great deal to gain, and lose, with these substantial online ventures. By comparing design decisions over time among the three sites, I hoped to discover lessons from their extensive and expensive design experience.

The companies
Competition is fierce in the online bookselling market, currently erupting in offers of “free shipping.” All three companies have annual revenues in the billions of dollars.

Barnes and Noble, which runs a large chain of stores in the United States, claims the largest audience reach of any bricks-and-mortar company with Internet presence.5 Yet, both they and Borders were put on the defensive when Amazon’s growth rocketed. During December, 2001 attracted over 10 million unique visitors,6 compared to Amazon’s 40 million visitors.7

Borders is the second largest bricks-and-mortar book chain in the U.S. 8 In April 2001, after operating their own online bookstore for several years, Borders announced an agreement to use Amazon’s eCommerce platform to power a co-branded website.

Amazon claims to be the leading online shopping site, having expanded their selection to music, video, auctions, electronics, housewares, computers and more.9 By February of 2002, Amazon, which had pursued a get-big-quick strategy typical of internet companies in the late 1990s, announced its first profitable quarter.10

I first studied these sites quantitatively looking for clear trends over time. I then critiqued them in a more qualitative way based on my own experience as both an in-house website designer and as an information architecture consultant.

There are many criteria that could be examined in such a study. I limited myself to those that would, I hoped, reveal as much as possible about the business intent of the design. I looked at criteria such as the type and size of layout, the type and amount of navigation, the amount of images and text, and functionality specific to the industry. Detailed results can be seen in the attached spreadsheet (PDF, 75k).


Chart showing growth in length of homepages over time
Click to enlarge.
Note: Missing data due to imperfect records at the Web Archive.

All three sites use very long screens to display content on their homepages.
Using a browser window with a constant width, we can compare the vertical size of each site (all screen references assume an 800 by 600 pixel monitor). The homepage grew from a vertical size of about 917 pixels in 1996 to over 3,000 pixels in 1999. Barnes and Noble’s homepage has hovered around 1,500 pixels for the last several years. Amazon’s homepage, which began at only 626 vertical pixels in 1995, stands at roughly 2,156 pixels today. In a web browser, that equals five scrolling screens of information. homepage above the fold, 1999 above the fold (1999) Click to enlarge.
Barnes and Noble homepage above the fold, 1999
Barnes and Noble above the fold (1999) Click to enlarge.
Amazon homepage above the fold, 1999
Amazon above the fold (1999) Click to enlarge.

Note: Incomplete web pages are due to imperfect records at the Web Archive.

All three sites evolved to use three-column layouts.
In 1995 and 1996 respectively, Amazon and used single-column layouts. By 1999, both of these sites as well as Barnes and Noble used three column layouts.

Amazon has consistently placed more links above the fold.
In 1999, the Borders site displayed only about eight links “above the fold” (the top portion of the screen that is viewable without scrolling). Both Barnes and Noble and Amazon had significantly more links above the fold in 1999, 30 and 48 respectively. Amazon averaged 43 links above the fold between 1999 and 2002 versus only 27 links for Barnes and Noble during the same period.

Through the years, the density of links on was half of that on Barnes and Noble or Amazon.
The density of links has varied over time, but as of 2002 both Barnes and Noble and Amazon stood at about one link for every 15 vertical pixels of screen real estate. Historically, the highest link density at was one link for every 28 vertical pixels.

Amazon communicates using images and links rather than text descriptions.
From 1999 through 2001, Amazon used more images and fewer text descriptions than Barnes and Noble. In 2002, both sites used about 560 words per page, yet the density of words was 33 percent lower on Amazon; Amazon distributes the words across the page as links rather than bunching them together in paragraphs. Over time, Barnes and Noble is becoming more like Amazon in this respect.

All sites eventually included navigation targeted at specific audiences.
Audience-based navigation—navigation labeled for a particular audience—appeared on in 1998, on Barnes and Noble in 2000, and on Amazon as early as 1999.

Invitations to subscribe to an email newsletter were offered inconsistently. didn’t include this feature until 1998. Barnes and Noble included it only in 1998 and 2001. Only Amazon consistently included this feature from 1995 to 2002.

Online and offline design
So what lessons can we learn about how these sites changed over time? How has design contributed to Amazon’s high growth and significant lead over the others? In general, Amazon found a winning formula and applied it consistently over time. In my mind, the successful design elements emulated offline shopping experiences in many ways.

Personally, I was surprised at how long these homepages had grown. Combined with the three-column layout, each page contains a great deal of information. This is quite like the perceptual experience of browsing in a physical store. When you walk down an aisle in a bricks-and-mortar store you can visually scan the shelves quite quickly. On these websites, the long, scrolling pages are analogous to aisles (major groupings of items) and the columns are analogous to shelves (more specific groupings of items). With a similarly natural, efficient motion, a visitor can scroll down the page and visually scan the three columns of product listings.

Amazon homepage
Amazon homepage
(January, 2002)
Click to enlarge.
Barnes and Noble homepage
Barnes and Noble homepage
(January, 2002)
Click to enlarge.

Amazon’s higher number and density of links, and placement of those links above the fold, also reminds me of the aggressive product positioning in a physical store. It’s like walking into a food market and immediately being overwhelmed with rows and rows of colorful fresh fruit, stimulating our eyes and engaging our appetites.

The prominent use of images and sparse use of text on Amazon again harks back to physical objects with simple labeling.

The arrival of navigation intended for specific audiences seemed inevitable. Especially for the book market, a children’s section was developed surprisingly late on these sites given the disproportionately high revenues that come from children’s books in traditional shopping venues.

In general, many of the functions of these pages have become commodities: search engines, shopping carts, authentication and store locators. But Amazon’s extensive personalization sets this site apart functionally. Personalization mimics a personal shopper or a local store employee who knows you. While the online recommendations aren’t always right on, neither is a human assistant.

Rate of change
Many studies have found that our performance using a software application improves over time as we become familiar with its interface. Gerald Lohse and his associates translated this finding into the realm of eCommerce websites using statistical analysis.11 They also found that website visitors learn to use a site more efficiently over time and that this increases their purchase rate. In simpler terms, it means familiar sites are easier for people to use, so familiar sites are where visitors will make purchases.

It follows that sites that can be learned more quickly will more quickly become familiar, increasing the amount of purchases. So a faster learning rate equals a higher purchase rate.

Furthermore, Lohse found that familiarity with a particular website makes visitors less likely to switch to a competitive site because of the effort and time needed to become familiar with another site. He refers to this behavior as “cognitive lock-in.” Essentially, we are creatures of habit. He applied this analysis to several eCommerce websites by measuring the number of visits per person, length of sessions, and timing and frequency of purchases. He found the learning rate significantly faster at Amazon than at Barnes and Noble.

The rate of design change supports this finding. Amazon had no major redesigns from 1999 to 2002, only adapting their design gradually to changing needs. Barnes and Noble significantly altered their navigation in 2000 and 2001. implemented major homepage changes in 1998 and 2000. Fewer redesigns make it easier for visitors to remain familiar with the site.

Many design elements on these websites are reminiscent of physical store layout, an approach to web design we should investigate further. Like physical stores, those designs should only change gradually to keep visitors buying. Continued analysis of other sites will hopefully help confirm or deny these findings.

It may be a fallacy to state, “Amazon is a successful business, therefore their website design is successful,” since many factors have contributed to their business success. And yet it’s hard to imagine them having such great success with a mediocre site. A similar eCommerce site launching today could do worse than examine and emulate the design elements that Amazon utilizes.

View all End Notes
Victor Lombardi writes, designs, and counsels, usually in New York City. His personal website is