The Story Behind Usability.gov

by:   |  Posted on

When Detroit’s automotive engineers design a new car, they often bring in real drivers who sit in the seats, mash the gas pedals, and pump the brake. This is the engineers’ approach to involving users in the process of designing new cars that people want to drive—and can drive. Their approach is similar to the thinking that led the National Cancer Institute’s (NCI) Communication Technologies Branch to formally encourage the designers of government information websites to involve users in the design process. We created Usability.gov, a place to share our knowledge about user-centered web design and why it works with our colleagues.

We are gratified to see clear results from Usability.gov. Government web designers are using more user-centered design practices, and web designers in general appear to be more cognizant of the user’s mindset.

Today, Usability.gov has earned a following among technology professionals. For the uninitiated, Usability.gov is a one-stop source for government web designers to learn how to make websites more usable, useful, and accessible. Our site addresses a broad range of factors that go into web design and development: how to plan and design usable sites by collecting data on what users need; how to develop prototypes; how to conduct usability testing; and how to measure trends and demographics. We have packaged our core knowledge into a specific set of evidence-based guidelines for user-centered web design. In addition, the site offers case study information in a section called Lessons Learned.

Home Page of the Usability.gov website  

What many do not know is the story behind Usability.gov, and knowing that story puts our work in context. It’s a story that underscores the critical role that Usability.gov plays in the electronic communication of complex cancer information to very diverse audiences. One minute, a researcher seeking grant information is pulling up an NCI website for details on what grants are available and where to apply. The next minute, an ordinary citizen is frantically searching NCI websites for any informationæany cluesæabout a type of cancer for which the doctor is testing them. Every day, NCI disseminates life and death information. Usability.gov ensures that users and their web behaviors are kept in mind when designing sites.

The seeds for Usability.gov were sown in early 1999 when the popular CancerNet web site came up for a redesign. As usual, we began by seeking input for the new design from technical professionals: web designers, content writers, engineers. Our “kitchen cabinet” also included users. But the opinions from this broad group of professionals and laymen were as diverse as their backgrounds. Whose ideas were right?

Our director, Janice Nall, decided that we needed a methodology to show that what we were doing would produce an end result that was better than what we started with. In fact, we had to be able to quantifiably measure that CancerNet’s new face was better than the old face, to offer proof beyond a lot of people saying it looked better.

To accomplish this objective, we decided to collect quantitative data about CancerNet’s users and their needs as part of the design process. An online questionnaire and in-person interviews turned up some revealing information. We learned that one-third to one-half of CancerNet users were first-time visitors who were often totally unfamiliar with the site. This fact raised obvious questions: With so many new users, was the site easy enough to use? Could users find the information they needed on the site quickly and easily? These were critical questions in light of the kind of information that CancerNet provided to the public.

Given these questions, we began testing the site, an experience that furthered the need to develop a formal way to collect and share our knowledge for future reference. We conducted user tests with doctors, medical librarians, cancer patients, researchers, and others who we expected would be regular visitors. What we learned from testing was as surprising as what we learned from our questionnaire and interviews: some icons were not clearly clickable, many links were confusing, our terminology did not match our users’, and core information appeared to be buried or lost within the site. These were not mere glitches, but conceptual and foundational challenges that needed to be addressed.

To be thorough, our testing was iterative; we built on prototypes and brought in new sets of users to test each new version. We continually collected information to see if new problems cropped up, seizing on every comment, even something as simple as, “What is that there for?” We were like those automotive engineers in Detroit, watching test participants’ every move and examining their every facial expression.

 
User-centered design tips on CancerNet from Usability.gov’s Lessons Learned section. Click to enlarge.

Today, when you visit Usability.gov, you get a sense of how these tools help government and other web designers to avoid our early mistakes. Whether you read our case study about the redesign of CancerNet in our Lessons Learned section, or read our guidelines about testing issues such as scenario writing, user recruiting, goal establishment, or data compilation, you will see our picture of user-centered web design in action.

We are pleased with CancerNet’s redesign. In the past year or so, the site has won four content and design awards, and CancerNet recently merged with several existing sites, including Cancer.gov, into one portal site. But just as importantly, we are gratified to see clear results from Usability.gov. Government web designers are using more user-centered design practices, and web designers in general appear to be more cognizant of the user’s mindset. What Usability.gov demonstrates is that web design is not about flash and splash. It’s about transmitting useful information that users want—and need—in a way that helps them find what they are looking for.

Sanjay Koyani works for the Communication Technologies Branch of the National Cancer Insitute. He can be reached at .

Taking the “You” Out of User: My Experience Using Personas

by:   |  Posted on

The best laid plans…
In 1999, I co-founded a small San Francisco-based start-up called Pyra. Our plan was to build a web-based project management tool and we chose to focus initially on web development teams for our target audience since, as web developers ourselves, we had intimate knowledge of the user group. At the time the team consisted of three people: my co-founder, our lone employee and myself. We considered ourselves to be good all-around developers: competent in both interface and back-end development. We also assumed we were developing our product (called “Pyra” for lack of a better name at the time) for people just like us, so we could make assumptions based on our wants and extrapolate those desires for all users.

At this time, Microsoft had just released Internet Explorer 5 (IE 5) for Windows and we were anxious to use its improved standards support and DHTML in our application to make the interface as whizbang as possible. By limiting our audience to IE 5, we decided we would be able to deliver the most robust application, one that was sure to impress potential users and customers. Later, we told ourselves, we’d go back and build out versions with support for Netscape and Macintosh. So we set to work building the coolest web application we could, taking full advantage of the latest wizardry in IE 5 for Windows. Development was chugging along when Alan Cooper’s “The Inmates Are Running the Asylum” was released and I picked it up. When I got to the chapter discussing the use of personas, I was intrigued. Though I was confident in our approach, creating personas sounded like a useful exercise and a way to confirm we were on track.

Discovering Personas

“Not only were the personas not all like us—our personas wouldn’t even be able to use the system we were building for them!”

Cooper’s personas are simply pretend users of the system you’re building. You describe them, in a surprising amount of detail, and then design your system for them. Each cast of personas has at least one primary persona, the person who must be satisfied with the system you deliver. Since you can’t build everything for every persona (and you wouldn’t want to), the establishment of the primary persona is critical in focusing the team’s efforts effectively. Through the use of personas, the design process moves away from discussions that are often personal in nature (“I’d want it to work this way.”) or vague (“The users like to see all the options on the home page.”) . It becomes a series of questions and answers based on a concrete example from which the team works (“Mary, the primary persona, works from home via dialup four days a week, therefore downloading an Access database isn’t an option.”). In our case, the development of personas helped us recognize that the target audience we’d chosen, web development teams, wasn’t as homogenous as we first assumed. Not everyone who’s involved in web development is gaga for DHTML or CSS—some people on the team might not even know what those acronyms stand for, a simple fact we’d failed to consider up until this point.

Our team stopped working to discuss personas and Cooper’s approach and we agreed it sounded important enough to devote some time to it. As we sketched out our various personas (a project manager for a large company whose corporate standard was Netscape 3, a web designer who worked on a Mac, an independent consultant who worked from home), it became apparent we had made some bad assumptions. Not only were the personas not all like us—our personas wouldn’t even be able to use the system we were building for them! We’d been so blinded by our own self-interest we failed to realize we were building a useless team product. Sure, it would have been great as an example of what we hoped to build, impressive to any engineer or web developer, but a manager might not be able to access it. We were cutting ourselves off from the people who would most likely make the decision to use the tool—and no project team would signup for Pyra because an entire project team couldn’t use it.

We were a month away from releasing the beta version of Pyra at this point, but we knew what needed to happen. We had to go back and redo our application to work for Netscape and IE, for Windows and Macintosh, and in doing so, we needed to reevaluate our tool using our personas (specifically our primary persona) rather than ourselves or the mythical “user” to guide our decisions. So that’s what we did, pulling out all our beloved DHTML and remote scripting so our 37-year-old project manager persona could access the application from her home office in Seattle on a Saturday afternoon. Though the rework delayed our beta release by two months, it resulted in a tool our potential customers could use immediately.

Learning hard lessons
Through the process of developing personas, the mistakes we’d made became clear to us:

Mistake #1: We chose flashy technology over accessibility.
We allowed the geeky part of our personalities, with its lust for the newest and greatest ways of doing things, to overwhelm the decision-making process. Though there was a sense at the beginning that we needed to support other platforms, we let our desire to use the newest “toys” change the priority of doing so. This is a common mistake programmers and engineers make but one which can be avoided through the use of personas. Interestingly, when we redid Pyra based on our personas’ needs, we didn’t lose any of the previous functionality. We only changed how it was done, e.g., reverting to less elegant page reloads rather than DHTML client-side changes. The previous version had only been impressive to fellow geeks like ourselves, but we hadn’t realized that. More importantly, the essential quality of the tool was never lost, but by redoing it, it become available to many more people.

Mistake #2: We assumed users would be more impressed by a robust interface they couldn’t use than by a less elegant application that they could use.
Again, our technical hubris blinded us into thinking that potential customers would be impressed by how we built our functionality, not by what the underlying features were. We let our wants come between our product and our users.

Mistake #3: We thought we were the primary persona.
While we shared common goals with our some of our personas, and though one of the personas we developed was very similar to the members of our team, none of us were the primary persona. This crucial distinction between primary personas and secondary personas forced us to realize the interface we designed shouldn’t be driven by our wants or needs, even as members of a web development team. Defining a primary persona prevented us from releasing our original tool with its accessibility failures.

Less than a month after the beta release of Pyra, we released a second tool, Blogger. Though we didn’t create formal personas for Blogger users, the experience we gained by using personas infused our company’s approach to building web applications. Any time the word “user” was mentioned, questions flew: “What user? Who is she and what’s she trying to do?” Our work with personas increased our awareness of our audience and their varying skill levels and goals when using the application. The use of personas helped move all our discussions about the application, not only those related to the interface, away from the realm of vagaries and into tangible, actionable items. (“It should be easy to create a new blog.” “Easy? Easy for whom?” “It should take less than a minute to get started.” “It should take less than a minute for my grandmother to get started,” etc.) We developed a system of familiar, conversational personas on the fly, focusing on the primary persona without going through the formal process.

In retrospect, some of this sounds like common sense, and yet time and time again I find myself looking at an interface and making assumptions based on how I’d like it to work. Like a recovering substance abuser, it’s a constant challenge for me to refrain—I can always imagine that I’m the user. Even if your budget or timeline doesn’t allow for the development of formal personas, you can still benefit through the use of informal “conversational” personas, like we did while building Blogger. It takes discipline to break the old assumption habit but the more I use personas, the easier it becomes. I’ve carried the lessons I’ve learned through their development with me for the past three years to other projects and engagements; the use of personas resulted in a fundamental shift in the way I approach not only interface design but application architecture as a whole.

For more information:
Alan Cooper, The Inmates Are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity. SAMS, 1999.

The Evolving Homepage: The Growth of Three Booksellers

by:   |  Posted on

Web design is expensive. Web designers earn upwards of $50,000 a year1, information architects earn even more.2 During the heyday of web design—the late 1990s—designing a large commercial website could cost as much as designing a medium-sized building. During this period, commercial websites were created and then often completely replaced with redesigned versions a short time later. Today the redesigning continues, albeit at a slower pace. What is the return on this design investment? A report on online ROI from Forrester finds that many commercial sites fail to even try to measure the effectiveness of design changes.3

What lessons have we learned about how design improves the interface between customers and companies?

The web has been with us for about a decade now. We’ve seen some obvious trends, such as greater use of multimedia, search engines, and increasingly sophisticated markup techniques. But these trends were facilitated by changes in technology. What lessons have we learned about how design improves the interface between customers and companies? Perhaps we can start by asking how websites have actually changed over time, and from that we can learn how websites should change in the future.

To start working toward an answer, I compared three eCommerce sites: Borders.com, BarnesandNoble.com, and Amazon.com. Much of the media’s coverage of these websites, especially coverage of Amazon.com, discusses the business models, corporate cultures, and finances of the companies. Since the medium of interaction with these companies is the website, it’s ironic that the media rarely critiques the site design and its effect on business performance.

Because it is the homepage that carries the most responsibility for guiding customers, I examined the homepages of all three sites from a number of years, using screenshots from the Web Archive4. Presumably these large retailers had a great deal to gain, and lose, with these substantial online ventures. By comparing design decisions over time among the three sites, I hoped to discover lessons from their extensive and expensive design experience.

The companies
Competition is fierce in the online bookselling market, currently erupting in offers of “free shipping.” All three companies have annual revenues in the billions of dollars.

Barnes and Noble, which runs a large chain of stores in the United States, claims the largest audience reach of any bricks-and-mortar company with Internet presence.5 Yet, both they and Borders were put on the defensive when Amazon’s growth rocketed. During December, 2001 BarnesandNoble.com attracted over 10 million unique visitors,6 compared to Amazon’s 40 million visitors.7

Borders is the second largest bricks-and-mortar book chain in the U.S. 8 In April 2001, after operating their own online bookstore for several years, Borders announced an agreement to use Amazon’s eCommerce platform to power a co-branded website.

Amazon claims to be the leading online shopping site, having expanded their selection to music, video, auctions, electronics, housewares, computers and more.9 By February of 2002, Amazon, which had pursued a get-big-quick strategy typical of internet companies in the late 1990s, announced its first profitable quarter.10

Criteria
I first studied these sites quantitatively looking for clear trends over time. I then critiqued them in a more qualitative way based on my own experience as both an in-house website designer and as an information architecture consultant.

There are many criteria that could be examined in such a study. I limited myself to those that would, I hoped, reveal as much as possible about the business intent of the design. I looked at criteria such as the type and size of layout, the type and amount of navigation, the amount of images and text, and functionality specific to the industry. Detailed results can be seen in the attached spreadsheet (PDF, 75k).

Analysis

Chart showing growth in length of homepages over time
Click to enlarge.
Note: Missing data due to imperfect records at the Web Archive.

All three sites use very long screens to display content on their homepages.
Using a browser window with a constant width, we can compare the vertical size of each site (all screen references assume an 800 by 600 pixel monitor). The Borders.com homepage grew from a vertical size of about 917 pixels in 1996 to over 3,000 pixels in 1999. Barnes and Noble’s homepage has hovered around 1,500 pixels for the last several years. Amazon’s homepage, which began at only 626 vertical pixels in 1995, stands at roughly 2,156 pixels today. In a web browser, that equals five scrolling screens of information.

Borders.com homepage above the fold, 1999
Borders.com above the fold (1999) Click to enlarge.
Barnes and Noble homepage above the fold, 1999
Barnes and Noble above the fold (1999) Click to enlarge.
Amazon homepage above the fold, 1999
Amazon above the fold (1999) Click to enlarge.

Note: Incomplete web pages are due to imperfect records at the Web Archive.

All three sites evolved to use three-column layouts.
In 1995 and 1996 respectively, Amazon and Borders.com used single-column layouts. By 1999, both of these sites as well as Barnes and Noble used three column layouts.

Amazon has consistently placed more links above the fold.
In 1999, the Borders site displayed only about eight links “above the fold” (the top portion of the screen that is viewable without scrolling). Both Barnes and Noble and Amazon had significantly more links above the fold in 1999, 30 and 48 respectively. Amazon averaged 43 links above the fold between 1999 and 2002 versus only 27 links for Barnes and Noble during the same period.

Through the years, the density of links on Borders.com was half of that on Barnes and Noble or Amazon.
The density of links has varied over time, but as of 2002 both Barnes and Noble and Amazon stood at about one link for every 15 vertical pixels of screen real estate. Historically, the highest link density at Borders.com was one link for every 28 vertical pixels.

Amazon communicates using images and links rather than text descriptions.
From 1999 through 2001, Amazon used more images and fewer text descriptions than Barnes and Noble. In 2002, both sites used about 560 words per page, yet the density of words was 33 percent lower on Amazon; Amazon distributes the words across the page as links rather than bunching them together in paragraphs. Over time, Barnes and Noble is becoming more like Amazon in this respect.

All sites eventually included navigation targeted at specific audiences.
Audience-based navigation—navigation labeled for a particular audience—appeared on Borders.com in 1998, on Barnes and Noble in 2000, and on Amazon as early as 1999.

Invitations to subscribe to an email newsletter were offered inconsistently.
Borders.com didn’t include this feature until 1998. Barnes and Noble included it only in 1998 and 2001. Only Amazon consistently included this feature from 1995 to 2002.

Online and offline design
So what lessons can we learn about how these sites changed over time? How has design contributed to Amazon’s high growth and significant lead over the others? In general, Amazon found a winning formula and applied it consistently over time. In my mind, the successful design elements emulated offline shopping experiences in many ways.

Personally, I was surprised at how long these homepages had grown. Combined with the three-column layout, each page contains a great deal of information. This is quite like the perceptual experience of browsing in a physical store. When you walk down an aisle in a bricks-and-mortar store you can visually scan the shelves quite quickly. On these websites, the long, scrolling pages are analogous to aisles (major groupings of items) and the columns are analogous to shelves (more specific groupings of items). With a similarly natural, efficient motion, a visitor can scroll down the page and visually scan the three columns of product listings.

Amazon homepage
Amazon homepage
(January, 2002)
Click to enlarge.
Barnes and Noble homepage
Barnes and Noble homepage
(January, 2002)
Click to enlarge.

Amazon’s higher number and density of links, and placement of those links above the fold, also reminds me of the aggressive product positioning in a physical store. It’s like walking into a food market and immediately being overwhelmed with rows and rows of colorful fresh fruit, stimulating our eyes and engaging our appetites.

The prominent use of images and sparse use of text on Amazon again harks back to physical objects with simple labeling.

The arrival of navigation intended for specific audiences seemed inevitable. Especially for the book market, a children’s section was developed surprisingly late on these sites given the disproportionately high revenues that come from children’s books in traditional shopping venues.

In general, many of the functions of these pages have become commodities: search engines, shopping carts, authentication and store locators. But Amazon’s extensive personalization sets this site apart functionally. Personalization mimics a personal shopper or a local store employee who knows you. While the online recommendations aren’t always right on, neither is a human assistant.

Rate of change
Many studies have found that our performance using a software application improves over time as we become familiar with its interface. Gerald Lohse and his associates translated this finding into the realm of eCommerce websites using statistical analysis.11 They also found that website visitors learn to use a site more efficiently over time and that this increases their purchase rate. In simpler terms, it means familiar sites are easier for people to use, so familiar sites are where visitors will make purchases.

It follows that sites that can be learned more quickly will more quickly become familiar, increasing the amount of purchases. So a faster learning rate equals a higher purchase rate.

Furthermore, Lohse found that familiarity with a particular website makes visitors less likely to switch to a competitive site because of the effort and time needed to become familiar with another site. He refers to this behavior as “cognitive lock-in.” Essentially, we are creatures of habit. He applied this analysis to several eCommerce websites by measuring the number of visits per person, length of sessions, and timing and frequency of purchases. He found the learning rate significantly faster at Amazon than at Barnes and Noble.

The rate of design change supports this finding. Amazon had no major redesigns from 1999 to 2002, only adapting their design gradually to changing needs. Barnes and Noble significantly altered their navigation in 2000 and 2001. Borders.com implemented major homepage changes in 1998 and 2000. Fewer redesigns make it easier for visitors to remain familiar with the site.

Conclusion
Many design elements on these websites are reminiscent of physical store layout, an approach to web design we should investigate further. Like physical stores, those designs should only change gradually to keep visitors buying. Continued analysis of other sites will hopefully help confirm or deny these findings.

It may be a fallacy to state, “Amazon is a successful business, therefore their website design is successful,” since many factors have contributed to their business success. And yet it’s hard to imagine them having such great success with a mediocre site. A similar eCommerce site launching today could do worse than examine and emulate the design elements that Amazon utilizes.

View all End Notes
Victor Lombardi writes, designs, and counsels, usually in New York City. His personal website is http://www.noisebetweenstations.com.