It all started so well
In the spring of 2000, Egreetings engaged Argus to help with a redesign effort to improve the number of online greetings sent by users. At the time, Egreetings was typically ranked #3 among online greeting sites (behind Blue Mountain and American Greetings). The company’s core content was its collection of online greeting cards comprised of flash animations, animated GIF images and still images. Some were created in-house by a highly talented team of graphic artists, and the rest were licensed from outside art sources such as cartoons from the New York Times. Anywhere between 40,000 and 200,000 cards were sent each day, depending on the season. Argus was called in by Tim Scheele, the Senior Director of Publishing, to help with a number of goals:

• Increase card-sending statistics,
• Reorganize the collection (taxonomy/controlled vocabulary),
• Improve navigation and searching,
• Suggest key places for ads and promotions (need to “monetize”),
• Find an approach for organizing the music greeting collection,
• Improve the checkout process.

The team consisted of four Argonauts: a Lead Information Architect (myself), an Assisting Information Architect (Michele de la Iglesia), a Project Manager (Shawn Stemen), and a Usability Specialist (Keith Instone) who worked part time on the project to advise us.

We began our work by conducting a strategy and recommendations phase, knowing that Egreetings was hoping for major look-and-feel redesign with the target of a fall 2000 relaunch. An information architect’s approach should always involve an investigation of the content, the organizational context and the users. Often the user research part of the methodology gets less emphasis than it deserves because of time and budget constraints. However, this project contained user testing and research during each phase.

During the first two-thirds of the project, we accomplished a great deal. In the strategy and recommendations phase, we used techniques like card sorting and content analysis to determine facets for categorizing online greetings. (Facets are attributes or aspects used to classify content. For example, fruit facets might include color, region, size and type.) For more on exactly what we did during this phase, please see my article in the June/July 2002 issue of the Bulletin of the American Society for Information Science and Technology. Our deliverable at the end of this phase was a report that contained a number of recommendations on how to reorganize the site. We drafted a revision of the top level of the site’s taxonomy and recommended that it be made more consistent by focusing on “reason to send.” We suggested that the facets “recipient” and “card image content” be used as a means to systematically subdivide lower-level categories in the site. The “emotion” and “format” facets were to be used as additional metadata indexing elements separate from the “reason to send” taxonomy so that users could filter, narrow and search according to these secondary facets.

Furthermore, we encouraged them to create controlled vocabularies for these facets so the cards could be consistently indexed. We also delivered wireframes at this point, including one for the new main page of the site to show how to integrate our taxonomy suggestions into the site.

Egreetings liked our recommendations enough that some of our interface and category recommendations were implemented right away. They also hired us to help in the next phase in which we delved deeper into the task of redoing the taxonomy and controlled vocabularies. We created a new version of the taxonomy, taken three levels deep, consisting of over 850 terms.

In addition, we drafted lists of controlled terms for the other facets. Then we tested these with users and made changes accordingly. After we delivered the new taxonomy to Egreetings, we worked with their team to provide guidance on how to apply the terms consistently as they reclassified the entire collection of cards. By the middle of summer, the client was busy handling all of the details and issues that go with a major redesign.

The problem of search
At that point we began our work on the search interface, which was planned as a future enhancement to be added after the fall relaunch. From our first meetings with Egreetings, there was controversy about how to best implement search. From the experience of the Egreetings team and our own observations during testing, we knew that users were strongly drawn to browsing rather than searching when selecting cards. This has a lot to do with the mental model formed by shopping for traditional paper cards. However, after talking to several rounds of users, I felt that I had a good idea of what they would want in a search interface. While the majority seemed to enjoy the shopping and browsing process, there was a great opportunity to shorten this process for people in a hurry. Many users came to the site with a particular occasion, recipient or emotion for a card in mind. Some also looked for particular types of subject matter or images.

For a time, the site included a search interface which was intended to allow users to select different card criteria from categories like “collection” and “publisher.” There was a lot of overlap between these categories, and users frequently got zero results when they selected more than a few choices. We didn’t shed many tears when technical changes to the content management system made this search interface disappear by surprise. This allowed us to start from scratch.

Most search interfaces offer an open text box for a user to type in a query. In this case, we felt that the ubiquitous search box could be optional. Egreetings was cautious about getting involved with a search engine vendor because of the costs involved. From a practical perspective, any free-text search on a collection of a few thousand objects (rather than hundreds of thousands of objects) would need to be fairly sophisticated in order to avoid offering users null results. We felt we could provide a great deal of utility to users by exposing the choices and controlled vocabularies for selections that would be guaranteed to deliver results. The content management database Egreetings had built could be adapted for fielded searching.

Lastly, I had some definite opinions and ideas about how search should work:

• I felt that search should leverage the work we had done to define the facets and metadata for the cards.
• I was inspired by sites like Epicurious and Virtual Vineyards. These sites combine searching and browsing via databases of content objects and products that are well classified with rich metadata.
• The new search interface should NOT disappoint users with an empty results page. On an ecommerce and advertising site like Egreetings, it is important to suggest something to the user even if it doesn’t meet all of the criteria.

With these parameters in mind, I set out to create draft wireframes of a design. My philosophy was that by using a step-by-step wizard interface, we would create an interface that would be a shopping assistant to the user, which would allow them to narrow their choices down using faceted criteria. Each page of the wizard would concentrate on a separate facet. This would give users a reasonable number of cards to browse while making it less likely that they would be returned null results.

When we next met with Egreetings they liked many of my ideas, especially the “My philosophy was that by using a step-by-step wizard interface, we would create an interface that would be a shopping assistant to the user, which would allow them to narrow their choices down using faceted criteria. ”idea of making search an assistant. Their graphic designers came up with a superhero-like figure to be our “Card Finder.” However, they definitely didn’t like the idea of having the search interface separated into multiple wizard screens. Even when I explained that each page/facet could be optional, they felt that the users would be frustrated with too many steps and too many choices. The Egreetings team encouraged me to try creating a search interface with just one page for input and a smaller set of controlled vocabulary choices. We decided that the best way to settle on a design direction would be to create prototypes of both approaches and let the users help us decide.

The test
The test took place over the course of three days with 12 users. We used a market research firm in Southfield, Michigan to recruit a variety of representative users. We were lucky enough to be able to perform the tests in this firm’s well-appointed facilities, complete with a two-way mirror for observation and videotaping equipment so that Egreetings could also review the tests.

While planning this round of user testing, I got really excited about the idea of prototyping and how to get the most out this kind of test during the design process. A colleague and close friend, Dennis Schleicher, had just returned from the UPA 2000 conference with some great ideas on prototyping. I found that different professionals had diverging opinions on how to create effective prototypes. I learned a great deal by considering the arguments for both low- and high-fidelity prototypes, and came to some of my own conclusions about how to conduct this particular test. (See What IAs Should Know About Prototypes for User Testing for some of my ideas and research on prototyping.) After some pondering, we decided to proceed with testing the search inputs with a low- to medium-fidelity prototype created with Visio printouts that we cut up into pieces users could interact with. This made sense because it meant that we didn’t need much help from the Egreetings technical and creative teams to create high-fidelity interactive prototypes. At the time, they were much too busy with the relaunch to worry about that. However, they did help us by providing some high-fidelity screen comps to use in our test sessions to get reactions from users (we showed one of each style of search interface and another of the main page access points for navigating to the search page). In order to make the test feel as automated as possible, we asked the users to imagine interacting with a computer to perform tasks with both interfaces.

We prepared about eight tasks, such as, “You regularly share jokes with your favorite brother and his birthday is next week. Find a card for him.” We made sure to compose these so that they included multiple facets and there was more than one possible answer. During testing, we varied the order in which we gave the users the tasks and we also alternated the prototype presented first between users to eliminate any first-last bias. For each task, we asked the user to interact with the interface on the tabletop and to pretend that they were using the computer. We had laminated the Visio printouts so that the users could write on them with a thin whiteboard marker. One of us took notes while the other facilitated and simulated the feedback given by the computer. For example, in the wizard interface we wrote down the number of matching cards on a slip of paper as the user made each choice.

The wizard-style prototype was divided into five steps. Each step was presented on a separate page and showed as many choices as possible. Since the “reason to send” taxonomy was so large, we allowed users to drill down from main categories to sub-categories on this page. Any of the steps could be skipped, and users could elect to view the cards in their “bucket” at any point. Each time the user entered choices with the “continue” button, we wrote on a slip of paper to show the choices made so far and the number of cards in the bucket. We felt that this interactive feedback would help them understand the narrowing process. However, it didn’t work quite as well as we had planned.

For the one-page prototype we cut the interface into horizontal strips, one for each facet or screen element. Since there were only so many things that could be shown on the page, we subdued some of the options and presented only short lists of representative terms for each facet. If a user clicked the “show more reasons” link, we swapped the strip for one that showed the complete set of options for that term. This simulated a screen refresh that would expand the page.

Since it would have been very difficult to show users actual results, we stopped each task when users told us they were ready to hit the “search” button. The best way to get feedback from the users under these circumstances was to determine how confident they felt about the search. So, after each task, we asked a series of questions:

• How confident are you that the Card Finder would find cards that match the task?
  1 – Not confident at all
  2 – Not very confident
  3 – Somewhat confident
  4 – Confident
  5 – Very confident
• How many cards do you think the Card Finder might find?
• Do you have any comments on this version of the Card Finder?

We also devoted the second half of each test session to a separate activity devoted to the design of the results page. We asked them to select from cutouts of elements that could appear on a results screen and build their ideal search page.

Facing the music
Nobody likes being wrong. I pride myself in my efforts not to bias tests by leading with my own opinions. I must have done a good job. I was able to hide my pain over the three days of the test as the majority of users chose the one-page search interface over my wizard approach. Our testing and analysis revealed the following:

• Users preferred to see multiple criteria on a single page.
• They had difficulty noticing “show more” functionality, which expanded their options. Some preferred to see complete lists of options by default.
• Users offered opinions on the ordering and priority of criteria. “Reason to send” and “tone” were both high priorities. “Recipient” was more important than we anticipated.

The decision was clear —I may have lost, but the users won. In the aftermath of the test and the subsequent report we delivered to the client, I needed to create an interface based on the one-page paradigm. So I updated my design for the test according to the feedback from the users. In particular, I reorganized the way the facets were presented so that “Reason to send” was the most prominent and the other facets were given equal secondary emphasis. I relied heavily on the idea that users would see a relatively short page at first that could expand as needed. We also recommended that search provide “smart” results. Because the one-page search interface presented a high risk of offering null results, we specified that the search engine would need to present best-bets if not all criteria matched.

All for naught?
Once I got over my angst about losing the battle over the search interface, I felt really great about the conclusion of the project. I had swallowed my pride and designed a direction for the interface based on what the users wanted. Moreover, I felt good because our months of working with Egreetings finished with a very successful relaunch which happened on time. Even better, the initial statistics after the launch showed a positive impact from the new card categories and navigation that we’d recommended. Transactions (cards sent) and the number of visits went up immediately —a rare achievement in any redesign because it usually takes users some time to adjust when a site undergoes major changes. Egreetings had implemented roughly half of our recommendations and the others were put onto their priority list for future updates. We even received thanks from the CEO and VP.

However, a month or two after our consulting engagement with Egreetings ended, we began getting some disturbing news from them. First came the announcement of layoffs of a portion of the large and talented team that had been assembled to complete the work of the relaunch. By the end of the year, we read the news that the company had been sold to one of their largest competitors, American Greetings. The San Francisco office closed after a transition period and the staff was mostly dispersed as the site’s operations were transferred to AG’s headquarters in Ohio.

This was certainly disturbing news for me. I felt sorry that the Egreetings team would no longer be together. I also worried that the site on which I’d spent such a considerable amount of time and effort would be wiped out. That has not happened so far, and my solace is that the site lives on. Even though it has now adopted a new subscription model as a way to generate revenue, much of the taxonomy and interface recommendations remain. Of course, there have been some changes, but a good information architecture should be flexible enough to adapt with a site’s needs over time. My short list of grievances includes my opinion that the tab navigation is no longer relevant. Also, my recommendations about filtering and searching never saw the light of day —Card Finder never got to fly. Nonetheless, I got to have the experiences of low-fidelity, paperprototyping and designing a faceted search interface. And I can always fantasize that, maybe someday, one of the folks at American Greetings will find our report, dust it off and give our ideas a try.

Degree of fidelity

Low fidelity
You can use hand-drawn sketches to create a paper prototype. If you go this route, you may also want to help your users get into the spirit of things during the test by creating a complete low-fidelity, paper environment. This could include a cardboard box made to look like a computer and an object to hold to point and click with. These techniques help users to suspend their disbelief and get their imaginations involved so that they can better visualize the interface. The advantage of using rough sketches is that users will have an easier time suggesting changes. They may even grab a pen and start making their own changes (Datz-Kauffold and Henry).

In theory, low-fidelity sketches are also a time-saver, but this really depends on your point of view. Personally, I like to draw diagrams and wireframes in Visio where I can revise and move things around without erasing and redrawing. If you prefer to work this way too, and if time allows, you can always have those Visio drawings hand-traced or use them as a roadmap for making sketches to test with. You might even find a graphics tool with a filter that will convert a Visio-generated graphic into a hand-drawn sketch with wavy lines.

High fidelity
This approach takes you as close as possible to a true representation of the user interface —screen-quality graphics. All of the blanks on the page are filled in, and it looks good. However, you might not have all of the technical or backend problems worked out yet, or you might have only a small part of the entire site rendered. That’s why it’s still considered a prototype. For example, it might consist of a small series of Photoshop images or HTML pages with just enough functional links to convey the feel of the site’s flow. You may need to enlist the help of a graphic designer or web developer to build these in a reasonable amount of time. Advocates for high-fidelity prototypes argue that they are easier for users to understand just by looking at them. There is no disbelief to overcome, and it is easier to determine when they really do not understand the design. If you choose a high-fidelity prototype, make sure the you have enough of the design fleshed out so that users can complete several tasks. Decide on these tasks early, so you know which areas of the design need to be represented for your tests. Otherwise, you will be in for a great deal of preparation work.

Medium fidelity
In the grand tradition of Goldilocks, I find myself drawn to the middle approach. A medium-fidelity approach tends to include some visual design and a level of detail somewhere between high and low fidelity. Does this sound familiar? As an information architect, I’m accustomed to creating wireframes I can hand off to decision-makers, graphic designers, web developers and programmers. An information architecture wireframe is NOT graphic design. I swear, it’s really not!!! But… I’ll admit that it has enough visual design to convey a rough version of the user interface. Because I create these with drawing tool software, they tend to have more polish than hand-drawn diagrams. Hakim and Spencer are champions for medium-fidelity prototypes because they fit more seamlessly into the design process while providing more realism for users. I found this to be true during a project to design a search interface for Egreetings with my colleagues at Argus. I created rough draft wireframes for the prototype, and after testing I revised them for use in my deliverables.

Interactivity
Interactivity describes how your prototype behaves. Does your prototype react to user inputs with feedback? Can they “click” on something to go to another page or fill in a form? Will buttons appear to depress and drop-down menus work?

Static prototypes
Prototypes used for testing are static if they are pages or page elements shown to users, which don’t provide any feedback. It can sometimes work well to show a page to a user and ask them to explain it to you or to guess where they can go from here. In this kind of test, the user interprets the prototype rather than interacts with it. This is a good way to validate your design by checking to make sure users understand it. It’s also easy to score this sort of test when you have a standard list of questions to ask about each page.

Automated
Automated prototypes allow users to make choices that cause changes. The testing prototype provides the user with feedback. Elements are “clickable” and forms can be filled out. The interface reacts to the user while the tester observes. One way to do this is to create the prototype in HTML or some application that allows interactive elements such as Flash, Visual Basic or even PowerPoint.

Another way to achieve a kind of pseudo-automated interactivity when you have chosen a paper prototype is to pretend (Datz-Kauffold and Henry). Have you ever seen children at play pretend that they are driving a car by setting up chairs for the front and back seats, drawing a dashboard on a cardboard box, and using a Frisbee for the steering wheel? If you have set up the right environment for your users, you can ask them to pretend scraps of paper on a table are their computer screen. When they “click” on a drop-down menu by touching the element with a pointer, a tester assigned to the role of the computer provides feedback by swapping the closed menu for an open one that shows choices. The “computer” may need to write on some elements before showing them to the user, i.e., “Your search retrieved 523,621 hits.” It takes a few minutes to get test participants used to the idea, but if you encourage them to have fun with it you will learn a great deal. You can also easily try out different possible reactions to user input.

This method worked well during the Egreetings project. We especially emphasized the technique of asking the users to click and then provide feedback. We found it useful to laminate the screen components so we didn’t need to produce a clean copy of the test for every subject. The users could write on the laminated pieces with thin whiteboard markers when making selections and entering search criteria. Of course, this meant that we needed to take careful notes because of the need to erase between each test subject.

Here are some other tips to try for low-fidelity testing with simulated interactivity:

• Bring extra paper so you or the respondent can sketch out an idea if the opportunity arises.
• As with any user test, it really helps to ask the respondent to think aloud.
• If you have the luxury, bring a team of three to the test: someone to take notes, someone to play the “computer” and another to facilitate.
• Use a piece of posterboard as your “screen.”
• Cut your design into separate pieces or zones as appropriate and ask the user to rearrange them in the order they prefer.
• Attach the folder tabs that come with hanging files to components so they are easier to grab.
• Invite users to throw away or cross out components that they don’t think are important.
• Number the pieces so that you can easily refer to them in your notes and keep them organized.
• If you do decide to bring separate copies of the test materials for each session, tape down the components to a larger piece of paper as arranged by each user so you have these artifacts to analyze later.

Prepare a kit for yourself containing:

• Scissors and tape,
• Different sizes and varieties of sticky notes (which make great drop-down menus),
• Markers and pens in various colors and sizes,
• Paper clips and binder clips for keeping slips of paper organized, and
• Objects that the user can pretend are the mouse pointer, such as a feather or a small toy.

Medium
There are many possible combinations to choose from for building your prototype. One of the first choices to make is whether you want to have your prototype viewed on an actual computer screen or if you’ll be working on a tabletop with a paper prototype. Believe it or not, fidelity and interactivity are independent of the medium you choose. It’s probably most natural to think of the extreme cases. An automated HTML prototype is often high-fidelity and, of course, the medium is a computer screen. Likewise, a natural medium for a low-fidelity automated interactive prototype is hand-drawn sketches on paper. However, you can also have the following:

• Low to medium-fidelity wireframes built in PowerPoint that show only lines and boxes with text;
• animation features provide automated interactivity,
• Static Photoshop prototype pages shown to users on a computer screen, or
• Same as above, but printed out in color on paper.

Mixing the variables
You can mix these three variables (fidelity, interactivity and medium) in many different combinations. The exact combination you choose should match the goals you determine for your testing. Possible goals for an IA prototype include:

• Testing the effectiveness of labels and icons.
• Finding out the right balance of depth and breadth of a topical hierarchy.
• Determining the right options to offer for narrowing a search.
• Choosing the most important metadata elements to show on a search results screen.
• Settling the question of whether your target audience accomplishes tasks better with a task-oriented organization scheme or with a topical organization scheme.

If you live and breathe NetObjects Fusion and don’t have much time, your preference might be to create a medium-fidelity prototype. That way you could test that sitemap you are working on using some rough placeholder graphics or text instead of the finished graphic design. How you mix the variables depends on the time and budget you have available, as well as your work style. Try experimenting with different approaches to learn how prototyping will work best with your design process.