Card Sorting: A Definitive Guide

“Card sorting is a great, reliable, inexpensive method for finding patterns in how users would expect to find content or functionality.”

Introduction

Card sorting is a technique that many information architects (and related professionals.) use as an input to the structure of a site or product. With so many of us using the technique, why would we need to write an article on it?

While card sorting is described in a few texts and a number of sites, most descriptions are brief. There is not a definitive article that describes the technique and its variants and explains the issues to watch out for. Given the number of questions posted to discussion groups, and discussions we have had at conferences, we thought it was time to get all of the issues in one place.

This article provides a detailed description of the basic technique, with some focus on using the technique for more complex sites. This article does not cover some issues such as the use of online tools, which will be covered in a future article.

Why

Card sorting is a quick, inexpensive, and reliable method, which serves as input into your information design process. Card sorting generates an overall structure for your information, as well as suggestions for navigation, menus, and possible taxonomies.

While card sorting might not provide you with final structure, it can help you answer many questions you will need to tackle throughout the information design phase. For example, more than likely there will be some areas that users disagree on regarding groupings or labels. In these cases, card sorting can help identify trends, such as:

• Do the users want to see the information grouped by subject, process, business group, or information type?
• How similar are the needs of the different user groups? >
• How different are their needs?
• How many potential main categories are there? (typically relates to navigation)
• What should those groups be called?

Card sorting can help answer these types of questions, making you better equipped to tackle the information design phase.

Definition

Card sorting is a user-centered design method for increasing a system’s findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality, into groups that make sense to users or participants.

According to Information Architecture for the World Wide Web, card sorting “can provide insight into users’ mental models, illuminating the way that they often tacitly group, sort and label tasks and content within their own heads.”

Card sorting is a great, reliable, inexpensive method for finding patterns in how users would expect to find content or functionality. Those patterns are often referred to as the users’ mental model. By understanding the users’ mental model, we can increase findability, which in turn makes the product easier to use.

Method

There are two primary methods for performing card sorts.

• Open Card Sorting: Participants are given cards showing site content with no pre-established groupings. They are asked to sort cards into groups that they feel are appropriate and then describe each group. Open card sorting is useful as input to information structures in new or existing sites and products.
• Closed Card Sorting: Participants are given cards showing site content with an established initial set of primary groups. Participants are asked to place cards into these pre-established primary groups. Closed card sorting is useful when adding new content to an existing structure, or for gaining additional feedback after an open card sort.

Closed card sorting will be detailed in a future article.

Advantages and disadvantages

As with any other method, card sorting has both advantages and disadvantages. Keeping these in mind will help you determine whether the technique is appropriate for your situation and make decisions about how you run the activity.

Advantages

• Simple – Card sorts are easy for the organizer and the participants.
• Cheap – Typically the cost is a stack of 3×5 index cards, sticky notes, a pen or printing labels, and your time.
• Quick to execute – You can perform several sorts in a short period of time, which provides you with a significant amount of data.
• Established – The technique has been used for over 10 years, by many designers.
• Involves users – Because the information structure suggested by a card sort is based on real user input, not the gut feeling or strong opinions of a designer, information architect, or key stakeholder, it should be easier to use.
• Provides a good foundation – It’s not a silver bullet, but it does provide a good foundation for the structure of a site or product.

Disadvantages

• Does not consider users’ tasks – Card sorting is an inherently content-centric technique. If used without considering users’ tasks, it may lead to an information structure that is not usable when users are attempting real tasks. An information needs analysis or task analysis is necessary to ensure that the content being sorted meets user needs and that the resulting information structure allows users to achieve tasks.
• Results may vary –The card sort may provide fairly consistent results between participants, or may vary widely.
• Analysis can be time consuming – The sorting is quick, but the analysis of the data can be difficult and time consuming, particularly if there is little consistency between participants.
• May capture “surface” characteristics only – Participants may not consider what the content is about or how they would use it to complete a task and may just sort it by surface characteristics such as document types.

When should card sorting be used?

Card sorting is a user-centered, formative technique. It should be used as an input to:

• designing a new site
• designing a new area of a site
• redesigning a site

Card sorting is not an evaluation technique and will not tell you what is wrong with your current site.

Card sorting is not a silver bullet to create an information structure. It is one input in a user-centered design process and should complement other activities such as information needs analysis, task analysis, and continual usability evaluation. It is most effective once you have completed:

• research into what users need out of the site
• a content (functionality) audit/inventory (for an existing site) or detailed content list (for a new site). For an existing site, it is crucial that the content inventory is examined carefully to include only content that is needed by users.

Card sorting will provide benefit to most sites, but can be challenging to use against some sets of information. The table below summarizes when card sorting works well and provides good results, and when it is challenging both to run and to analyze.

	Easy	Challenging
Site size	Small	Large
Type of content	Homogeneous (e.g., product catalogues, lists of services, directories of web sites)	Heterogeneous (e.g., intranets, government web sites)
Complexity of content	Participants understand most of the content	Complex or specialist content

Table 1.1

For sites with characteristics listed in the last column, card sorting will provide less direct input into the information structure; you may need to undertake a range of card sorts and more user-centered design activities.

Card sorting can be useful to demonstrate to people that others think differently. We have successfully included it as an exercise in workshops for web site and intranet authors.

Preparation

Preparing for a typical card sorting exercise requires the following:

1. Selecting content
2. Selecting participants
3. Preparing the cards

Selecting content
The first step in conducting a card sort is to determine the list of topics. This list should be drawn from a wide variety of sources:

• existing online content
• descriptions of business groups and processes
• planned applications and processes
• potential future content

By including potential future content it becomes possible to create a structure that not only works now, but also will work for future content and functionality. Adding new items in the future should require minimal rework if the structure is designed correctly.

Granularity and sampling content.
Content selected for the cards can be individual pages, functionality, small groups of pages, or whole sections of the site. Be consistent with your chosen granularity — participants will find it difficult to group content at different levels of granularity.

If you choose to use small groups of pages or sections of the site, ensure that the groups are of items that belong together. For example, don’t include a grouping of “media releases,” as this may not suit users and their tasks (they may prefer individual media releases to be grouped with other pages of similar topic.). Instead, include some individual media releases and see what participants do with them.

The content for the card sort should be representative of the site (or the part of site that you are investigating). It is important to ensure that the content has enough similarity to allow groupings to be formed. If the content chosen is too varied, participants will not be able to create natural groupings.

Selecting participants

Card sorting may be performed individually or in groups. Keep in mind that the exercise will be performed multiple times. So, if you’re using individuals, try and get seven to ten for a good sampling. If you’re using groups, our preferred method, five groups of three participants per group (a total of 15 participants) works best. Whether you choose to use individuals or groups, the most important aspect of selecting participants is that they come from and are representative of your user group. (If you have multiple user groups, it is important to include a representative sample from each, as they may view the information differently).

Scheduling individuals can be easier than scheduling groups of participants, especially if you have individuals located remotely. However, individuals can find it difficult to sort larger numbers of cards, providing less valuable input.

A benefit of group sorts is that they typically provide richer. data than individual sorts. Whereas individuals need to be prompted to “think aloud,” groups tend to discuss their decisions aloud openly. Combine this with the group’s ability to handle larger numbers of cards effectively and their tendency to walk each other through questions about content or functionality, and you have a rich data set with greater insight into users’ mental model.

The number of groups needed may depend upon the size and complexity of the site or product. However, we’ve found that patterns tend to emerge within five groups. These patterns become the basis for the site or product’s information architecture.

When inviting participants, it’s not necessary to tell them they’ll be performing a card sort. Instead, simply tell them they’ll be asked to perform a simple task, or exercise that will help you (re)design the site or product. Additionally, let them know they don’t need to prepare ahead of time; they should simply come as they are.

Preparing the cards

Each item on your list should be placed on a card. The labels you use on the cards are extremely important. They should be short enough that participants can quickly read the card, yet detailed enough that participants can understand what the content is. When necessary, the label can be supplemented with a short description or image on the back of the card.

Labels may be printed on standard (Avery) mailing labels, or printed by hand. We recommend using mailing labels as this saves time and the labels will be more legible.

Mark each card with a letter or number to make analysis easier once the sorting is done.

You can use whatever cards you have on hand, but we recommend 3″ x 5″ (10cm x 15cm). Index cards are durable, easy to see from a distance, and readily available at office supply stores. You may also use Post-it® notes, but it is our experience that cards are more durable and easier to handle.

Number of cards.

While there is no magic number, we have found that between 30 and 100 cards works well. Fewer than 30 cards typically does not allow for enough grouping to emerge and more than 100 cards can be time consuming and tiring for participants. However, we have performed successful card sorts with over 200 cards where participants understood the content well.

In addition to the labeled cards, be sure to include some blank cards in case participants need to add something. And don’t forget a pen.

Execution

For the purpose of this article, we will describe an ideal execution for a card sorting exercise. Keep in mind that there are several variations, as described above.

The cards have been labeled using Avery labels on 3″ x 5″ index cards. On the back of each card is a letter/number combination, as well as a short description or image as necessary. The letter/number combination will be used during analysis; the short description or image is provided to clarify titles that might prove confusing. The cards are shuffled prior to participants entering the room. The shuffled cards, a stack of 20 blank cards, and an ink pen are placed on the table. Three participants are brought into the room and given an introduction with some basic instructions, like these:

 

First of all, we’d like to thank you for coming. As you may be aware, we’re in the initial stages of (re)designing a (web site, product, intranet). In order to make it as easy to use as possible, we’d like to get some input from the people who will be using it. And that’s where you come in. We’re going to ask you to perform a very simple exercise that will give us some great insight into how we can make this (web site, product, intranet) easier to use.

Here’s how it works. In front of you is a stack of cards. Those cards represent the content and functionality for this (web site, product, intranet). Working together, you should try and sort the cards into groups that make sense to you. Don’t worry about trying to design the navigation; we’ll take care of that. Also, don’t be concerned with trying to organize the information as it is currently organized on your (web site, product, intranet). We’re more interested in seeing how you would organize it into groups you would expect to find things in.

Once your groups are established, we’d like to have you give each group a name that makes sense to you. You are allowed to make sub-groups if you feel that’s appropriate. If you feel something is missing, you can use a blank index card to add it. Additionally, if a label is unclear, feel free to write a better label on the card. Finally, if you think something doesn’t belong, you can make an “outlier” pile.

Oh, and one last thing. Feel free to ask questions during the exercise if you feel the need. I can’t guarantee that I can answer them during the exercise, but I’ll do my best to answer them when you’re finished.

Facilitating card sorts can be tricky. During the exercise, your main job is to observe and listen. Your secondary job is to keep the momentum going without leading the participants. Take notes on a small notepad to keep track of insightful comments made by participants, or questions that come up during the session.

Try to make sure each participant has the opportunity to provide input. If one of the participants tries to “take over” the sort, gently prompt the other participants. If one participant sits back, gently prompt that participant. If the group creates a “miscellaneous” group, ask them if they are satisfied with that group, or if they would like to take another look at it to see if it needs to be sorted further. Make sure not to lead them too much.

Once the sort is complete, you may see something that looks like this:

Once the participants are finished, walk them through a particular task. This helps validate the results. For example, if the site has some type of account management, or profile feature, ask them to walk you through updating their address information.

Analyzing the results/next steps

Analyzing card sort data is part science, part magic. Analysis can be done in two ways: by looking for broad patterns in the data or by using cluster analysis software.

When performing analysis on smaller numbers of cards, you may be able to see patterns by simply laying the groups out on a table, or taping them on a whiteboard. You will be able to see patterns through similar groupings and labeling.

When performing analysis on larger numbers of cards, we suggest using a spreadsheet. Enter the results into a spreadsheet, making sure to capture the title and number on each card. If the participants changed the label on a card, record the new label and place the old label in parentheses. Once you’ve entered the data, begin looking for patterns across the groups. Keep in mind the discussions held between the group participants during the sort, as they provide additional insight that might not appear in the spreadsheet. At this point, you are not looking for a definitive answer, but for insights and ideas.

Another technique for analyzing data can be found in “Analyzing Card Sort Results with a Spreadsheet Template“; by Joe Lamantia.. Follow the instructions in Lamantia’s article to prepare the spreadsheet. As he mentions, look at the results for high-agreement cards and low agreement cards.

In both types of analysis, patterns will emerge. These patterns will likely be sensible for the actual users. It is important to note that areas of difference also provide useful insights. Areas of difference tell us about:

• content that participants haven’t understood well
• content that could belong to more than one area
• alternative paths to content (for example, a list of all “how-to” articles could be created)
• how different types of participants see information

There definitely is some magic in the analysis step, and it is difficult to provide exact instructions on what to look for. Allow yourself some time to explore more than one organizational model based on the information provided from your analysis. Remember that it is not necessary to jump straight to a taxonomy at this point. Your card sort results can be supplemented with additional user research and task analysis.

Issues/Variations.

There are a range of additional tasks that you can ask participants to do during the exercise, including these:

• Home page content: ask participants to put to one side content that they would use so often that they would want a link on the home page to it.
• Information- seeking task: after the exercise, bundle up the piles of cards on the table so only the top level is showing. Ask participants where they put particular content. (It is worth doing this if you suspect that the participants were not thinking about how they would use the content as they sorted)

The resulting draft information architecture can be evaluated using Donna’s card-based classification evaluation. This technique provides additional information about the grouping of the content, as it focuses on tasks that users would do rather than just focusing on content. Frequently, participants will create groupings of content in a card sort that they then cannot use when asked to perform a scenario.

Summary

In summary, card sorting is a simple, reliable, and inexpensive method for gathering user input for an overall structure. It is most effective in the early stages of a (re)design. And while it’s not intended to be a silver bullet, when done correctly, it is instrumental in capturing helpful information to answer questions during the information design phase – ultimately making the product easier to use.

Invitation

One reason we wanted to write this article was to get a detailed explanation of card sorting in one place. Please expand this article into a definitive card sorting resource by adding comments with your own variations or observations.

 

Donna Maurer works as a usability specialist and information architect for Step Two Designs, an Australian consultancy focusing on intranets, content management, usability and information architecture. She is currently researching, designing and testing information systems for Australian government and public sector clients, and is presenting usability evaluation workshops.

In her spare time Donna tutors Information Systems Design at the University of Canberra, studies for a Masters in Human Factors, and maintains a weblog, imaginatively called DonnaM, about IA, usability, and interaction design.

Todd Warfel is a Principal User Experience Architect at MessageFirst in upstate NY. With over 10 years of experience practicing user research, information architecture, interaction/interface design, and usability his work has produced several industry firsts and patented products. His work has included projects for Fortune 500 firms, government agencies, and educational institutions, including Adobe, Albertsons, Apple, AT&T Wireless, Bank of America, Charles Schwab, Cornell University, Dell, EDS, Macromedia, Palm, and Philips Electronics. In 1996, Todd developed DIVE©, a proprietary process for improving products’ ease-of-use. The DIVE process has been used across more than a hundred products, many of which are industry firsts.

Todd is currently working on a PhD in Information Science at Cornell University and has a B.A in English and Cognitive Psychology from Ball State University.

Additional Resources

• The IAwiki has a page on card sorting that is updated frequently.
  http://www.iawiki.net/CardSorting
• Gaffney, Gerry. “What is Card Sorting?” Information & Design (2000).
• Maurer, Donna. “Card-Based Classification Evaluation,” Boxes and Arrows, April 2003.
  http://www.boxesandarrows.com/view/card_based_classification_evaluation.
• Rosenfeld, L. and Morville, P. Information Architecture for the World Wide Web: Designing Large Scale Web Sites. O’Reilly & Associates, 2002.
• Warfel, Todd. “Modeling Organization – Methods for Increasing a System’s Findability,” Message First Corp., 2001.
  http://messagefirst.com/downloads/ModelingOrganization.pdf

 

SaveSave