Investing in Usability: Testing versus Training

“Usability professionals offer so much more than just testing. Usability dollars can be spent in other ways; in fact, I argue that usability training is often a far better investment than usability testing.”Assume that you are in charge of a development project and you have about $10,000 to spend on usability. What would you do? What is the best way to use the money? What will make the project a success? What is the right thing to do for the organization? What will be best for customers?

This line of questioning is important because it makes you think about how money should be invested in usability. It gives you a chance to think about what you really value. It forces you to think about usability as a process and a set of tools, as something that must be balanced against other business needs. Unfortunately, most people are too worried about getting money for usability in the first place, but not worried enough about how to spend that money once they get it.

In my experience, usability professionals use their budgets to run usability studies. That is, when given money, they immediately start setting up usability programs to solve particular problems. This shouldn’t surprise anyone because many usability professionals think the value of usability is derived entirely from the results produced through usability tests.

Most people think usability is synonymous with usability testing. It isn’t, and this misconception frustrates me.

Usability is rooted in research and testing
Usability can be considered both an attribute and a process. Usability is an attribute when people characterize something as having “good usability.” For example, if a cell phone menu is easy to use, we might say it has good usability. But usability is also a process. There are many ways to increase a product’s usability; the usability process can be supported by a wide variety of tools and methods.

Usability professionals like to focus on research. This is not surprising, since usability has strong academic roots in basic and applied psychological research. Because usability professionals are often minted as researchers, they feel that getting results means that they must do testing. This approach is blatantly wrong. There are many ways to get incredibly useful business results from usability practices without performing usability tests.

Further, the lack of business focus in the usability community is appalling. In many organizations, usability is a poorly defined business concept. Even though usability offers incredible returns, it is hard to sell. It is hard to sell, I think, because it is sold as usability testing. Most business managers have a difficult time understanding how research (i.e., usability testing) generates revenues or reduces costs. The benefits of usability as a whole are lost because the usability specialists are so narrowly focused on usability testing to the exclusion of everything else. Business managers and project managers are confused; I cannot blame them and, in fact, I am on their side.

This focus on testing is a failure in the usability community that needs to be corrected. Usability professionals offer so much more than just testing. Usability dollars can be spent in other ways; in fact, I argue that usability training is often a far better investment than usability testing. It is often a lot easier to sell training than to sell research. The value to an organization is much easier to grok.

Designers and developers versus usability specialists
It might shock people to hear this from me, but when I am acting as a project manager, I’d rather have a great designer on my side than a usability specialist. Given limited time and money, I need someone who can get the job done right. Designers and developers produce tangible results, and the great ones produce incredible work. For most projects, I don’t need a usability professional if I have a great designer. Furthermore, most usability professionals can’t design or develop their way out of a wet paper bag. They are often limited because they can only do research and testing, which doesn’t mean anything until it is applied to a design. So, usability specialists are usually limited in two ways: they want to test everything and most of them can’t design worth a damn.

My guess is that, at this point, designers and developers are smiling and rejoicing. They probably feel vindicated. At the same time, usability specialists are loading their shotguns and getting out their pitchforks. They’re coming to get me. Hold your horses! Now I’m going to aim my guns at designers and developers. They have their faults too. Believe me.

In my experience, there aren’t many great designers and developers. Some are very good, certainly, but most are mediocre at best. Too many get along by using the same old tricks and boring effects. As a project manager, I will trust only the very best designers and developers to do things right. Most of the time, in a project management role, I have to spoon-feed designers and developers so that they can get the job done. I’m forced to go back to most designs again and again because they are poorly implemented. To spin this another way, designers tend to design for themselves, not for users. If I have limited time and money, which is virtually always the case, I don’t have time for below-average designers. The best people get my money and attention, because they produce the results I need.

I’m sure by now I’ve made a few people angry but I’m trying to make a critical point. I know that I am wrong in some ways and that I am generalizing too much, but I’m trying to illustrate how business people think. Usability specialists are seen as snobbish academics in white coats, obsessively focused on research and testing. They also tend to have poor design skills. On the other hand, most designers and developers can’t generate results that are good for users, and rework is often needed.

So, we are at an impasse.

If only we could get more usability knowledge into the minds of designers and developers. If only we could get usability specialists to expand their horizons, beyond usability testing. If only we could help good designers and developers become great. If only we could get everyone to add real value to the bottom line.

Usability training bridges the gap
There is at least one solution: usability training. Instead of spending so much time and energy on usability testing, usability specialists should spend more time training the designers and developers.

The idea is rather simple. Teach designers and developers to better understand usability as both an attribute and a process so that these intelligent folks understand how usability can be added to a product or service. Training can take many forms:: interactive workshops, hands-on exercises, user test observations, live user tests, usability heuristics application, card sorting, listening sessions, and so on. In many of these exercises, designers and developers are watching users use their products. Just imagine: a usability boot camp.

As a result, designers and developers end up with a ton of usability knowledge. More importantly, they can apply simple usability methods to their work going forward. Usability can be built into products by the people building the products. Admittedly, the improvements probably won’t be at the same level as those recommended by highly educated and trained usability professionals, but they will still be quite useful.

A final point: usability training will help designers and developers eliminate many pesky issues that detract from product usability. Similarly, training gives people the ability to see non-critical issues more easily. These issues are more general in nature, and may actually be overlooked by highly focused usability specialists. Designers and developers, given a little training, have the power to see the human gestalt of what they are building. They start to see how their products drive emotions, including satisfaction, appreciation, and happiness. In summary, training brings forth the human side of design to designers and developers who are often unaware of the issues that people regularly face.

Let’s get the usability community to transfer knowledge. Of course not all knowledge can be transferred, but we can make usability professionals more productive by improving the skills of designers and developers.

Maybe you don’t agree. That’s fine. But below are some reasons why usability training is a better investment than usability testing.

1. Usability testing is a one-time investment.
   A research program is designed, testing is done, results are analyzed, and recommendations are provided. The research doesn’t generate much value beyond the specific recommendations for that specific research program. In plain talk, the results don’t generalize. However, if you train people, they can use the knowledge on project after project after project. To use an old metaphor, usability testing is about giving designers fish, whereas usability training is about showing designers how to fish.
2. Usability testing is often done at the end of a project when it is too late.
   Of course, this isn’t how it should be done, but that is how it often works out. However, by necessity usability training is provided before a project. Indeed, even if it comes after a project, training is useful on nearly every subsequent project. Designers and developers can apply tools and methods to their own work, without the intervention of a usability specialist. They can do quick-and-dirty testing, apply heuristics, and so much more, if they have had training.
3. Usability testing is more complex than usability training.
   Because it is generally more complex and more focused, it consumes more time, and it is also more expensive.
4. Usability testing is often too focused and too isolated.
   It is nearly impossible to run a usability test and get results that apply to other projects. The results are targeted and do not generalize. However, usability training is nearly always general. It is about helping people think about customers. It is about designing with the customers at the center of the project, not the technology.
5. Usability specialists who are focused on research and testing may have a hard time explaining their complex results to designers and developers.
   In my experience, knowledge transfer issues are far less prevalent with usability training. Indeed, usability training is focused primarily on the issue of knowledge transfer. It is about applying a process, whereas usability testing produces isolated results, which don’t necessarily translate to action items for other projects.
6. Usability testing is nearly always more expensive than usability training.
   As a long-term investment, usability testing generally produces one-time results, so returns are limited to one project. In contrast, usability testing is about generating long-term returns.
7. Usability testing often leaves developers out of the loop until it is too late.
   Developers often have key insights that are left out of usability research projects. For example, a usability test might generate great recommendations that are not technically feasible. Usability training eliminates this disconnect.
8. If you teach developers to apply even simple usability techniques, then usability specialists can focus on the hard questions.
   This is a win-win proposition because it gets the developers and designers thinking about usability and users, but it frees up the professionals to tackle the harder problems, which they tend to prefer.
9. Usability has the greatest impact when it is part of the culture.
   Usability testing doesn’t facilitate cultural change. However, usability training is all about changing the culture. If it is done well, training cooks usability concepts into an organization.

Overall, training is often a better investment than testing because it takes less time and energy, produces long-term benefits, and transfers essential knowledge to designers and developers. It helps usability specialists, designers, and developers be more productive. Training provides huge value in getting more people to focus on customers, which is ultimately what is needed to improve project quality and the bottom line.

How to choose between usability testing and usability training
I’ll conclude with a refinement of the points made above. It can be hard to choose between spending money on usability testing and usability training. You should do training if you want to eliminate the more basic usability issues, and if you want to “bake” usability into the culture of an organization. Even if training is applied, there are still many, many usability problems to solve (the most difficult ones!), so you certainly still need specialists. In a sense, usability training moves basic usability testing into the hands of designers and developers, whereas the most difficult and perplexing usability issues will continue to be solved by usability specialists.

Here are some heuristics to help you make the right choice.

• Budgets
  When should you test and when should you train? If budgets are limited, it usually makes the most sense to train designers and developers. Even two or three days of usability training (a small investment) can make a big difference. Usability testing is expensive-research isn’t cheap if you want to do it right. Of course, you need to pick a usability specialist who can do more than testing; training presents its own challenges.
• Scope of issues
  It often makes sense to train when there are broad issues to tackle. If you want to focus on a specific usability problem then you will probably want to do usability testing, but if you simply want to improve the skills of your designers and developers, then usability training is the right choice for you. Usability training is great for addressing non-critical design issues, whereas usability testing is perfect for answering specific and critical design questions.
• Culture shift
  Usability testing is the perfect tool to change the culture of a company. If there is a willingness to listen to customers and learn from them, then training is the perfect vehicle. Listening to customers is a skill that can be taught to designers and developers. Once usability is delivered via training, and the developers and designers buy into it, the culture of the company will slowly change. Usability training is ideal for your top developers, those who are willing to move to the next level of design. Note that executives and managers can benefit from usability training too. You’ll get a much more rapid culture change by involving top management. You can easily take a top-down or bottom-up approach to training, or you can do both at once. In either case, you can more easily change the culture of company through usability training than usability testing.

In short, if you have a big budget, focused questions, and your designers and developers already understand usability, then, by all means, attack problems with usability testing. However, if you want to maximize your investment in usability, if you want to bridge the gap between designers and usability specialists, or if you want to create a customer-focused culture, then I strongly recommend usability training.

John S. Rhodes cannot fly. However, he is the founder and principal of Oristus, a consulting organization that is focused on solving the problems that people experience with technology. He also runs WebWord, which is one of the oldest and best known blogs on usability. In his spare time, John runs marathons, juggles flaming objects, and travels around the globe. Throw him an email at john (at) webword.com, if you want.