What is the biggest problem I face almost every time a client hires me to do something about a web project going awry? They don’t know a thing about their users. They don’t have a clue, whatsoever. Unbelievable but true!
Good designers will certainly argue that THEY don’t need user data to do proper design. That if THEY like it, EVERYBODY will… sure! This probably explains why so many web projects fail in so many levels: Usability, aesthetics, emotions, and profitability.
What’s the remedy to this world-wide infection? User research… but not in the typical version, meaning lengthy ethnographic studies that seem to take forever before obtaining some data. I’m talking about a simpler way, a faster way of doing it. I call it "extreme user research." What’s so extreme about it? Well, it can be done in 30 minutes per interviewee, and it generates loads of useful data that will have a real impact on design, thus making your website more profitable.
Getting information from surrogate users
Doing user research doesn’t have to be tedious and cost lots of money. In many cases, you should be able to do it in a few days, even a few hours, depending of the scope your project. The main idea behind extreme user research is that instead of going for the real users, we go for surrogate users. Those are the ones within a company who talk directly to the customers. We want to talk to the people who talk to the people.
For instance, let’s say we do an e-commerce project for a cable company. The surrogate users would be those in the call centers—the first-line personnel who provide information about products and the second-line personnel who provide customer support for billing issues and technical problems. Talking to those people means having access to tens, hundreds, or even thousands of clients. Not bad at all!
Doing extreme user research is simple. We simply perform individual semi-structured interviews that last no longer than 30 minutes. This time limit is a profitable constraint. It adds a stress that forces the interviewee to focus on the core, on the essentials. During those 30 minutes, we want to know as much as possible about the customers:
- What triggered the call? For example, was it a problem, advertisement, word of mouth, season, news in the media, life event like a birth, the first job, moving to another place?
- What is the whole purpose of the call?
- What are the callers’ main concerns? Are there any misconceptions or incomprehensions about the company’s products or services?
- What words do the callers use to express their needs?
Do what people do during speed-dating sessions: Focus on the essence. Ask for the top ten questions from customers. Ask for the five things you should know if you’d have to replace a surrogate user for an afternoon. You’ll see, it works!
Go for the individuals. Don’t, I repeat don’t go for group interviews, the infamous focus groups. Otherwise, you’ll have to deal with strong-minded individuals whose influence biases the group and thus the whole process.
How many surrogate users should you interview? About five per job description. You want a certain degree of repetition among the interviewees to avoid anecdotes or personal perceptions. Because of the speed factor, you can interview up to 12-14 people in a single day, which means more than 60 interviews in a week! Yes, these will be long days. Yes, at a certain point, it will be tedious to hear the same thing over and over again. But that’s the whole point. We want to make sure we have solid data based on facts, not perceptions.
Okay. Now, you have done your 40 interviews. You’re swamped with data. What’s next? Well, all you have to do is:
- Extract all the facts that you’ve found.
- Write them on sticky notes.
- Tag each note appropriately using a word or a symbol. I usually use words like user, goal, trigger, concern, FAQ, love factor, hate factor, and incomprehension. These tags will really help you later on for documentation. You can also use different colored sticky notes for this purpose.
- Find patterns and create groups around types of users.
- Create some first-version personas and then refine them.
- Show and tell everyone about your findings.
Designing using facts, not opinions
During the Québec city website redesign (which is not yet online), we interviewed five call-center employees and discovered that citizens interact mostly with city hall for:
- their home (garbage collection and recycling, permits, taxes)
- their street (parking, lighting, pavement and road works, snow removal)
- public services schedules (library, swimming pools, skating rinks, etc.)
Knowing these interactions really helped us focus on what citizens really need. Garbage collection by definition is not very sexy, but when 30% of the calls are about this topic (based on interviews and call log analysis), it becomes clear that the city website has to address this subject before anything else!
Call-center personnel also told us that citizens always ask the same four questions about a topic:
- How to get a service from the city? They want to know the procedure (for example, how to get rid of an old sofa).
- When is it going be done? What is the schedule?
- How much does it cost?
- Who’s in charge?
Having this information helped us design page templates with placeholders answering those four questions. It’s that simple and straightforward.
Conclusion
Knowing, I mean really knowing your users has great benefits. Your design will be based on facts, not on suppositions or false perceptions.
Knowing your users means that you’ll spend money on what users really need, NOT on what you suppose they would need or like. It usually leads to simpler solutions. Having facts reduces those never-ending discussions where everybody has his own solution based on his own personal needs and preferences. It has been said before but I’ll say it again: We, the designer and the client, are NOT the users.
Get out of your cubicle. Get out of your meeting room. Go and get those surrogate users and know as much as you can about your users. You’ll see: Your users are not who you think they are.
One comment you make is “We want to make sure we have solid data based on facts, not perceptions.” Agreed. In fact, it’s something of a truism perhaps – but I do wonder whether this method is susceptible to this very problem. By talking to the people who talk to users, we introduce another layer of bias (political, cognitive, whatever) and thus could end up a bit further from facts than if we’d gathered them ourselves.
Don’t get me wrong, your approach can be useful in some circumstances, and I’ve used it myself in the past when I’ve had zero (and I do mean zero) budget to do more thorough research. But, given the choice, face-to-face testing / ethnographic research has to be preferable.
As a postscript, I’d say that anyone who argues that “if I like it, everybody will…” is neither a good designer, nor indeed a designer at all.
Daniel, it is a nice read. Use of surrogate sources to gather information is a good idea, however, it takes the discussion back to the relevance of the data collected. What we are gathering here is second hand information and not the first hand had we talked to the users. But sometimes we designers are left with no choice but to use such methods.
Despite of the low cost and less time needed to do extreme user research, we shall always discourage our clients to go for it. Extreme user research should be practiced with caution and care. Nothing could replace the joy of talking to the users and designing for them.
Good article, and it has the perfect length!
@ Praveen: Yes, talking to the users is always a good idea. But: in my view this is also second hand information. Because you are asking people what they think they would do on your site. You never know what they would really do. So the only first hand information we can get our hands on are usability tests and log files.
@Jens: I think I should have been more explicit while writing. Anyway, when I said that my intent was to talk to the users about their tasks, needs, aspirations, desires, and motivations. I did not mean to ask them about their interaction with the site. We must keep in mind the famous quote, “Users are not Designers and Designers are not Users.”
Interesting article, though I agree with Cennydd that surrogate users can be problematic. As an example, my current project involves a lot of product development where most of the team has frequent access to users, and some of the team would even qualify as end users themselves. That said, the feedback we get from our internal surrogate users is often clouded by their knowledge of business priorities, and whatever ethnographic research we’ve managed to do externally has often revealed dramatically different things.
This is a good approach where no possibility for ethnographic research exists, but it would be interesting to look at a more guerilla approach to observation studies and other first-hand research.
Hi, Daniel–nice article. I especially liked your Québec city example as it is a practical reflection of unmet user needs in the architecture. I do think, however, that it is not that difficult to derive great customer insights “on the street”. Even if you walk out of your office or blast an email out to friends and family you can find at least one or two users.
We sometimes are forced by shortages of budget or time (often both) to perform “napkin ethnography”, which is admittedly not as reliable as a fully designed research protocol, but often enough to get a decent direction for what features users would find valuable.
For example, we were looking at a large-scale integration project for a soft drink brand–they wanted to build a site that would make it easy to replenish fountain syrup. We had TONS of features we could have placed on a very crowded homepage, but when we talked to small retailers we found that most ordering/reordering interactions happened over the phone and in-person with the driver (who had a hand-held device). The user was SO busy, they rarely sat down at a computer. That changed how we proposed to design the interface considerably: focusing on automating replenishment phone calls, resolving service issues, and being a lot more selective with new product merchandising. We placed all of the required advanced features in the architecture, but in secondary nav areas that didn’t draw as much visual focus.
All of this is to say that it is always a good idea to talk to users…and it is often easier and faster than we think. Call center employees are definitely great stakeholders to start with.
…and what I neglected to mention above was that these small retailer users were within a 3-block radius of our office. We performed 5 interviews in just a few hours. That sample size was even enough to begin to rough in a persona.
I think we can all agree (author included) that dealing directly with the users is better, but I see situations (no budget/time + hard to access users ) where this technique can be valuable.
Cennydd makes an important point in saying: “By talking to the people who talk to users, we introduce another layer of bias (political, cognitive, whatever) and thus could end up a bit further from facts than if we’d gathered them ourselves.”
I think the key to using this method, or others like it, is to always remember that their is that extra layer of bias, and attempt to take it in to account when working with the information. Designers should think about how the information gathered might be biased and draw design insights accordingly. It is when designers pretend that the information they gain from these once-removed research methods is ‘pure’ that they run in to problems and get blindsided.
Firstly, the length of this article is great – short and snappy. Much appreciated!
There’s a sliding scale of user involvement, in my opinion. At one end we have an idealised process with unlimited time and resources, in which we conduct ethnographic research across truly representative samples of users or potential users, and continue this engagement throughout product development. At the other end is grabbing someone in a corridor and showing them a screen. This method definitely falls towards the latter end on the spectrum, but utilising information within the organisation is an important area to highlight.
Most organisations that have spent some time developing products for a specific domain do actually contain a lot of good domain information. Using all the information from people who contact customers is a really valuable way to increase the degree of proxy-user input, possibly enabling you to focus your limited time and resources on the areas where real users truly are key.
Thank you for speaking out on this! When it comes to this type of user research, I sometimes feel like a lone voice crying in the wilderness. Over the last 5 years, I’ve been trying to convince clients that quality user data IS within their reach thought this type of research. I still find that I run into barriers of time cost and scope. What have you found to be effective in overcoming these types of concerns?
While I think that user research by proxy isn’t a bad idea per se, it seems to me that the article isn’t quite egalitarian enough. When designing a web site, it would seem a bit odd not to account for the obvious possibility that people who call the call center are different from people who prefer to use web. The approach would seem to demand some corresponding research of other channels before making hard design decisions – an online survey, perhaps, seeing as we’re not too bothered by bias.
Oh, and may I add my thanks for writing a mercifully short article. Let’s hope it starts a trend.
Great article Daniel! I think it’s important to be aware of interesting alternatives to the ideal methods of gathering user data. Even though there is some danger involved in relying on this second hand information, the reality is that in most companies, there is still a lot of resistance to budgeting any significant amount of resources towards real user testing.
For myself, this is something I am already practicing to some degree and I find it to be a valuable tool in a pinch.
Thanks again for nice, short, insightful article.
To add to my point, I believe this is simply one tool in our tool belt. I don’t think any single source of data is sufficient to make any remotely important design decisions.
Wow! Thank’s to all of you for your great comments =).
@Cennydd: I do agree with you that there is a risk of bias. As a matter of fact, there is a bias, but, by interviewing more than one surrogate user per job profile, we reduce it is some way. Of course, when it’s possible, I do prefer interviewing + observing real users in their environment (work/home). However, in the projects I was involved in the past years, doing a “real” ethnographic study would have been impossible for different reasons (cost, time and users access).
@Jonathan: you said “When designing a web site, it would seem a bit odd not to account for the obvious possibility that people who call the call center are different from people who prefer to use web.” Absolutely. However, it gives us a good indication of what is mostly needed/asked by users. Questions asked over the phone or email should be aswered on the Website (well, most of the time ;-).
@Ken: you said “I believe this is simply one tool in our tool belt. I don’t think any single source of data is sufficient to make any remotely important design decisions.” Absolutely. I usually combine this approach with web analytics (when there is an existing website), call log analysis, etc.
@Matthew: you said “I still find that I run into barriers of time cost and scope. What have you found to be effective in overcoming these types of concerns?” Well, doing this kind of user research takes usually no more than 20 days (this includes interviews, analysis + report), which is not expensive. The cost of NOT doing user research greatly exceeds the cost of doing it. User research pays by itself just by knowing on what we must focus (what users want/needs). Also, having “hard” facts, as opposed to suppositions, reduces those never-ending discussions during design sessions.
Sorry to bother but a website that discusses about usability and knows that reading on a screen is painful for many of us, i’m sad to see no one thought that people like me and you would try to get a print version to enjoy easily reading the very relevant articles suggested here!!!
thx for your help
Hi Osmane,
Your point is well-taken. We have many things in the development queue, but I understand the want to be able to easily print the stories. I’ll see if we can move this one up. Thanks for being honest about your frustration.
Updated 3/29: One of my colleagues reminded me why this is somewhat farther down the list than seems reasonable.
Which browser and operating system are you using?
You should be able to print either directly or via “Save to PDF” in the . I’ve tested this on Firefox and Safari on the Mac, and it works perfectly. On Windows, I’ll need to verify, but we’re using a Print style sheet, so it should work.
Best,
Chris
Daniel,
In my recent article on UXMatters I argued the merits of using customer interaction channels as a source of design research. Touch-points such as call centres, customer enquiry forms, feedback forms &etc provide a rich source of insight into unmet customer needs or pain points.
Your points about the relative cost & efficiency of these sources of user insight are well made.
Cheers
Steve
Hi,
Nice artilce. I found the steps that should be followed after gathering the data really useful. Also, I agree with the concept that you can gather the needed information by only asking a few and critical questions and not troubling the user with a big list of questions. However, I believe that in some projects either because of time or budget limitations there is no user task analysis step before starting the design process. I am against that, but experience has proved that this is the common phenomenon. Nevertheless, “extreme user research” could be a solution to these cases so thanks again for the article.
Marianna
It’s hard to disagree that dealing with the users personally and getting their opinion on various issues is a valuable tool. But it is also worth noticing that what we are realle getting is an opinion, not a fact, therefore possible outcomes can differ considerable from what we expect to achieve. We should bear in minds that users are not oracles, and quite often their declarations are different than what they really do or want. So a bit of more statistical and scientifical approach is advised. Obviously the best results can be achieved by combining the two methods together.
Daniel, thanks for the article. I’ve found that this approach (combined with other approaches as noted by previous commenters) works really well. I’ve also found that working with surrogates can sometimes help engage people throughout your company, turning them into allies that you can approach in the future for assistance.
One additional thing to keep in mind when using surrogates: it’s helpful to match surrogates to the users they serve. That way you get a usable amount of information on each user group. For example, in your Quebec city example you interviewed call center reps to get info on consumers but you probably also spoke to people in the permits area to get data on the needs of building contractors
Daniel, to the point article. Interviewing surrogate users is no doubt a very effective way to gather users needs and has added advantages (as pointed out by everyone) like rich user data, time saving, cost saving and the most important of all helps get the buy-in from other groups of the company by keeping them involved. This technique is a nice compliment (not a replacement in any ways!) to the other methods of user research (e.g. Ethnography, Contextual enquiry) and no practitioner should limit its use.
Just a thought on the title ‘Extreme user research’ was a bit confusing 🙂 as I was expecting a rather direct method of collecting user data than the one explained here.
Thanks Daniel for putting this out.
In fact getting the surrogate users into picture is really a nice bargain over actual users. If not all, atleast in many cases this approach is well capable of getting actual users pain and gaining usability insights.
At the same time, agree to what few points says here that its a nice compliment and useful when one has constraints over approching actual users, but should not be treated as a replacement technique over existing ones.
Daniel thanks for the article. I have found that with some of my customers the fear me actually getting to the direct users of the system. This approach will surely assist me in capturing the data I need to produce a better product for them. The interesting thing I continue to hear from customers is that their customer base is too large to get data from all of them. I continuously have to explain to them that what we are doing is research and would only require a sampling of the users. At times I have found that not even letting them know it is doable helps the cause because in the end they are afraid of what the users have to say. Again thanks for the article.
Daniel, Really nice article. it actually works fine for all unilateral appeal based websites/single purpose web application. In the past i ve got both good and bad experiences based on this way of data capture.
I accept as everyone agree on the size of the article, specifically when talking about the MOST CRUCIAL PART OF ANY PROJECT. the project shines or fails based on the effort and outcome of this step. But website or the web application project where the intention is not just corporate exposure or business focus but if its more than that. in those kind of sites or application the surrogate users actually disrupts the linearity thread of either the concept or the basement of idea. Because its not a simple single line information that forms the base of the project design. – just my views
But having them in the loop can definitely help us as the guiding principle for taking some key points. Probably easy to say – depends on project by project basis. But when considered only option or the least of an option in data collection i would definitely salute this idea of getting them in the loop…
Really good article to address very important stage of the Project Initiation. Thanks for the article.
Sathish Sampath
great reading Daniel
i wanna share my strategy , use when, i get project. one thing about clients is true they don’t have any idea about target (potential) customers. i usually , brainstrom , with my client to agree him on his/her target audiance. and after this , we enjoy speed dating with 1 to 3 persons of every job and ask them pinching questions.
your boss hired me to develop for you,
1. what is the problem you are facing with current working system(paper system of else)
2. what are your individual opinion about your job ?
3. what are your interest with this project as a stakeholder.(what benefit you can give to your company using automated software(web based system)
4. what are business needs accoridng to your job at your level. (e.g. an accountant may be asked to tell what kind of transactions he/she is handling, what does he not do with tradition sytems(current system))
5. most important i asked all about interface. do they like menu driven system , tab driven, or else. is there someone who is color blind.
6. check marketing documents to select suitable colors.
this list non-ending
Regards.
Hi Daniel. Nice article, it is funny (and perhaps not as remarkable as I thought it was) to read that the citizens of quebec like to know when their garbage is collected and where to park their cars. It’s exactly the same outcome which we got for a Dutch city last year.
The only difference is that we spoke real users instead of surrogate users. In this case your solution worked out well. But in many cases I think surrogate users may have other interests besides the clear truth. Extreme example: Not in every case a call center employee would want a user to find parking information on the website. Where would her job go if 90% of the calls is about that information?
Don’t get me wrong if you think I disapprove this kind of user research, gathering any kind of user information is a good thing. I just think it’s user research, extreme user research would be talking to the real user then 🙂
Great article, thanks. I’m going to share with some past colleagues and a client – with context. As a consultant who is often called upon to perform project rescues, I feel that I should point out that I have, indeed, spent considerable time (and watched clients spend a lot of money) fixing projects that have resulted only from surrogate data. These instances have resulted from logic that is similar to the primary driver logic referred to herein (typically, executives who refused to honestly answer the question of what NOT doing proper UX integration would cost in the long run). I have been in the business long enough to be able to empathize and sympathize with the rationale cited for resorting to the Extreme method. I’m NOT saying its destined for failure. But I want to add my voice to the cautionary ones on this thread.
My personal recommendation might be disputed by a statistician, but I would argue for forced dilution across multiple axes vs. taking a purely “Extreme” route. In other words, if you can only interview 15 users and 45 surrogates, the 15 end-user samples could result in exponentially more unique insight than otherwise. Also, there is a third axis I’d like to add here: 1) “Real” end user; 2) Surrogate (as cited); 3) (NEW!) Victim Surrogate. What’s a Victim Surrogate? A third-string actor affected by the poorly designed system but who is more obscured in the workflow.
This invites a deeper conversation than a comment board has room for, but it points to the need for examining the site from the perspective of public-facing front end; back end (admin interfaces frequented by say, call center personnel); and downstream tangential actors (anyone ranging from high level executives who only care about analytics to warehouse personnel who process printed or electronic order slips; this is always case-specific).
I’m all for radical solutions to radical/extreme problems and think this Extreme approach is well-developed. But for those of us who are teachers, consultants, etc. let’s make sure our clients/bosses/students don’t take this lesson home karaoke style for a quick fix out of laziness vs. Extreme need/externally driven desperation. Thanks.
I’ve found these brief, snappy types of interview great for doing a first pass over a user group. But also insufficient to really get to know users, which does take time. So though I think this is a good approach to have in the toolbag, it doesn’t replace longitudinal studies where you get to follow real users for a while.
Hmmm… I’m getting a feeling that folks want shorter articles. So we don’t muddy the water here, please share your opinion in the forum http://boxesandarrows.com/topics/view/18454
i like the idea of being flexible in doing experience consulting within often timely narrow schedules. it seems to me a liitle bit like the usability engnieering concept from nokia: just-in-time usability. For me that means not to focus on a tradiotonal user centred design process but to pinpoint the user experience methods and actions with the greateast impact within a timeframe and business context. time pressure sometimes makes creative. if threre is not time for doing classical usability tests in the development phase, it`s better to codesign the first product features with lead/power users in a 2day workshop.
thanx
Interesting article, than you! We’ve had great success with similar techniques, it’s great to see that procedural patterns are beginning to develop around these activities. It becomes easier to convince clients that stuff like this is a good idea when more and more IAs are following a similar path to user need/behavior discovery.
As discussed, the benefit of doing stuff like this is also to create data and knowledge that backs your design decisions. When just user surrogates are interviewed, however, we are missing out on several opportunities. Yes, it’s a lot more work and more money to interview real users and other constituents, but if it’s possible, we should always aim to collect data from them as well. It makes the overall picture more complete and it gives people (especially if you fold in key stakeholders into the interview process) a sense of participation – which leads to buy-in. The extra work (even if you have to eat the time) is worth the extra texture, unearthed dimensions, and overall buy-in.
Interesting article – and it sparked a lot of discussions. I think that Daniel lined out an interesting approach for feature research – what are the most interesting features a site should address? Daniels examples clearly state out that he used this technique to elicit the functionality a new or redesigned site should provide, e.g. what are the most needed features on the cities website? If you ask the citizens you will elicit invididual needs but not get the overall picture – except if you ask a very large group of citizens.
This method of course can not give you any insights if it comes to non-functional or implicit requirements. Asking “power users” for those may even bias the requirements engineering process – they use the system every day and even will be able to use workarounds to circumvent non-usable functions. Also usability – for this you have to ask end users or conduct tests. So use different tools for different purposes.
I was talking with a colleague the other day who did about 30 user interviews and 20 stakeholder interviews and was having trouble collating the results. I think the steps for “Once I interview all these people, what do I do? I feel overwhelmed” are very helpful.
I used to work in tech support before going into usability (actually quite a good career path!). As a suggestion, I think it would be an awesome idea to ask that tech support people keep track of their calls in one week. They don’t need to do anything but have a sheet of paper and for every call write down the main words associated with that call. Then when you ask these surrogates for what are the top problems, they can refer back to this sheet… Just something to jog their memories, otherwise it just becomes one giant problem w/o any specific examples.
From the comments here, I think some people are confused about the purpose of surrogates. We’re not talking to surrogates as members of the target audience, nor are we asking them to pretend they are users. We’re talking to them about the target audience. So using surrogates is not the same as “grabbing someone in the corridor” and asking their opinion (such an approach has it’s place, but you need to be very careful in terms of what you take away from your corridor encounter because that person is unlikely to be representative of your target audience). Likewise, surrogates are not “power users” either.
Surrogates (or proxies or intermediaries) are used to learn more about the target audience, indirectly, but based on their direct experience with the target audience.
I wrote a blog post on this topic: Intermediaries in user research. Happy to hear more feedback.
Totally agree with your suggestion… Very nice post and good information here… Thanks for posting that….
http://star-drama.com/