Five Things They Didn’t Teach Me in School About Being a User Researcher

Written by: Chelsey Glasson

Graduate school taught me the basics of conducting user research, but it taught me little about what it’s like working as a user researcher in the wild. I don’t blame my school for this. There’s little publicly-available career information for user researchers, in large part because companies are still experimenting with how to best make use of our talents.

That said, in the midst of companies experimenting with how to maximize user researchers, there are a few things I’ve learned specific to the role of user researcher that have held true across the diverse companies I’ve worked for. Some of these learnings were a bit of a surprise early on my my career, and I hope in sharing them I’ll save a few from making career mistakes I made in the past for lack of knowing better.

There’s a ton of variation in what user researchers do.

In my career, I’ve encountered user researchers with drastically varying roles and skillsets: many who focus solely on usability, a few who act as hybrid designers and researchers, some that are specialists in ethnography, and yet others who are experts in quantitative research. I’ve also spoken with a few who are hybrid market/user researchers, and I know of one tech company that is training user researchers to own certain product management responsibilities.

If you take a moment to write down all of the titles you’ve encountered for people who do user research work, my guess is that it will be a long one. My list includes user experience researcher, product researcher, design researcher, consumer insights analyst, qualitative researcher, quantitative researcher, usability analyst, ethnographer, data scientist, and customer experience researcher. Sometimes companies choose one title over another for specific reasons, but most of the time they’ll use a title simply because of tradition, politics, or lack of knowing the difference.

At one company I once worked for, my title was user researcher, but I was really a usability analyst, spending 80% of my time conducting rapid iterative testing and evaluation (RITE) studies. When I accepted the job at that company, I assumed–based on my title–that I’d be involved in iterative research and more strategic, exploratory work. I quickly learned that the title was misleading and should have been usability analyst.

What does this all mean for your career?

For starters, it means you should do a ton of experimentation while in school or early on in your career to understand what type of user research you enjoy and excel at most. It also means that it’s incredibly important to ask questions about the job description during an interview to make sure you’re not making faulty assumptions, based on a title, about the work you’d be doing.

Decisions influence data as much as data influences decisions.

I used to think the more data the better applied to most situations, something I’ve recently heard referred to as “metrics fetishism.” I’ve now observed many situations in which people use data as a crutch, end up making mistakes by interpreting “objective” data incorrectly, or become paralyzed by too much data.

The truth is that there are limitations to every type of data, qualitative and quantitative. Even data lauded by some as completely objective–for example, data from website logs or surveys–oftentimes includes a layer of subjectiveness.

At the beginning and end of any research project there are decisions to be made. What method should I use? What questions should I ask and how exactly should they be asked? Which metrics do we want to focus on? What data should we exclude? Is it OK to aggregate some data? What baselines should we compare to? These decisions should themselves be grounded in data and experience as much as possible, but they will almost always involve some subjectivity and intuition.

I’ll never forget one situation in which a team I worked with refused to address obvious issues and explore solutions without first surveying users for feedback (in large part because of politics). In this situation, the issues were so obvious that we should have felt comfortable using our expertise to address them. Because we didn’t trust making decisions without data in this case, we delayed fixing the issues, and our competitors gained a huge advantage. There’s obviously a lot more detail to this story, but you get the point: In this circumstance, I learned that relying on data as a crutch can be harmful.

What does this mean for your career?

Our job as user researchers is not only to deliver insights via data, but also to make sure people understand the limitations of data and when it should and shouldn’t be used. For this reason, a successful user researcher is one who’s comfortable saying “no” when research requests aren’t appropriate, in addition to explaining the limitations of research conducted. This is easier said than done, especially as a new user researcher, but I promise it becomes easier with practice.

You’re not a DVR.

Coming out of school, I thought my job as a user researcher was solely to report the facts: 5 out of 8 users failed this task, 50% gave the experience a score of satisfactory, and the like. I was to remain completely objective at all times and to deliver massive reports with as much supporting evidence as I could find.

I now think it’s old-school for user researchers to not have an opinion informed by research findings. Little is accomplished when a user researcher simply summarizes data; that’s what video recordings and log data are for. Instead, what’s impactful is when researchers help their teams prioritize findings and translate them into actionable terms. This process requires having an opinion, oftentimes filling in holes where data isn’t available or is ambiguous.

One project I supported early in my career involved a large ethnography. Six user researchers conducted over 60 hours of interviews with target users throughout the United States. Once all of the interviews were completed, we composed a report with over 100 PowerPoint slides and hours of video footage, summarizing all that was learned without making any concrete recommendations or prioritizing findings. Ultimately we received feedback that our report was mostly ignored because no one had time to read through it and it wasn’t clear how to respond to it. Not feedback you want to receive as a user researcher!

What does this mean for your career?

The most impactful user researchers I’ve encountered in my career take research insights one step further by connecting the dots between learnings and design and product requirements. You might never be at the same depth of product understanding as your fellow product managers and designers, but it’s important to know enough about their domains to translate your work into actionable terms.

Having an opinion is a scary thought for a lot of user researchers because it’s not always possible to remain 100% objective in bridging the gap between research insights and design and product decisions. But remember that there’s often always limitations and a subjective layer to data, so always remaining 100% objective just isn’t realistic to begin with.

Little is accomplished when data is simply regurgitated; our biggest impact is contributing to the conversation by providing actionable insights and recommendations that helps decision makers question their assumptions and biases.

Relationships aren’t optional, they’re essential.

As a student, my success was often measured by how hard I worked relative to others, resulting in a competitive environment. I continued the competitive behavior I learned in school when I first started working as a user researcher; I put my nose to the grindstone and gave little thought to relationships with my colleagues. What I quickly learned, however, is that taking time to establish coworker relationships is just as important as conducting sound research.

Work shouldn’t be a popularity contest, right? Right–but solid coworker relationships make it easier to include colleagues in the research process, transforming user research into the shared process it should be. And trust me, work is way more fun and meaningful if you enjoy your coworkers!

What does this mean for your career?

Take the time to get to know your coworkers on a personal level, offer unsolicited help, share a laugh, and take interest in the work that your colleagues do. I could share a personal example here, but instead let me refer you to Dale Carnegie’s book How to Win Friends and Influence People. Also check out Tomer Sharon’s book It’s Our Research.

Expect change–and make your own happiness within it.

Change is a constant for UX’ers. I’m on my eighth manager as a user researcher, and in my career I’ve been managed by user researchers, designers, product managers, and even someone with the title of VP of Strategic Planning. I’ve also been through four reorganizations and a layoff.

What does this mean for your career?

Change can be stressful, but when embraced and expected, you’ll find that there are benefits to change. For example, change can provide needed refreshment and new challenges after a period of stagnation. Change can also save you from a difficult project or a bad manager.

I remember a conversation with a UX leader in which he shared he once quit a job because he couldn’t get along with a peer who just didn’t get the user experience process. A few months after he quit, the peer was fired. If only he had stuck around for a while.

The U.S. Navy SEALs have a saying: “Get comfortable being uncomfortable,” which refers to the importance of remaining focused on the objective at hand in the middle of ongoing change. Our objective as user researchers is to conduct research for the purpose of improving products and experiences for people. Everything else is secondary–don’t get distracted.

For more detailed recommendations on how to deal with change as a user research, I highly recommend watching Andrea Lindman’s talk “Adapting to Change: UX Research in an Ever-Changing Business Environment.”

Concluding thoughts

I’ve been happy to see in the past two years that the user experience community has stepped up in making career advice more readily available (we could do even better, though). For user researchers wanting advice beyond what I’ve shared in this article, here are four of my favorite resources:

  • Judd Antin’s talk in which he covers many opportunities and challenges of doing user research: http://vimeo.com/77110204.
  • You in UX, an online career conference for user experience professionals.
  • Tomer Sharon’s book It’s Our Research.
  • A special issue of UXPA’s UX Magazine, with the theme of UX careers.

User Experience Research at Scale

Written by: Nick Cawthon

An important part of any user experience department should be a consistent outreach effort to users both familiar and unfamiliar. Yet, it is hard to both establish and sustain a continued voice amongst the business of our schedules.

Recruiting, screening, and scheduling daily or weekly one-on-one walkthroughs can be daunting for someone in a small department having more than just user research responsibilities, and the investment of time eventually outweighs the returns as both the number of participants and size of the company grow.

This article is targeted at user experience practitioners at small- to mid-size companies who want to incorporate a component of user research into their workflow.

It first outlines a point of advocacy around why it is important to build user research into a company’s ethos from the very start and states why relying upon standard analytics packages are not enough. The article then addresses some of the challenges around being able to automate, scale, document, and share these efforts as your user base (hopefully) increases.

Finally, the article goes on to propose a methodology that allows for an adjustable balance between a department’s user research and product design and highlights the evolution of trends, best practices, and common avoidances found within the user research industry, especially as they relate to SaaS-based products.

Why conduct usability sessions?

User research is imperative to the success and prioritization of any software application–or any product, for that matter. Research should be established as an ongoing cycle, one that is woven into the fabric of the company, and should never drop-off nor be simply ‘tacked on’ as acceptance testing after launch. By establishing a constant stream of non-biased opinions and open lines of communication which are immune to politics and ever-shifting strategies, research keeps design and development efforts grounded in what should already be the application’s first priority–the user.

A primary benefit in working with SasS products is that you’re able to gain feedback in real-time when any feature is changed. You don’t have to worry about obsolete versions, or download packages–web-based software enables you to change directions quickly. Combining an ongoing research effort with popular software development methods such as agile or waterfall allows for immediate response when issues with an application’s usability are found.

Different from analytics

SaaS are unique in that there is not the same type of tracking needed in-product. Metrics such as page views or bounce-rates are largely irrelevant, because the user could be spending their entire session on configuring functions of a single feature on a single page.

For example, for our application here at Loggly, the user views an average of ~2 pages (predominantly login and then search) and spends on average 8x as long on search then any other page. Progression is made within the page-level functions, not among multiple pages within the application’s structure.

Javascript-heavy applications don’t have the same URL and tree structure content-heavy sites are built around but instead make calls to different states of the application from within the same page.

Say your analytics package gives an indication that something is wrong with the setup flow or configuration screen, but you don’t yet have a good concept of at what point in the process the users are getting stuck.

Perhaps a button might be getting click after click because it is confusing and unresponsive, not because its useful. Trying to solve this exclusively with an analytics package will pale in comparison to the feedback you’ll get from a single, candid user who hits the wall. As discussed later in this article, with screensharing, you’re able see the context in which the user is trying to achieve a specific task, defining the ‘why’ in their confusing becomes more apparent than just the ‘what’ are they clicking on.

Determining a testing audience

The first component of defining any research effort should be to define who you want to talk to. Ideally, you’ll be able to have a mix of both new users and veterans that are able to provide a well-rounded feedback loop on both initial impressions of your application as well as historical perspective on evolution and found shortcomings after repeated use, but not all companies have this luxury.

Once in the door

Focus first on the initial steps the user has to take when interacting with your application. It seems obvious, but if these are not fulfilled with maximum efficiency, the user will never progress into more advanced features.

Increasing the effectiveness of the flow through set-up, configuration, and properly defining a measure of activation will pay dividends to all areas of the application. This should be a metric that is tested, measured, and monitored closely, as it functions as a type of internal bounce rate. Ensuring that the top of the stream for the majority of application users is sound will guarantee improved usage further down the road to the deeper, buried interactions.

These advanced features should be also be tracked and measured with the correlation that starts to paint a profile of conversion. Some companies define conversion as free-to-paid; others do so in a more viral sense–conversion being defined as someone who has shared on social media or similar.

As you start itemize these important features, you’ll get a better sense of the usage profile for where you’re trying to point the user to. For example, adding a listing record, or perhaps customizing a page–these might match a profile for someone who is primed for repeat visitation, someone who has created utility and a lasting connection, and ultimately ready to convert.

Avoiding overlap

If there is a focus on recruiting participants who are newly signed-up users, then you’ll likely overlap with outbound sales efforts. Because your company’s sales and marketing funnel tries as hard as possible to convert trial users to paid, or paid to upgrade, the company’s priority will likely be on conversion, not on research.

Further, if a researcher tries to outreach for usability surveys at this point, from the user’s perspective (especially those deemed potential high-value customers) it would mean different prompts for different conversations with different people from various groups within your company, all competing for spots on their calendar. This gives a very hectic and frenetic impression of your company and should be avoided.

In the case of a SaaS product, sometimes the sales team has already made contact with potential customers, and many of these sales discussions involve demonstrations around populated, best-case scenarios (which showcase the full features) of your product.

As a result, you may find the participant has been able to ‘peek behind the curtain’ through watching the sales team provide these demonstrations, giving them an unfair advantage as to how much he / she knows before trying to finally use the product themselves. For the inexperienced user, your goal is to capture the genuine instinct of the uninitiated, not those who have seen the ‘happy path’ and are trying to trace back the steps to get to that fully-populated view.

To make sure you’re not bumping heads with the sales and conversion team, ask if you can take their castoffs–the customers they don’t think will convert. You can pull these from their CRM application and automate personalized emails asking for their time. I’ll outline this method in further detail in the section following, because it pertains to the veteran users as well.

Photo of people in a conference exhibit hall.
Conferences are a great way to survey new and existing users.

As described in a previous post, guerrilla testing at conferences is a great way of fulfilling what gets seen and what parts of the interface or concept get ignored. These participants are great providers of honest, unbiased feedback and haven’t been exposed to the product other than some initial impressions of the concept.

Desiring the messy room

But what about the users that have been using your product for months now, those who have skin in the game, have already put their sweat and dollars behind customization of their experience? Surveying these participants allows us to see where they’ve found both utility and what areas need be expanded upon. Surveying only the uninitiated won’t provide feedback on any nagging functional roadblocks, those which are found only after repeated use. These are the participants that will provide the most useful feedback, sessions where you can observe the environment that they’ve created for themselves, the ‘messy room.’

Making an observational research analogy, a messy room is more telling of the occupants’ personality than an empty one. Given your limitations, how has the participant been forced to find workarounds? Despite these workarounds, they’ve continued to use the product, in despite of how we’ve expected them to use it–and these two can be contrastingly very different.

Online feedback form for Loggly UK.
Example of a feedback form, initiated via email.
User is able to schedule a 1:1 screensharing session on the confirmation page.

Automated recruitment

Find your friendly marketing representative/sales engineer at your company (or just roll your own) and discuss with them the best way to integrate a user experience outreach email into the company’s post-funnel strategy. For example, post-funnel would be after their trial periods have long since expired and the user is either comfortable in their freemium state or fully paid up.

As mentioned earlier, you can also harvest leads from the top of the funnel in the discarded CRM leads. However, you’ll likely have a greater percentage of sessions with users that are misfires–those indifferent or only just poking around the app, with not yet a full understanding of what it might do. Thankfully, the opt-in approach for participation filters this out for the most part.

Focusing again on the recruitment of the veteran, experienced users, another, more complex scenario would be to trigger this UX outreach email once a specific set of features have been initiated–giving off the desired signature of an advanced, informed user.

Going from purely legacy-based perspective, six months of paid, active use should be enough time to establish a relationship with a piece of software, whether they love or hate it. If there exists enough insight into the analytics side of the sales process, it would behoove you to also make sure that the user has had a minimum number of logins across these six months (or however long you’ll allow the users to mature).

Outreach emails triggered through the CRM should empower the recipient to make the experience of the product better, both for themselves and their fellow customers. Netflix does a great job of this by continually asking about the streaming quality or any delays around arrival times of their product.

I also recommend asking the users a couple of quantitative and qualitative questions, as this metric something you should be doing for your greater UX efforts already. These questions follow the guidelines of general SUS (System Usability Survey) practices that have been around for decades. Make the questions general enough so that they can be re-used and compared going forward, without fear of needing the change the goalposts when features or company priorities change.

Screen grab of the user's desktop.
A peek into an active user’s work environment.

When engineering this survey, be sure to track which tier of customer is filling out these surveys, because both their experience and expectations could be wildly different. Remember also to capture the user’s email address as a hidden field so you can cross reference against any CRM or analytics packages that are already identifying existing customers.

Setting boundaries

It depends on the complexities of your product, but typically 20-30 minutes is enough time to cover at least the main areas of function. Any longer, and you might encounter people not wanting to fit in an entire hour block into their schedule. If these recorded sessions are kept to just a half-hour, I find that a $25 is sufficient compensation for this duration of time, but your results may certainly vary.

In any type session, do iterate that this is neither a sales, nor a support call. You’re researching on how to make the product better. However, you should be comfortable on how to avoid (or sometimes suggest) workarounds to optimize the participant’s experience, giving them greater value of use.

Tools of the trade

For implementation of the questionnaire, I hacked the HTML / CSS from a Google Form to exist as self-hosted page but still pushing results through the matching form and input IDs to the extensible Google Spreadsheet.

There are a few tutorials that explain how to retain your branding while using Google’s services. I went through the trouble so I can share the URL of either the form or the raw results with anyone, without the need to create an account or login. As we discuss the sharing component of these user research efforts, this will become more important. Although closed systems like SurveyMonkey or Wufoo are easy to get up and running, the extensibility or a raw, hosted result set does not compare.

Insert a prompt at the end of the questionnaire for the user to participate in a compensated user research survey, linking to a scheduling applications such as Calend.ly. This application has been indispensable for opt-in mass scheduling like this. The features of gCal syncing, timezone conversion, daily session capping, email reminders, and custom messaging all are imperative to a public-facing scheduling board. Anyone can grab a 30-minute time slot from your calendar with just your custom URL, embeddable at the end of your questionnaire.

To really scale this user research effort to the point where it can be automated, you cannot spend the time trying to negotiating mutually-available times, converting time zones and following up with confirmations. Calend.ly allows for you to cap the number of participants who can grab blocks of your time, so you can set a maximum number of sessions per day, preventing a complete overload of bookings in your schedule.

As a part of the scheduling flow within Calend.ly, a customizable input field asks the participant for their Skype handle in order to screen share together, and I’d advise for the practitioner to create a separate Skype account for this usability effort. With every session participant, you’ll begin to add and add more seemingly random contacts, any semblance of organization and purity for your personal contact list will be gone.

Screen grab of Calend.ly booking utility.
Calend.ly booking utility – a publicly-accessible reservation system.

Calend.ly booking utility – a publicly-accessible reservation system.

Once the user is on the Skype call, ask for permission to record the call and make sure that you give a disclaimer that their information will be kept private and shared with no one outside the company. You might also add ahead of time that any support questions that come up, you’ll be happy to direct to the proper technicians.

Permissions granted, be sure to re-iterate to the participant the purpose and goal of the call, and provide them with a license to say whatever they want, good or bad–you want to hear it. Your feelings won’t be hurt if they have frustrations or complaints about certain approaches or features of your product.

For recording the call, there are plenty of options out there, but I find that SnagIt is a good tool to capture video, especially given the resolution and dimension of the screen share tends to change based upon the participant’s monitor size. When compressing the output, a slow frame rate of 5/10 fps should suffice, saving you considerable file size when having to manage these large recordings.

Tagging annotations

When you’re walking the participant through the paces of the survey, be sure to annotate the time started and any high/lowlights you see along the way. While in front of your desktop, a basic note-taking utility application (or even pad and paper) should suffice. This will allow you to go back after the survey is finished and pull quotes for use elsewhere, such as powerpoint presentations or similar.

I always try to write a running diary of the transcript and a summary at the end just to cover what areas of the application we explored, as well as a quick summary of what feedback we gathered. Summarizing the typed transcript and posting the relative recorded video files should take no more than 10 minutes, which will still keep your total per-participant (including processing) time to under an hour each, certainly manageable as a part of your greater schedule.

Share the love (or hate)

I want to make sure that these sessions are able to be referred to by the executive and product management team for use in their prioritization strategy. Setting up an instance of MAMP / WordPress on a local box (we’re using one of the Mac Minis that power a dashboard display) which allows me to pass around the link internally and not have to deal with some of the issues around large video file sizes being uploaded, as well as alleviate any permissions concerns with these sessions being out in the wild.

Screen grab of the session archive interface.
Our UX session archive, with hundreds of recorded and tagged sessions.

Also important is to tag these posts attached to these files when you upload them. This allows faster indexing when trying to find evidence around a certain feature or function. Insert your written summary into the post content, and you’ll be able to better search on memorable quotes that might have been written down.

These resources can be very good for motivation internally, especially among the engineers who don’t often get to see people using the product they continually pour themselves into. They’ll also resonate with the product team, who will see first-hand what’s needed to re-prioritize for the next sprint.

After awhile, you’ll start to get a great library of clips that you can draw knowledge from. There’s also a certain satisfaction to seeing the evolution of the product in the interface through these screengrabs. That which was shown as confusing at one time may now be fixed!

Follow-up

Fulfillment of a participant compensation can be done through Amazon or other online retailers; you can wire a gift card through an email address, which you’ll be able to scrape as a hidden field from the spreadsheet of user inputs. Keep a running list of those that you’ve reached out to and contacted for responses.

You might also incorporate contacts met during sessions described in the Guerrilla Usability Testing at Conferences article, so you’ll be able to follow up when attending the next year’s conference to recruit again. After enough participants and feedback, think about establishing a customer experience council that you can follow up on with specific requests and outreach, even for quick vetting of opinions.

Conclusion

This article first outlined the strategies and motivation behind the research, advocating creating an automated workflow of continually-scheduled screenshares with customers, rather than trying to recruit participants individually. This methodology was then broken down to distinct steps of recruitment via email, gathering quantitative and qualitative feedback, and automating an opt-in booking of the sessions themselves. Finally, this article went on to discuss how to best leverage and organize this content internally, so that all might benefit from your process.

User research is imperative to the success and prioritization of any software application (or any product, for that matter). Yet, too often we forget to consume or own product. Whether it be server log management as I’ve chosen, or apartment listing or ecommerce purchases, shake off complacency and try to spend 30-mins a week trying to accomplish typical user tasks from start-to-finish.

Also make it a point to conduct some of these sessions among those you work alongside; you’ll be surprised what you can find just by the simple repetition with a fresh set of eyes and ears. The research process and its dependencies does not have to be as intricate as the one listed above.

 

When your company starts to incorporate user opinion into a design and development workflow, it will begin to pay out dividends, both in the perceived usability of your application as well as the gathered metrics of user satisfaction.

 

Honing Your Research Skills Through Ad-hoc Contextual Inquiry

Written by: Will Hacker

It’s common in our field to hear that we don’t get enough time to regularly practice all the types of research available to us, and that’s often true, given tight project deadlines and limited resources. But one form of user research–contextual inquiry–can be practiced regularly just by watching people use the things around them and asking a few questions.

I started thinking about this after a recent experience returning a rental car to a national brand at the Phoenix, Arizona, airport.

My experience was something like this: I pulled into the appropriate lane and an attendant came up to get the rental papers and send me on my way. But, as soon as he started, someone farther up the lane called loudly to him saying he’d been waiting longer. The attendant looked at me, said “sorry,” and ran ahead to attend to the other customer.

A few seconds later a second attendant came up, took my papers, and jumped into the car to check it in. She was using an app on an tablet that was attached to a large case with a battery pack, which she carried over her shoulder. She started quickly tapping buttons, but I noticed she kept navigating back to the previous screen to tap another button.

Curious being that I am, I asked her if she had to go back and forth like that a lot. She said “yes, I keep hitting the wrong thing and have to go back.”

Seeing an opportunity to explore her use of the device, I asked if this happened a lot. She said it did because the buttons were too small for her fingers. She told me every time a new feature was added to the reservation system, more buttons appeared on the screen and they all kept getting smaller. The system, I have to assume, was designed to use as few screens as possible.

Within two minutes she completed the check in and printed my receipt. She did it using a small battery-powered printer she wore over her other shoulder that printed roughly three-inch wide receipts. I asked how often she had to recharge her devices: once a day, usually at break times.

Her activity and my interaction with her took no more than a few minutes, but this is what I was able to glean from it:

  • Airport rental car attendants work in a fast-paced environment, often dealing with agitated customers who are running late for their flights.
  • These attendants, at least for this company, carry two devices over their shoulders as they work the busy return lanes.
  • The app in use at this company wasn’t designed to be the most finger-friendly. A greater emphasis seemed to be placed on having more features on fewer screens.
  • In addition to monitoring incoming vehicle traffic, they have make sure their devices have enough battery power to make it to the next break.
  • They are doing this work, carrying these devices, and being polite to sometimes impatient customers in a climate where the summer temperatures easily exceed 100ºF. And they are on their feet in a concrete parking deck for much of their work day.

That’s a lot of information to gather in a few minutes by asking three questions. What I found most interesting in the experience, as someone who designs digital user experiences, was the reminder that people around us every day are having problems using the hardware and software we create. It reinforced for me that we can learn a lot about how to approach hardware and software design just by watching people use these products, even products we had no part in creating.

Now, I certainly can’t claim to have found ways to improve the reservation system in question. That would require more field study with additional users as well as talking to the product designers and other stakeholders to understand the decisions that led to the current user interface. But, the experience left me wondering if the company ever bothered to watch its employees use the system in a real-world setting.

My point here is that we can still practice these observational research techniques even if we don’t always get the chance to do so on the products we get paid to design. And we can document these moments to make a case at our companies for why this kind of research needs to be part of our product development cycles. There’s value in these exercises, even if the results don’t immediately show up in our companies’ products. It’s an activity I’d encourage everyone to try.

There are plenty of opportunities for this sort of activity, including:

  • Retail self-checkout lanes,
  • Any self-service airport application,
  • Self-serve vending machines like Redbox or mass-transit ticketing machines,
  • Hotel check-in/check-out kiosks,
  • And many others I’m sure I missed.

There are a few things you need to be aware of and take into consideration before attempting to interact with strangers in the middle of a frustrating technology moment:

  • The people in these situations may be angry and flustered, so look for body language cues that may signal you are better off leaving them alone. If they are hitting the machine and cursing, you may just want to hang back and observe.
  • Politely approach people and ask if it’s OK to talk to them about their experience with the system in question. In most cases it’s better to wait until they are done because right at the moment they are using it they are probably more focused on completing their task.
  • Make sure you don’t imply they are doing something wrong. Let them know you are interested in making products better for people and would like to ask them a few quick questions about the experience they just had. If they say “no,” respect that and move along.
  • Explain that you are not with the company in question and can have no impact on any future version of the product. The last thing you want them to think is that you are a conduit for complaints to the company or that you can actually have any impact on future versions. It’s important to be as clear as possible to avoid setting false expectations.
  • Be sensitive to your surroundings and other customers who may be waiting to use a system. The last thing you ever want to do is slow down a busy checkout lane or self-serve kiosk at an airport.
  • Be sensitive to the application at hand and the location and time of day. It’s safe to say you should never approach someone using an ATM or other financial services kiosk. You also shouldn’t approach someone using a self-serve vending machine that is outdoors late at night. That could make them feel threatened or even cause them to react physically.
  • Honor any request by the business in question to not approach their customers.

Although exercises like this won’t tell us the things we’d like to know about the products we work on, they do let us practice the techniques of contextual inquiry and observation and make us more sensitive to various design issues. These experiences may also help us build the case in more companies for scheduling time and resources for in-field research with our actual customers.

Creating Your Personal Mission Statement

Written by: Louis Rosenfeld

You’re weird. In a good way, but weird nonetheless.

Weird in the sense that people outside of work likely have absolutely no clue what it is you do. Maybe many at work as well.

For me, this weirdness manifests itself at parties. Inevitably, a new acquaintance asks me what I do. Beads of sweat form on my forehead. My eyes dart around, desperately seeking my far more articulate wife, Mary Jean. I find her, ask her to explain me, and flee.

If you’re in UX or a related field, congrats: You probably have more work than you can manage in a time when many people are underemployed. But that doesn’t diminish the discomfort those weird moments cause.

How might we explain ourselves better?

I created a simple exercise to help create personal mission statements—something short and meaningful to say about yourself—with some help and encouragement from Christina Wodtke and Anders Ramsay. It’s fun, simple, and quick; in fact, I recently tried it out with a group at the Re:Design conference and it seemed to work well. Here’s what to do.

Start with a context.

It could be public, like something to include on your business card, your Twitter bio, or blurt out at those nerve-wracking parties. Or something private, like a statement you write down and keep in your wallet for you and you only.

Tell your story.

Find a friend or colleague—someone you don’t mind being a bit vulnerable with—who will take notes while encouraging you to talk about yourself and what’s important to you. Ten minutes is plenty.

Basic questions like these can get you going:

  • What kind of change would you like to be part of?
  • What’s your superpower?
  • What’s the difference between what you’re expected to do and what you want to do with your life?
  • What has always pissed you off?

Craft a short statement.

Together, take those key terms and phrases from the notes and work them into something that fits the context you chose. Easier said than done, so plan to iterate.

At Re:Design, I was the guinea pig. My context was finding a new Twitter bio to replace this one:

Founder of Rosenfeld Media and veritable UX action hero.

“UX action hero” is an inside joke: pointless for 99.99% of the people who encounter my bio. Time to axe it.

I started with the “What has always pissed you off” question, and told my session’s attendees a story of 20+ years of frustration with the traditional business models (and their defenders) that I’ve encountered in higher ed, consulting, publishing, and professional associations. Telling one’s “true story” is never easy—especially in front of 70 people—but two things really helped:

  1. The floor is yours. Whether you’re telling your story to one person or 70, it’s your time. Take as much of the ten minutes as you wish, and make it clear that there will be opportunities to discuss your story and brainstorm after you’re done.
  2. Vulnerability is engaging. The goal isn’t to impress anyone with a slick presentation of your many fine attributes; rather, use this opportunity to begin figuring out what you’re about. Your stumbles and your quirky, imperfect, unfinished story will actually draw in your partner(s). If they have an ounce of empathy and interest in you, they’ll be able and eager to help brainstorm ideas about who you really are.

My group was anxious to brainstorm before I was even done telling my story. They came up with these options:

  • Happy but not satisfied
  • Re-architect/Assassin of business models since 1965
  • Shapeshifter
  • Internet sherpa
  • Learning by teaching
  • Step, pivot, repeat

My faves are the second and the last one especially, as verbs (like “step, pivot, repeat”) are fairly constant when it comes to describing how we live and what we do. And if I wasn’t happy with any of these, repeating the exercise with someone else would have made sense—after all, it takes less than a half hour. In fact, I’m considering repeating it as often as I get the urge—maybe once per year.

In any case, here’s my new Twitter bio:

Founder of Rosenfeld Media. Slayer of traditional business models. Step, pivot, repeat.

And here are shiny new Twitter bios from some of the session’s attendees:

helping people be less afraid of making things better

…and…

Persuade. Connect. Convince. Judging user experiences everywhere.

Though I’m not sure which questions got each of them started, I am sure they were excited. In fact, they were anxious to share their new personal mission statements with the group.

They found it fun, simple, and quick. I hope you will too.

How to Make a Concept Model

Written by: Christina Wodtke

IMG_0048

I can draw.

I went to art school. I studied painting until I fell out with the abstract expressionists and switched to photography. But I can draw.

What I cannot do is diagram. I always wanted to. I have models in my head all the time of how things work. But when it comes time to make a visual model of those ideas, I can’t figure out to to represent them. I find myself resorting to pre-existing models like four-squares or the Sierpinski triangle (I dig fractals.) For example:

Social-Architecture-Diagram

Other than the oh-god-my-eyes color choices, my social architecture diagram has deeper problems. For example, the ideas in it are limited to threes within threes because that’s the form triangles take. The model served to communicate my ideas well enough for the sake of my workshop, but… shouldn’t form FOLLOW meaning? If I had more than four elements for any section, I’d have to either collapse two, or fudge it in some other way. I was sacrificing accuracy for consistency. But I didn’t know how to make to make it better.

A concept model is a visual representation of a set of ideas that clarifies the concept for both the thinker and the audience. It is a useful and powerful tool for user experience designers but also for business, engineering, and marketing… basically anyone who needs to communicate complexity. Which is most of us, these days.

The best known concept model in the user experience profession is probably Jesse James Garrett’s “Elements of User Experience.” The best known in start-up circles is the lean startup process. Both of these models encapsulate the ideas they hold in such a memorable way that they launched movements.

Elements-User-Experience-Lean-Startup

If you wish to clearly present a set of ideas to an audience and represent how they fit together, a diagram is much more powerful than words alone. Dan Roam points this out in his latest book, Blah Blah Blah:

“The more we draw, the more our ideas become visible, and as they become visible they become clear, and as they become clear they become easier to discuss—which in the virtuous cycle of visual thinking prompts us to discuss even more.”

Concept models can serve many purposes. You can use concept models to show your teammates how a complex website is organized before the site is built…

Andrew Hinton’s model of a “virtual shared organizational ‘building’ where people spread all over the country were collaborating to run and participate in the org”.
Andrew Hinton’s model of a “virtual shared organizational ‘building’ where people spread all over the country were collaborating to run and participate in the org.”

… or to help teammates understand how the site currently works…

Bryce Glass’s concept model of Flickr use.
Bryce Glass’s concept model of Flickr use.

… or to show end users how a service works, to help sell it.

    Biblios uses a concept model to help users understand the power of social cataloging. What it lacks in elegance, it makes up in clarity.
Biblios uses a concept model to help users understand the power of social cataloging. What it lacks in elegance, it makes up in clarity.

I teach user experience design, and my syllabus always includes concept models. Students of mine who do a concept model before working on the interaction design and information architecture always make better and more coherent products. The act of ordering information forces them to think through how all the disparate elements of a product fit together.

Stephen’s handout from the workshop on representing types of visual relationships. Advanced and useful thinking.
Stephen’s handout from the workshop on representing types of visual relationships–advanced and useful thinking.

You can imagine how excited I was to take the Design for Understanding workshop at the 2014 IA Summit. Partly because I will go see anything Karl Fast or Stephen P Anderson talk about and having them together is Christmas come early. But mostly in hopes of learning a way to make a good concept model.

The workshop was brain-candy and eye-opening: They covered how the brain processes information and how ways of interacting with information can promote understanding. BUT I still couldn’t make a model to save my life. I didn’t know where to begin!

At lunch, Stephen was manning the room while Karl grabbed food for them. I had been struggling with a model for negotiation I wanted for a talk I was presenting later in the program. Seeing Stephen idle, I pounced and begged for help.

Stephan P. Anderson is author of Seductive Interfaces and the upcoming Design for Understanding. He’s also a patient soul who will put up with ham-handed diagramming and ridiculous requests. He started to sketch my model and tell me what he was thinking as he drew. Then I had my bingo moment: Stephen had forgotten what it was like not to know how to begin! This happens to all experts. After a while some knowledge is so deeply embedded in their psyche they forgot what it was like not to know. They then teach the nuances rather than the fundamentals.

I suggested we do a think aloud protocol while he made a concept diagram; he would draw, and I’d prompt him to talk about what was going through his mind. He was excited to have me reflect his thinking back to him so he could become a better teacher as well. We arranged to have a sketching session after the workshop.

 

    Stephen Anderson draws; I do a think aloud protocol to capture how he works.
Stephen Anderson draws; I do a think aloud protocol to capture how he works.

Later in the day, we met in the quiet hotel bar with wine and a sketchbook. I asked him what he wanted to draw. “Do you have something you are working on?” he asked. “That way I can focus on the model, rather than rethinking the ideas.”

Did I have a model I was struggling with? Always!  I shared my new theory of the nature of digital products. I’ll be writing that up in another article when it’s done, but for now, the short version is that one must iterate through the elements of digital design, which include the framework, interactions, information structure, and aesthetics. But a product doesn’t become an experience until a person interacts with it; your design cannot be known until you see what happens when a human shows up.

Stephen’s first step was to ask me about my goal for the model. I said it was for students and young practitioners to understand the interdependencies of the elements, so they have a more iterative approach. And for critics to be able to understand why things are different, both good and bad.

Next, he did what I’d call a idea inventory. He brainstormed more elements that might play into the model. He made sure no ideas were left out. He made notes of those he suspected might be important in the margins. He sketched as he thought, sometimes just making meaningless marks, as if warming up his hands.

He then carefully asked about each element in my theory, making sure he understood each. What was an information structure and what was a framework and were they different? I ended up telling a little story about a product to make sure he got what I was explaining. I began to draw too, encouraged by his easy scribbles.

Finally, Stephen noted the relationships of the items to each other. Were some things subsets of others? Were some overlapping, or resulting?

Playing with relationships (my drawing).
Playing with relationships (my drawing).

Once he knew what each item was, and how they were related to each other, he began to sketch in earnest. He said, “I always start with circles because edges mean something. They mean you have four items, or five. Circles leave room for play.” His circles quickly became blobs and then shapes.

I don’t know if he’d normally talk to himself out loud when not encouraged to do so, but it was fascinating to to hear him free associate concepts, then draw them out. A string of concepts became a string of beads; moving through an experience became moving through a tunnel; intertwined ideas were a braid. Any important idea got a drawing.

Here Stephen tries on various relationship metaphors, including moving through tunnels, holding something, string of ideas, braided together concepts.
Here, Stephen tries on various relationship metaphors, including moving through tunnels, holding something, string of ideas, and braided-together concepts.

Each time he completed a mini-model, he’d evaluate what was missing and what was working and take that insight to the next drawing. He made dozens of these little thumbnail drawings.

Stephen said, “one shape leads to another…a single word sparks a new representation—we’re always ‘pivoting’ from one thumbnail to the next…”

He pointed out what concepts were left out, or where they could be misinterpreted.

“You want to avoid 3-d, because it’s fraught with problems. You want to be able to sketch it on a napkin.” —Stephen Anderson, on keeping in mind the model’s goal

At one point, he became tapped out, and we spoke of other things. We stared out the window at the harbor, and I drank some of my wine, forgotten in the excitement of drawing and talking.

Then suddenly he started in again and produced a flurry of new drawings. I realized resting and mulling was important too. I was a bit annoyed with myself. An article doesn’t come out perfect in one writing session. Why should I expect a concept model to just materialize?

Finally he came to a stop, several pages filled with a jumble of images. We didn’t have a model, but we had many good directions. As we finished our drinks and headed toward the opening reception, Stephen told me, “You gotta get Dan Brown to do this, too.”

section-break

Dan M. Brown is best known in the user experience design community as author of Communicating Design and Designing Together. Both books benefit greatly by clear and succinct conceptual models, and the former even talks about how to use them in the design process:

Purpose—What are concept models for?
There really is only one reason to create a concept model: to understand the different kinds of information that the site needs to display. This structure can drive requirements for the page designs, helping you to determine how to link templates to each other. With the structure ironed out, you might also use the model to help scope your project—determining what parts of the site to build when.

Audience—Who uses them?
Use concept models for yourself. Ultimately, they are the most selfish, introspective, and self-indulgent artifact, a means for facilitating your own creative process.”

–Communicating Design: Developing Web Site Documentation for Design and Planning 2nd Edition, Dan Brown, 2010

Clearly, a guy I should be talking to!

The IA Summit was held in sunny San Diego in a hotel with not one but two swimming pools, so Dan had brought his family with him. When I asked him if I could watch him draw a concept model, he said, “I’m at the coffee shop with the boys around 6:30 every morning.”

You take what you can get.

The next morning Dan settled the boys in a corner with books, pastries, and an emergency iPad, and we got to work. We agreed he’d model the same concept, to control for variations. By now I had created a formula for the idea: (F+In+Is+Ae)+P=E. Framework, interactions, information structure, and aesthetics plus a person makes an experience. I was modeling in words as my friends were modeling in pictures.

I took Dan through the same story of an iterative product design process, since it had helped Stephen. I sketched it out. I felt like my hands were waking up from a long sleep, and they were eager to hold a pen now.

As I spoke, Dan wrote down key ideas and also began to scribble. He used the same process as Stephen: collecting the concepts then inspecting them for hidden complexity.

“A question I ask myself is ‘what needs unpacking?’ I can’t diagram an idea until it’s clear in my own brain.” —Dan Brown

He then took each concept and free associated all the sub-elements of the concept. He drew them out loosely, mind-map style.

Dan also started with the goal and wrote it out across the page.

Dan-Brown-Goal

He also asked explicitly who the model was for. To draw, he needed to visualize the audience. This reminded me of a recent presentation workshop at Duarte where we literally drew pictures of our audience. No work can be good unless you know who it’s for.

Duarte has you draw your audience before you design your presentation, so you remember who you are presenting to and how much attention they are (or aren’t) giving you.
Duarte has you draw your audience before you design your presentation, so you remember who you are presenting to and how much attention they are (or aren’t) giving you.

Dan made sure he didn’t carry anything in his head: All ideas were put on paper as a note or a sketch. When he had to turn a page, he ripped it out to lay it next to the other pages. I realized how critical it was to have plenty of room to see everything at once. I saw the same technique of storytelling and drawing of ideas.

Around now, Stephen joined us. He was excited to see what Dan came up with, enough to also climb out of bed at the crack of dawn. I listened as the two diagrammers discussed the poster session and the strengths and weaknesses of the ideas that had been presented.

Dan said, “You can look at people’s posters and see their process. They are so close to their own narrative…In one poster, the key framework was rendered in a very pale text. It was a good story, but there are things you want to jump off the page. For her, my guess is those steps were so self-evident she didn’t see need to highlight them.”

 You have to have a beginner’s mind to explain to beginners.

“Speaking of beginner’s mind, so much of my design process is to throw it all out start all over again.” —Dan Brown

Dan Brown draws it all.
Dan Brown draws it all.

Now Dan began to model the concept. He emphasized the importance of sticking with very simple geometry–circles, squares, triangles, lines–not fussing with trying to find a perfect model at the beginning, just exploring the ideas and their relationships.

He also mentioned he begins with any concept in the model and doesn’t worry about representing order at first. He starts with what catches his interest to get familiar with the ideas.

Dan then deviated from Stephen by seeking the focal point. What concept held all the others together? What was the most important or key idea? He tried out placing one idea, then the other, in the center to see if felt right.

After scrapping one bowtie model, he paused. “I sometimes retreat into common structures and see how these common structures might speak to me. For example, time is one of those fundamental aspects, so I ask myself: How much do I need to show time here?”

He demonstrated by drawing swimlanes and sketched the ideas and their relationships in time.

Swim lanes for moving across the elements.
Swimlanes for moving across the elements.

“Are there other elements you often look for, like time?” I asked

“People,” he replied. “People and time are familiar concepts, easy for an audience to relate to. By using them as a foundation for a model, I’ve already made it easier for people to ‘get on board.'”

He stared at the paper, deep in thought.

Stephen then pointed at the page. “What Dan did here,” he said, poking at where Dan wrote out goal and audience, “I did also but didn’t externalize. I was holding it in my memory, but I like having it on the paper better.”

Eventually Dan, too, was tapped out, and his sons began to play Let It Go on the iPad at higher and higher volumes. He separated his sons from the electronics and left to prepare for the swimming pool.

 

section-break

After Dan, I knew I wanted to try to get one more person to model. Since I was lucky enough to be at a conference full of diagrammers, I chased Joe Elmendorf of The Understanding Group. He had just given a talk on Modeling for Clarity that my friends were raving about. And, with my luck still holding, I got to have breakfast with him. Happily, at 8 am this time.

Joe Elmendorf brings pace layers into the discussion. My handwriting is the ballpoint; his, the nice black ink pen.
Joe Elmendorf brings pace layers into the discussion. My handwriting is the ballpoint; his, the nice black ink pen.

Again, I saw what were becoming familiar concepts (inventory, inspection, relationships, then talk-draw.) I then focused on how he differed from Stephen and Dan. He choose to use the title of the diagram as an element. He did not iterate as widely as Stephen. He was the first person to argue with me about the validity of my theory, which was a great way to understand it (and benefited me by making it better!).

As well, he reinforced something Stephen had mentioned in his workshop and that Dan was obviously doing: Joe had a large mental library of typical models to draw upon, which got him started. Stephen keeps a Pinterest board full of inspiration, if you want to start your own “lego box” of models.

Stephen’s Board http://www.pinterest.com/stephenpa/the-visual-display-of-information/
Stephen’s Pinterest board: http://www.pinterest.com/stephenpa/the-visual-display-of-information/

Overall, there were so many familiar patterns I saw in his approach, the differences were more interesting than important. I had my answer. I knew how they did it.

section-break

On the last day of the conference in the afternoon, Stephen and I were scribbling further on the model, playing with petals for the elements, when Dan Willis joined us. Dan is also a master of models as well as an inveterate sketcher.

Stephen further refining ideas, always generating.
Stephen further refining ideas, always generating.

Although Dan declined to diagram for me, claiming brain fatigue (a reasonable claim at this year’s Summit) he pulled up a chair and sat sketching next to us. It was companionable, to sit and talk and draw ideas. We moved back and forth from discussing life to discussing the ideas, teasing, joking, drawing. As we chatted, I realized this was a part of the secret. You need a thinking partner. Sometimes it’s paper, sometimes it’s friends; but it’s best when it’s both. It doesn’t always matter what you draw, just that you draw.

Dan Willis drawing nearby makes me happy.

Dan Willis sketch from a tweet
A Dan Willis sketch from a tweet that day.

Our brains work better when our hands are busy.

section-break

Later, sitting in the back of a session, I lobbed a model at Stephen, and he shot back with his own.

Refining an idea, mine on left, Stephen’s on right.
Refining an idea; mine on left, Stephen’s on right.

Then I saw another step, one which Dan had alluded to when he mentioned the poster with the key point too pale to read: You have to refine the model to communicate effectively. Type, color, and labels are all a key part of the communication process. While the model did stand alone without the color and type, adding those–and most especially getting labels right–made the model more effective.

section-break

DaveGrayAfter getting home, I started sketching how concept models were made. I drafted this article and then asked my friend Dave Gray if he’d do a quick edit. Dave was the founder of Xplane, a company that used diagrams–concept and other–to transform companies. Dave has been a proponent of visual thinking and clear modeling for years, and I consider him the master of making ideas visible.

Life then intervened and this article sat. I was busy with several things, including Lou Rosenfeld’s 32 Awesome Practical UX Tips. Dave presented right before me, and watching him sketch, I realized I just had to get one more diagramming session in. It was not enough to have him comment, I needed to see him draw. I was grateful I did; otherwise, I would have missed a crucial piece of the puzzle.

Dave Gray draws on cards so he can rearrange, manipulate, and overlay the concepts.
Dave Gray draws on cards so he can rearrange, manipulate, and overlay the concepts.

We hopped on a Google Hangout and he also drew out that same darn design model for me. I saw familiar patterns in his approach: inventory, unpack, relationship exploration. But he added a critical step I hadn’t thought of before: Test the model.

He’s currently writing a book on Agile, and it shows. He said, first design the test, then design the thing. For the model, he suggested using his WhoDo Gamestorming tool as a way to design a test of the effectiveness of the model. He lists who the model is for and what they will do if they understand the model.

If Dave didn’t fully understand the audience for the model, he might do an empathy map for those people.
If Dave didn’t fully understand the audience for the model, he might do an empathy map for those people.

Designing a test of the model’s success radically clarified the goals for the model. Testing it would make sure it did what you wanted it to do.

section-break

So then I sat down to make a model of how to make models. And it came easily.

  • Determine the goal: How will the model be used, by whom? What is the job of the model? To change minds, explain a concept, simplify complexity?

  • Inventory the concepts: Brainstorm many parts of your concept. Keep adding more in the margins as you go.

  • Inspect the concepts: Are there many concepts hiding in one? Do you really understand each idea?

  • Determine the relationships: How do the concepts interact?

  • Decision point: Do I understand the ideas and what I’m trying to communicate? 
    Test: Ask yourself if the model “feels” right.
    If yes, then continue.

  • Iterate with words and pictures: Talk to yourself and draw it out!

  • Evaluate with yourself/the client: Keep making sure the drawings match the ideas you wish to communicate. Don’t punk out early! Rest if you need to!

  • Decision point: Does my audience understand the ideas and what I’m trying to communicate? 
    Test: Can my audience answer key questions with the model? 
    If yes, then continue.

  • Refine: Use color, type, line weight, and labels to make sure you are communicating clearly.

A model for making models.
A model for making models. It may not be beautiful, but it’s clear.

The concept model is invaluable. But like so many useful things, it takes time to make.

When my daughter first started drawing My Little Pony, she expected to start at the ears and draw it perfectly down to the hooves. She was angry when it didn’t work that way, and it took some convincing to get her to block out key shapes then refine the whole, and to use pencil before ink. When I sat down to make a concept model, I made the same mistake! I’d start in Powerpoint or Grafio, and expect perfection to flow from my mind.

No more! Stephen, Dan, Joe, and Dave taught me to play, explore, refine, test, and play some more until the result was right. Thank you all!

Now go make a model!

section-break

Postscript

If your hands do not obey your brain, and/or you need more ideas for shapes and relationship models, I recommend Dave Gray’s Visual Thinking School.

See my interview with Dave on how he’d make the experience model