Benchmark Your UX at All Costs

In the user experience (UX) industry, benchmarking is a practice that measures the usability of a website. Benchmarking helps the UX researcher understand the current state of the site so the team can attack problem areas and improve them.

It is very difficult to fix or improve something when you don’t know what is wrong. Imagine taking your poorly-performing car to the mechanic and they ‘fix it’ without running any diagnostics or tests. They may change the spark plugs, rotate the tires, and add a few decals but will most likely miss identifying and addressing the root problem. This is similar to approaching a site redesign without first identifying its problems with real site users.

Comic showing two men talking to each other over a car's open engine compartment. The customer has his pants down. The mechanic says "No, I mean YOU need a new belt."
From http://www.thecomicstrips.com/subject/The-Auto+Mechanic-Comic-Strips.php/3

Too often, benchmarking is skipped due to constraints of time and budget—a big mistake. There are ways to still gain these valuable insights while working within a limited project scope and with a small budget. Guerrilla testing is a term used to describe efficient alternatives to testing, some of which are workarounds or shortcuts.

Let’s look at some of our favorite guerrilla testing methods that can be used to gather crucial insights into the user’s mind and the site’s performance, all without breaking the bank or the project plan.

Remove the middleman

Finding participants can be expensive, because you have to pay both recruiters and participants. Remove the middleman and find participants on your own. We have had great success using co-workers, family, and friends. Depending on the commitment, participants can be compensated with anything from gift cards to a larger monetary amount; in our experience, people are happy to help. Who doesn’t love giving their opinion?

One of the most successful methods for collecting data can be a Google form. Free to use and painless to set up, these forms are flexible and the results produce an easy-to-read data visualization for quick analysis. Pair a Google form with the family and friend participants and sit back and wait for the information to come pouring in.

Photo collage showing a basket full of treats on the left and two researchers holding the basket on the right.
Trading nutrition for research, what a deal! From http://alistapart.com/article/never-show-a-design-you-havent-tested-on-users

Other researchers have even taken their testing to the streets with bags of candy or fruit. If you live in a city, it won’t take long to find someone with a spare 10 minutes who would be willing to give you insights for a tasty treat.

Leverage current software

Your company may have the budget for video communication software but not testing software. Have no fear: There are alternatives to some of the expensive industry-leading user testing tools. It is very easy to take advantage of tools such as Join.Me, Blue Jeans, Silverback, and User Zoom to moderate and record your own tests. The direct cost savings is huge, because the tool itself could potentially be zero extra dollars. There is, however, an extra level of effort involved, since your team will be doing all of the logistics and preparation. Plan for those hours accordingly.

The paleo version of this would be to simply conduct your research in-person and audio record or (gasp) take notes for the team’s reference.

Go thrifting

Bookmark a list of tools that are neither brand new nor top of the line but get the job done in a pinch. One such tool that has worked well in this capacity is Zurb’s Verify. It can be used combine up to seven-plus different test types to gauge user feedback; types like the 5-second test, preference test, and click test are tried and true usability staples. With a monthly price starting at $19/month, it is a bargain and has proven its value time and time again.

Screen grab of test options offered by Verify.

Screen grab of results from a Verify test.

Involve the community

There are also community-driven web-based tools that can be used for free; one such example is usabilityhub.com. On this site, you have the option to create a few different types of default tests, ranging from five second tests to click tests. These are then opened up to a panel of their users to participate in; the majority of the users are in the industry (people like you and me). Although these insights are not the best of the best, we have found them helpful in a pinch.

Go on trial

Many robust applications also allow you to use their software in a trial or lite fashion. Optimal Workshop’s Tree Testing tool allows 10 users to perform three tasks on your site structure; look for other diet versions of your favorite software to take advantage of when budget is restricted.

Treemap result from TreeJack.

In conclusion

Removing the middleman, leveraging current software, going thrifting, involving the community, and going on trial are all sneaky ways to guerrilla test—but these techniques don’t answer what to test. The most influential tests we have run that produce actionable insights to our projects are task-based and are done when we narrow our participants’ focus to one of two areas:

  • The site structure and flow
  • The site features and content

The results of these types of tests can be quantitative (numbers that tell a story: the what) or qualitative (emotions, opinions, reasons, motivations: the why). It is imperative to balance your research with both.

If benchmarking does not play a significant role at the beginning of your user experience projects, you are not designing for the user; you are designing based on your own assumptions and biases—and the latter will very likely produce undesirable results.

Please share additional ways you benchmark with guerrilla testing methods. Add them in the comments below, and we may feature them in a future article.

10 Practical Tips for Increasing the Impact of Your Research Insights

User experience (UX) researchers tasked with improving customer-facing products face many challenges on a daily basis—perhaps none more daunting than translating their research insights into positive change. This article presents 10 tips I have learned over the course of my career to help UX researchers increase the impact of their research insights in applied settings. These tips are intended primarily for in-house research teams, but they may apply to consultancies as well.

1. It’s a long-term relationship, not a speed date.

For research insights to impact a specific product, the researcher must establish a deep connection with the product team. What is the context that the team is operating within? What are their constraints, challenges, and deadlines? What are immediate goals, and what objectives are longer-term? How is success measured by their superiors? Given those circumstances, what can you do to help them without adding to their workload? Above all, you want your product team to perceive you as a benefit rather than a hindrance. This is not to say that you cannot critique their work, but never lose sight of the their ultimate goal: to create products in a timely fashion that will drive business metrics when launched.

For example, suppose your product team is racing to launch the product by a specific deadline, and you discover a problem with the product during testing. In such situations, you should not immediately advocate that the launch be delayed. Instead, think about the best way to raise the issue (and to whom you should raise it) and frame the issue in terms of potential courses of action. Can the team reallocate resources to keep the current launch schedule? Can the problem be corrected in the next dot release of the product? Is the problem serious enough to warrant pushing back the launch schedule? Over the course of working with your team, you will have many opportunities to demonstrate empathy for their challenges. Doing so will build trust and help you become an integral member of the team.

2. Serve ‘ready to bake’ bread.

The best way to ensure that action will be taken to address your research insights is for someone else to take ownership. Researchers often view it as their responsibility to make specific recommendations from their research, but what really matters is that the insights lead to clear potential solutions. The solution doesn’t need to come from the researcher herself.

Instead of a recommendation that does not offer any unique value—for example, “Users were confused by X. My recommendation is to fix X”—a better strategy is to offer specific recommendations when they are not obvious and when you have confidence that they will add unique value to thinking about the problem.

Better still is something else entirely: Offer “ready to bake” bread.

In other words, present the problem in a way that clearly lends itself to a specific solution and let a product team member connect the final dots so that they consider it their idea and take ownership over it. In cooking parlance, give them bread that is basically already made but needs only a few more minutes in the oven. So, instead of giving them raw dough (raw data) or fully-baked bread (a specific solution that you explicitly offered), give them ready to bake bread. People are more likely to follow-through on something when it is their idea and everyone implicitly ascribes ownership of the idea to them.

One way to do this is to frame key research insights as “How might we?” questions to spur discussion. Doing so can effectively portray the researcher as a facilitator of collaboration rather than someone who provides heavy-handed mandates for the team.

For example, suppose users in a research study exhibited difficulty navigating to specific types of offerings. Instead of recommending that the team “Implement more top navigation menu options,” you can instead ask  “How might we convey the breadth of our offerings and provide quick access to different product categories?”

3. Know the whole story before writing the first chapter.

Researchers should strive to present their research insights in a clear, compelling, and elegant way. Instead of proceeding straight from raw notes to a Powerpoint presentation, synthesize the data and create an outline summarizing the main ideas and how best to convey them. Presentations that skip the important synthesis step can seem disjointed or disorganized and can lead to audience fatigue or lack of clarity about the most important points.

These best practices are not limited only to qualitative research, such as user visits or usability studies; they apply to quantitative research, such as surveys, as well. Common practice when presenting survey-based research is to devote one slide to data relevant to each survey question and present detailed information such as charts, graphs, statistical significance across numerous comparisons, and so on. However, a better practice is to first create an outline that summarizes the top 3 or 5 or 10 key points that you wish to convey to your audience based on the survey data as a whole.

4. Don’t lose the forest for the trees.

As difficult as it may be, researchers need to resist the temptation to share every interesting finding from a study. When a researcher presents findings, he is telling a story and needs to be mindful of tangents that detract from the overall message. So if you want to convince the audience of a few key themes, ask yourself if each point you are making ties back to one of those themes. If not, then perhaps another forum would be more appropriate to share it.

I tell researchers that there is a continuum with impact on one end and exhaustiveness on the other, and different researchers operate at different points on the continuum. In my opinion, it is best to land on the side of impact.

If you have one hour to present important insights to senior stakeholders, it is more important that those stakeholders leave the room with the key points to act upon. It is less important that they understand every subtle nuance of the data or that other stakeholders can find such nuances a year from now. I call the one-hour presentation “The Golden Hour.” Once those stakeholders leave the room, you may get another opportunity to convey the key points, but you’ve lost your best opportunity to make a lasting impact.

5. Don’t crawl across the finish line.

After presenting insights, researchers typically send an email that shares the deliverable with the immediate product team as well as with the entire product organization. This may seem like a minor, perfunctory step, but it is a very important factor in determining what happens next.

An effective announcement conveys the key takeaways in a clear and concise manner and serves as a catalyst to action. Frame the email in the context of the team and the project (e.g., If their overall goal is to increase engagement, frame it in that light.)

Or, if the product lead reframed your research in a specific (helpful) way, build on that. Once, in my experience, a project manager concluded from a research project about improving the map view of search results that the research instead underscored the need to improve the list view of search results. I built off of that in the announcement, both because it was provocative but also because everyone else in the room heard the comment as well—and I agreed with it.

And don’t lose sight of the basics. Have a clear and compelling title for your email that draws the reader in. You want to get other people interested in your research who may not have attended the presentation. Ask yourself which email you’d be more likely to open: One with a subject line of “Posted: Redemption Research” vs. “The Life of a Groupon: Eliminating Barriers to Redemption and Increasing Engagement”?

6. The end is only the beginning.

The research presentation is really the beginning of the journey rather than the end. You want to make it easy for your team to track your insights and take action to address them.

I’ve always been amazed at how dependent product managers and engineers are on tracking tools. I recall a time when I said “The one thing you need to remember is to do X” … to which they replied, “Ok, well then submit a ticket for that.”

I thought to myself, “It’s only one issue,” but now I realize that PMs and engineers are constantly flooded with problems to fix and it can be difficult for them to separate small issues from big ones. They’re all bugs—just with different levels of priority and required effort.

I’ve found that a shared Google spreadsheet is an effective way to track such issues. Each issue or insight is given a row in the sheet and is assigned an owner who is responsible for the status of the issue and next steps. That said, work with whatever tools your team uses. The last thing most product managers and engineers want to do is learn how to use another tool.

Another important consideration is to ensure that you have a clear path forward right from the outset for ensuring your research has impact. Establish a plan with the product team before you conduct research that makes explicit how and when product improvements will be made based on the insights. You may find that product teams are enthusiastic about research, only to discover after completing the research that they had not thought about how they would act on the findings. Prior to kicking off the research, establish a timeframe for when you’ll present your findings, when changes will be made, and who will be responsible for making them.

7. Know your allies…and how to find new ones.

Your immediate product team is obviously your best bet for addressing the issues surfaced in your research. That said, product organizations typically have many interdependencies, so there are probably other product teams who have some stake in what happens to your product.

Reach out to those product teams to share your research and determine if your research can be reframed to be more directly relevant to them. For example, one team may be focused on new user acquisition whereas another team may be focused on increasing engagement. Some of your findings may apply to both goals, so reframing a few key points can open up new audiences (and doors) for your insights.

To make this possible, know the product leaders in your organization, who they are, what they do, and what their goals are. You may find that they value your research and might even be willing to evangelize it across the organization.

8. Don’t give them what they want.

Teams often don’t know what they need, so they may say they want you to conduct a specific research methodology or present the data in a specific way.

Rather than reactively giving them what they want, think about what they need. Echo back your interpretation of what they’re asking for or what they’re trying to accomplish. For example, “So what I’m hearing you say is that you want to investigate whether users understand how to filter search results and whether we’re offering the right set of filters. Is that correct?” This will enable you to focus on the right methodologies to get the answers they need.

While your initial resistance to carrying out their demands may create friction in the short-term, it will establish your credibility and influence with the team in the long-term.

9. It’s not about the VP.

A researcher working at another company once contacted me looking for advice. She had recently completed a research project but she was having difficulty getting the product team to take action. She asked me for any tips about how she could get the research in front of the vice president leading the product area. As she put it, “If we can just get it in front of the VP, things will happen.”

In most corporate environments, executives don’t want to hear about problems; they want choices. It is up to the researcher to convince the PM and cross-functional team of the issues at hand and lead the charge to create potential solutions. If there are alternative solutions that the team cannot agree on, then the VP can make that decision.

If the cross-functional team is not interested in working with you to create solutions, then you need to re-evaluate if the problem you are trying to solve warrants team resources relative to other team priorities and/or if you have clearly and convincingly presented the problem.

Does the team even agree that it is a problem? Have you leveraged other data sources to make a more convincing argument?

10. Be your brand.

Your work is your brand. You want audiences to know that when you conduct research it will be high quality, and that when you present, your presentations will be engaging. As one PM told me recently, “You put a lot of love into that presentation.” He wasn’t talking about fancy graphics or illustrations but rather that I had clearly given a lot of thought as to the important insights and how I should best present them so that they were clear, concise, and well-substantiated.

Of course, there are situations where you cannot make a masterpiece of a presentation, and you shouldn’t always try. But remember that your brand stays with you, even when you leave your current company.

Beyond the quality of your work, you want to be thought of as someone who helps product teams progress toward their goals. You are someone who is there to help the team succeed, not slow them down, and you’re mindful of the teams’ constraints, deadlines, and challenges. You’re not just there to point out problems but to create and drive solutions.

Conclusion

Turning research insights into positive action is a combination of what you do but also what you are able to empower others to do. Knowing your audience and bringing the right mindset to the table can go a long way to making an impact in your organization.

When Words Are Not Enough

The frequently-raised objection when it comes to quality research, UX research included, is that the conclusions are drawn based on the participants’ declarations. However, there exist some methods which allow one to grasp the real behaviors of participants, and they can be easily implemented into the research scenario.

During exploratory research, the respondents are often unable to articulate their needs or opinions. In turn, when it comes to usability tests or satisfaction surveys, it very often happens that the respondents’ answers are limited to vague opinions which, without being further explored by the moderator, don’t bring in much data.

Very often, they hide their opinions, because something is “not quite right” to say, something makes them feel ashamed, or their behaviors are controlled by mechanisms which they don’t even perceive—because who would admit to having certain prejudices or not fully socially-accepted desires?

Then how does one find out the real opinions of respondents?

One method is observation, which is a fundamental element of every research. However, observation brings us limited information, and without other methods such as an interview, based on declarations it’s difficult to obtain a complete picture of the situation.

The other way is to use psychoanalytic techniques, such as projection, during the marketing survey.

From Freud to marketing

According to the classic concept of Freud’s psychoanalysis, people have a natural tendency to protect their ego. It happens through an inside “censor”—the “super-ego”—which is a part of person’s personality structure and filters all instincts and desires while allowing in our consciousness only those which are socially acceptable. That is the reason why, in many situations, people do not realize their basic attitudes, motivations, or desires. However, it turns out that people are able to assign them to others. That is called projection.

Projection is used most often in the marketing surveys, such as in research about a brand’s image. Usually, the projection methods consist of describing brands as people or objects. The respondent might be also asked to assign people with certain traits to a brand. For example, one might be asked to complete a sentence: “People with traits such as ___ drink X brand beer.”

How to describe modernity?

A current Studio EDISONDA project is a sales platform for a client working in the tourism industry. We’ll call them Green Roller Coaster. During the exploratory research, I used one of the projection methods to deepen the respondents’ answers.

In the initial stage of design for Green Roller Coaster’s new web site, we wanted to find out what kind of image our client has among business partners. During the in-depth interviews, we asked how Green Roller Coaster’s business partners would rate their cooperation with company.

Most of the answers were positive: The respondents praised their relationship with the Green Roller Coaster. The surveyed partners were also asked to describe Green Roller Coaster as a person, a personalisation technique. The company was most often described as a nice and reliable but also as quite old, old-fashioned, not progressive, or even conservative.

This example illustrates some of the many benefits that accrue from the use of projection techniques. On the declarative level, the company was rated neutrally or positively, probably in accordance with the real opinions of the respondents. The interesting thing is that nobody mentioned that they view the company as not really modern. Only after using the projection technique could we obtain the complete image.

In UX research, projection methods might bring a range of valuable information which will be useful while designing the project, such as information about colors, shapes, or words associated with a certain brand, situation, or impulse. Thanks to that, we can utilize the potential hidden in the users, and they can actually participate in the design we are working on.

My dream university

Not too long ago, we worked on redesigning Adam Mickiewicz University’s web site. To get to know the environment better—its problems and the needs of different groups of recipients—we used exploratory research.

We conducted focus interviews (group interviews) with research workers, administration employees, doctoral candidates, students, and student applicants. Additionally, we conducted several in-depth interviews and computer-assisted web interviews (such as online surveys) with the Adam Mickiewicz University’s students.

During focus interviews with applicants and first year students, apart from the standard scenario, we also used the collage technique. The surveyed group was asked to create a collage titled “My dream university.”

On the declarative level, the respondents talked mainly about the didactic offer, the university as a preparation for a future job, the university’s infrastructure, and international cooperation. Of course, these things were also reflected in the collages prepared by the surveyed group, but what really stood out were values which were never mentioned during the group interview.

The collage works contained many themes alluding to having fun, fine dining, sex, sport, and slogans mentioning freedom, openness, and lack of restraints. When creating a web site which aimed to convince the applicants to choose Adam Mickiewicz University, such knowledge was invaluable for the designer.

What convinces young people is not only an attractive way of presenting one’s didactic offer but also showing what will they be able to experience at the university aside from the classes. Collages not only helped us to obtain very valuable data, but they were also an inspiration for graphic designers and copywriters, and they showed how to effectively communicate with the applicants.

Projection as a supplement

The projection techniques allow one to look “deeper” into the respondents’ minds and in many cases let one obtain valuable information which wouldn’t be otherwise easily found with classic survey methods.

It has to be kept in mind, though, that the results obtained from the projection techniques are often ambiguous and usually difficult to interpret correctly without a wider context. Interpretation of the data is based mainly on intuition and experience of the researcher, not on specific criteria.

It also sometimes happens that projective methods do not bring any valuable information, such as when it is difficult to observe any trend in the results. For instance, if students have created collages titled “My dream university” and we could not see any factor in common, a projection as an independent method does not bring valuable results.

Because of this ambiguity, projection techniques should be treated more like a valuable supplement method than the main way of obtaining data. However, in conjunction with other, more established techniques, projection often provides key marketing information which we would not be able to reach based only on declarations or observation. Projection methods are based on natural human psychological reactions to a stimulus and in combination with other methods often have a full picture of the situation and help in interpreting the other test results.

 

User Research With Small Business Owners: Best Practices and Considerations

The majority of our work at Google has involved conducting user research with small business owners: the small guys that are typically defined by governmental organizations as having 100 or fewer employees, and that make up the majority of businesses worldwide.

Given the many hurdles small businesses face, designing tools and services to help them succeed has been an immensely rewarding experience. That said, the experience has brought a long list of challenges, including those that come with small business owners being constantly on-call and strapped for time; when it comes to user research, the common response from small business owners and employees is, “Ain’t nobody got time for that!”

To help you overcome common challenges we’ve faced, here are a few tips for conducting successful qualitative user research studies with small businesses.

Recruiting

Recruiting tip #1: Give yourself an extra week, and then some

It generally takes more time to recruit for research projects with small businesses than what’s typical for consumer studies. There are several reasons why this is the case.

Existing user research participant pools tend to be light on small business representation, meaning recruiting for your project may have to start completely from scratch. Also, it can take time to track down the appropriate person at a small business to talk to—are you trying to reach the owner, the accountant, customer service staff, or…?  

Finally, small businesses are accustomed to companies trying to sell them new offerings or get them to sign up for product pilots, and many have been scammed in signing up for “free” pilots or services that end up turning into a perpetual sales pitch. Because of this, the chances of a small business owner or employee saying “No!” to participating in your user research is especially high.

Recruiting tip #2: Make sure you’re crystal clear on what type of business you want to recruit

There’s quite a bit of variation in terms of business environment, priorities, strategies and other factors across different types of businesses. Accidentally overlooking important criteria could be detrimental to a study.

For example, do you want to talk to a certain type of business, such as professional service, service area, or brick-and-mortar? Does it matter if your study participants are from B2B vs. B2C companies? What about online vs. offline businesses? Additional points of consideration include number of employees, business goals (e.g., does the business want to grow?), and revenue.

If you’re not sure if you’ve overlooked important criteria, ask for feedback from product managers, marketing professionals, and other user researchers who may have relevant information. It can also be helpful to see how entities such as the Small Business Administration categorize business types.

Recruiting tip #3: Make sure you’re crystal clear on whom you want to interview

When conducting research with small business owners, it’s common to assume that the business owner is involved in most decisions, but that’s often not the case.

Is it actually the business owner you’re interested in speaking to? Or, do you need to talk to someone who’s responsible for a specific task, such as someone who managing online marketing decisions or handles the company’s financials? The larger the company, the higher the chances are of the company owner delegating responsibilities.

We typically ensure we’re speaking to the right type person by asking screening questions specific to roles and responsibilities (see examples at the end of this article).

Recruiting tip #4: Avoid hobbyists disguised as business owners

It’s common for hobbyists—for example, people who casually sell certain services or offerings for personal enjoyment—to sign up for user research involving small business owners. On the surface they pass many of the screening criteria, but in reality their motivations and behaviors are quite different from a full-time business owner or employee of a business. We typically screen out hobbyists via recruiting screening surveys by asking if potential study participants spend at least 30 hours per week in their role as business owner or employee of the business.

Recruiting tip #5: Recruit extra participants

When conducting research with consumers, we always recruit one extra participant in the event there is a no-show. When conducting research with small businesses, we’ll increase that number to two or more. Given how unpredictable the small business environment  can be, we’ve found that the chances of last-minute cancellations or no-shows is much higher with small business owners and employees than it is with consumers.

Incentives

Incentive tip #1: Provide incentives other than cash

While incentives are a nice gesture, cash or gift cards are not a huge motivator, as they aren’t viewed as a worthwhile tradeoff for inconveniences that come with stepping away from running a business in order to participate in an interview.

What is motivating is providing small business owners and their employees with information and tips on how to run the business successfully: things like offering free accounting software, coaching on social media best practices, and personal access to a member of the support team for assistance. Another approach is to offer 15 minutes after the interview for free coaching and/or advice on a topic that makes sense given the study focus.

The small business community is tightly knit, and small businesses are often invested in each other’s success. Because of this, another option is to frame the study as an opportunity to improve offerings for all small businesses.

Even better, small businesses owners and employees love the opportunity to share feedback on tools and services they routinely use to run their business. If the product you’re testing or exploring touches upon tools and services already in use, it can be motivating to frame user interviews as an opportunity to shape the future of the offering being reviewed.

Finally, consider offering small business owners and employees the opportunity to participate in an exclusive Trusted Testers community, which provides the option to share feedback, receive “insiders” information and tips, and interact with and learn from other small businesses. We’ve found this option can be especially motivating for engaging in user research.

Interviewing

Interview tip #1: Consider in-person interviews

It can be hard for small business owners and employees to take time away from the business to participate in research that might be conducted at your lab or office. Likewise, for remote interviews, small business owners and employees don’t always have convenient access to needed technology at their place of business.

For these reasons, we’ve found that small businesses are much more likely to participate in user research if interviews take place at their place of business. This way they can tend to the business during interviews if needed and don’t have to waste valuable time setting up technology to participate in the interviews.

Also, conducting in-person interviews provides context often needed to understand complicated processes and workflows that business owners and their employees face.

Interview tip #2: Be flexible with scheduling

We’re also always especially flexible with scheduling when conducting research with small business owners and employees. In addition to leaving extra time between interviews, we usually also leave an interview slot open in the event we have to move the schedule around suddenly. We’re also mindful of offering early morning or late evening interview times, especially if the verticals we’re focused are service oriented (restaurants, spas, etc); trying to conduct a field visit during peak hours can be really intrusive for these types of businesses.

Interview tip #3: Be prepared for last-minute changes

The world of small business owners and their employees can be unpredictable, which is why we always schedule extra, backup participants for research. We’ve run into countless situations where a research participant cancels an interview at the last second on account of unexpected business or emergencies.

It’s also common for small business owners or employees to request location changes at the last second. For example, one time I (Chelsey) was scheduled to interview a business owner at his home (which is where he ran the business). He called five minutes before the interview explaining he wanted to be respectful of his roommates and asking if we could meet elsewhere. Good thing I had scoped out the area before this happened and had a nearby coffee shop in mind where we could talk!

Interview tip #4: Emphasize participant expertise early

When interviewing small business owners and employees, it’s common for them to want to seize the opportunity to get insider information or training on whatever topic is being explored. When this happens at the start of an interview, the interviewer becomes the expert for the remainder of the conversation which can prevent an open, honest dialogue.

To establish the participant as the expert early on in the conversation, there are a few things we’ll typically do. For starters, we always state that the goal of the study is to learn from the participant.

Next, we’ll ask the participant to give a tour of the business (if a site visit) and to explain what the day-to-day looks like in running it. During the tour and/or day-to-day explanation, we’ll call out pieces of information that are new to us and ask a few follow-up questions. This strategy usually does the trick in placing the research participant in expert mode and researchers as the student.

Interview tip #5: Bring extra NDAs  

I (Chelsey) will never forget, in kicking off an interview with a business owner in India, when I unexpectedly discovered several family members waiting to enthusiastically participate in the conversation!

The reality is that running a small business—whether in India, the US, or elsewhere—is rarely a solo operation. Consequently, we’ve found it’s common for family, friends and employees to be asked by interviewees to join interviews. This isn’t a bad thing. In fact, in many situations it’s a wonderful surprise that can lead a an engaging, insightful conversation.

Because of how frequent this scenario can be, we now always make sure to bring extra copies of NDAs.

Reporting

Reporting tip #1: Provide context

When socializing your findings to a product team, building empathy for the participants and their challenges is key. Reporting on consumer insights is relatively easy because most of us face similar challenges in our daily lives and we can easily identify with the participants. However, small business owners and employees face challenges that are less relatable.

Therefore, it’s important to create a narrative that includes the context of the merchant’s business and business practices. For example, what vertical do they work in? What does their day to day look like? How has their business evolved? What do they feel their customers’ needs are, and how does that in turn translate to their use of your product?

Additionally, keep in mind that your product won’t exist in a vacuum. Small business merchants are experimental, and are willing to try out numerous tools and services until they find one that meets most (usually not all) of their needs. Small business owners also value integration and may find creative ways to DIY integration that doesn’t already exist. It’s therefore not unusual that small business research may occasionally graze the edges of competitive analysis.

When crafting your report, create a story around the participants — what are their challenges and successes? How do they feel about their customers? How does (or could) your product fit into their business processes? Finally, video recordings and direct quotes are highly impactful and help emphasize the person behind the findings.

Reporting tip #2: Limit, but embrace, variation

Because small businesses are so varied in terms of vertical, structure, and practices, it takes a careful eye to draw unified or cohesive themes across what can sometimes seem like disparate participants.

Often in user research there is an impulse to sweep outliers under the rug. However, in small business research it can actually be helpful to call out and explain the outliers. They may represent an edge case that your team has an opportunity to address, or they might reveal something new about a vertical, business, or merchant type.  

Of course, as we mentioned earlier, it’s important to clearly define your intended participant group. Even with a clearly definition of who you want to talk to, you can expect to see a healthy amount of variation among your study participants.

Concluding thoughts

Small businesses have different pressures and motivations than consumers that are important to consider in setting up a successful user study with business owners and those who help run small businesses. To get the most out of your time and theirs, study up on what might relieve these pressures and speak to motivations, and adjust your recruiting, incentives, and interview techniques accordingly.

Sample screening questions

Which of the following best describes the business where you work? Please select one.

Food and dining (e.g., restaurant, bar, food truck, grocery store) 1
Retail and shopping (e.g., clothing boutique, online merchandise store) 2
Beauty and fitness (e.g., nail salon, gym, hair salon, spa) 3
Medical and health (e.g., doctor, dentist, massage therapist, counselor) 4
Travel and lodging (e.g., hotel, travel agency, taxi, gas station) 5
Consulting services (e.g., management consulting, business strategy) 6
Legal services (e.g., lawyer, paralegal, bail bondsman) 7
Home services and construction (e.g, contractor, HVAC, plumber, cleaning services) 8
Finance and banking (e.g., accounting, insurance, financial planner, investor, banker) 9
Education (e.g., tutoring, music lessons, public or private school, daycare, university) 10
Entertainment (e.g., movie theatre, sports venue, comedy club, bowling alley) 11
Art / design (e.g., art dealers, antique restoration, photographer) 13
Automotive services (e.g., auto repairs, car sales) 14
Marketing services (e.g., advertising, marketing, journalism, PR) 15
Other 16

Thinking about the next 12 months, which of the following are overall goals for the business you own or work for? Select all that apply.

Acquire new customers 1
Conduct more business with existing customers 2
Target specific customer segments 3
Improve operational efficiency/capabilities 4
Expand to more locations 5
Develop new products/services 6
Offer training or development for my employees 7
Invest in improvements to physical locations (e.g., new paint, interior remodeling, etc.) 8
Maintain current business performance 9
Acquire competitors 10
Other 11
None of the above 99

 

How does your business operate? Please select all that apply.

You have a physical business location that customers visit (e.g., store, salon, restaurant, hotel, doctor’s office etc.) 1
Your business serves customers at their locations (e.g., taxi driver, realtor, locksmith, wedding photographer, plumber) 2
Your customers can purchase products and services from any location, online or by phone 3
Other 4

Which of the following best describes your role in your business?

Owner 1
Employee 2
Other 3

Which of the following are you responsible for at the business you own or work for? Please

select all that apply.

Hiring employees 1
Managing employees 2
Business planning/Strategy 3
Marketing/Promotions 4
Finance/Accounting 5
Sales/Customer Service 6
Legal 7
IT 8
Other 9
None of the above   99

Which of the following best describes your current employment status? Please one.

Work full-time (30 or more hours per week) 1
Work part-time (fewer than 30 hours per week) 2
Not employed 4
Student 5
Retired 6