Guerrilla Usability at Conferences

Does your company have display booths at trade shows and conferences? Typically, these are marketing-dominated efforts, but if you make the case to travel, working the booth can be used for user research. Here’s how I’ve done it.

Positioning and justification

At times it can be a hard internal sell to justify the costs and diversions to take your one- or two-person show on the road, all the while piggybacking off of another department’s efforts. Yet, standing on your feet for 12 hours a day doubles as a high-intensity, ‘product booth-camp.’ Say what you will about sales folk, but they are well trained on knowing how to (or finding someone who can) answer any question that comes their way. As an in-house UX professional, the more I can technically understand about our SaaS product, the more context I can have about our user’s needs.

I’ve found that having prospective customers participate in a usability session is a great way to show that we were taking the time to invest in them and their opinions of the product. As a result, there have been specific features that have been rolled into our application during the next sprint, which were proposed as small sound bites of feedback during these sessions. It shows we were listening, and makes a great justification for a follow-up phone call.

Recruiting and screening

To recruit, I scan Twitter to find those who tweet that they are excited about attending the upcoming conference. I cross-reference the Twitter handles to the names in LinkedIn to see if, based on job title and industry, they would be good participants.

I reach out to them to see if they’d be willing to sign up for a slot, proposing times between presentation sessions or before/after lunch to not conflict with their conference attendance.

Because the expo halls are generally open the entire day, even if there is no one booked on the calendar in specific spots, I also grab people just milling about to keep the sessions going. If you do this, be sure to quickly do a visual scan of their badge, as you can get a good sense of what they do and what knowledge they might have by where they work.

Booking

For the time bookings, I find that Calendly.com is a flexible, free, user-friendly way to book time slots with random people, using just a URL with no account sign-ups needed. In addition to custom time buckets (18 minutes, anyone?), Calendly also provides the option of a buffer increment after every session, so I can take notes and regroup.

Calendly does a good job of reminding participants when to show up and how find me–all the important things, including integrating well with all the major calendaring applications.

Come conference time, I have a slate of appointments along with contact information and reminders when they were coming. Couldn’t be easier. If expo hall hours change, I can easily message participants to let them know of the reschedule.

Duration

In a normal, controlled setting, I would typically want to go a full hour with a participant to properly delve into the subject matter and go through a number of different tasks and scenarios. “Pick a few and grade on a curve,” as Neilsen once said.

However, with the participant’s attention scattered given the sensory overload of the conference floor, anything more than 20 minutes gets to feel too long. At conferences, you’re going for quantity over quality. An advantage to this staccato method is when you find a vein of usability that you want to continue to explore in further depth and detail, there’s likely another participant right around the corner (either scheduled or random) to confirm or refute that notion.

Script and tone

The main challenge of this technique is that you’re not supposed to ‘sell’ in the role of testing moderator but rather to guide and respond. I wear many hats when working a booth; when not conducting these sessions, I sell the product alongside marketing.

As a result, 90% of the conversations in the booth are indeed sales, and switching roles so quickly is sometimes hard. I try to check myself when the testing script bleeds into ‘did you know that there are these features…’, because after 3+ days and what feels like a thousand conversations, I tend to put my conversations on a programmed sales loop, letting my brain rest a bit by going off of a script.

A pre-written task list helps keep me on point as a moderator. However, with the variety in participant group, I use the script much more as a guide than a mandate.

As with any usability session, I let the participants veer into whatever area of the app interest them the most and try to bring them back to the main road ever so subtly. With so many participants in such a short period of time, sometimes these unintended diversions became part of the next participant’s testing script, as it is easy to quickly validate or refute any prior assumptions.

Tools

Following the ‘guerrilla gorilla’ theme of this article, I use Silverback for my recording sessions. Silverback is a lightweight UX research tool that is low cost and works very well.

At one event, without my Bluetooth remote to use Silverback’s built-in marker/highlights, I paired an iPhone with an app called HippoRemote. Meant initially to provide ‘layback’ DVR/TV functionality, Hippo can also be written with custom macros to allow you to develop third-party libraries.

In the case of integrating with Silverback, this meant Hippo marked the start of new tasks, highlights of sound bytes, and starting/stopping recording–all the things that the Apple Remote should have done natively.

Despite some of the challenges in peripherals, Silverback is absolutely the right tool for the job. It’s lightweight, organized, and marks tasks and highlights efficiently.

I recommend a clip-on microphone or directional mic given the background noise from the conference floor. Any kind of isolation that you can do for the participant’s voice will save you time in the long run, because you won’t have to try to scrub the audio in post-processing. Moving the sessions to somewhere quiet is a hard proposition, as the center of activity is where the impromptu recruitment tends to occur.

Wi-Fi

As a data-intensive SaaS product, the biggest challenge comes when trying to use the conference wi-fi. With the attendees swamping access points, there is no guarantee that I can pair the testing laptop and the iPhone used for marking, because they both need to be on the same network router for integration with with Silverback.

An ad-hoc network for the Mac won’t work, because I still need web access to use the application. Using my mobile phone as an access point has bandwidth constraints, and choppy downloads are not a good reflection on the speed of our application.

Unfortunately, then, every session begins with an apology on how slow the application is performing due to the shared conference wi-fi. A high-speed, private access point or a hardline into your booth cures all of these issues and would be worth the temporary investment for sales demonstrations and usability sessions alike.

Summary

There are a few adaptations we, as usability professionals, have to make from a traditional sit-down, two-sided-glass setting. Conference booth testing is a much more informal process, with an emphasis on improvisation and repetition. Some of the tools and methods used in guerilla testing certainly are not as proven or stable, but the potential recruitment numbers outweighs the inconveniences of a non-controlled setting.

From an educational standpoint, being inside the booth for days at a time will raise your knowledge-level considerably. You’ll hear again and again the type of questions and responsive dialog that prospective customers have around the product, and you’ll start to recognize the pain points coming from the industry.

After a half-dozen conferences, you’ll start to understand the differences in the average participant. In the case of the technology-centric attendees, some conferences provide a recruitment base of high-level generalists, with others being much executionally closer to the ground and detail-oriented. I tend to tailor my scripts accordingly, focusing on principles and concepts with the generalists, and accomplishment of specific tasks with the more programmatic participant.

One good thing about working for Loggly o’er here in the startup world is the ability to create paths and practices where there were none before. Pairing with the marketing team, using a portion of the presentation table to recruit participants off the expo hall floor, and sitting them down for a quick walkthrough of the product is a great way to become inspired about what you do and who you’re working for. As someone who still gets excited to travel, meet new people, and play off crowds, these sessions are always a highlight for me to conduct guerilla usability in front of my customers, peers, and my co-workers.