MindCanvas Review

MindCanvas describes itself as a remote research tool that uses Game-like Elicitation Methods (GEMs) to gather insights about customer’s thoughts and feelings. It was developed by Uzanto Consulting, a web product strategy firm. When I first learned about MindCanvas, I understood it to be an online card sorting tool. Happily, it’s much more than that.

As a veteran IA consultant, I have used MindCanvas a handful of times during the course of different projects. I have also conducted card sorting exercises without the tool. I am thrilled to have a useful—and user-friendly—tool at my disposal. One of my main reasons for selecting MindCanvas was the reputation of one of its creators, Rashmi Sinha. She is well known and respected, and I felt assured that any tool designed by a fellow IA for IAs couldn’t be all that bad. I was right.

MindCanvas provides open and closed card sorting capabilities, as well as a host of other UT tools: Divide-the-Dollar, Clicky, Sticky, Concept Test, and FreeList. Clicky and Sticky allow users to react to a wireframe or prototype by answering questions about images and content, or applying stickies (Post-it–like notes) with attributes to a visual image. FreeList and Divide-the-Dollar allow you to elicit product ideas and prioritize them by having participants list and rank the features they find most useful. All of these methods offer easy-to-use interfaces to help your research participants along.

Deciding which MindCanvas method to use is one of the more complicated parts of the tool. It’s card sorting methods are good for validating a site’s navigation or information hierarchy. You can also explore user needs and values and gather feedback on brand and positioning by using some of its more specialized UT methods. MindCanvas’ website and supporting help wiki provide information on selecting the appropriate testing method for your website or product.

Using MindCanvas

The basic process for using MindCanvas is as follows:

1. After payment, sign an agreement to obtain a login and password.
2. Decide which method (i.e. Sticky, FreeList, etc.) addresses your research needs.
3. Create potential research questions and tasks based on the MindCanvas method you have selected.
   (I’ve used OpenSortand TreeSort).
4. Upload questions to MindCanvas’ Workbench.
5. Test the research study and make changes until you are satisfied with it.
6. Send out the test site URL to your participants.
7. Monitor the study (i.e. see how many people have completed all the tasks).
8. When the study is concluded, send a report request to the MindCanvas team.
9. Receive the reports in visual form and download raw data from the MindCanvas site.
10. Embed reports into PowerPoint or Word document and review results with client.

I usually take several days to review the reports before showing them to my consulting clients. Doing so allows me to more easily explain the results. (Here’s a pointer to anyone using MindCanvas: To view the results properly make sure PowerPoint is in “Slideshow” mode).

Strengths

MindCanvas has a couple shining strengths I’d like to illuminate:

• An engaging, easy-to-use interface for your customers or end users. It’s fairly self-explanatory and makes routine UT tasks fun.
• Stellar data visualization tools once your study is completed.

User interface

MindCanvas’ interface is what sets it apart from other UT software I’ve seen. Its creators took their inspiration from the world of digital gaming to develop an interface that’s engaging for the person using it, while gathering important data for researchers. Its card sorting methods employ a floating hand to deal cards, which are then sorted by users. Another method gives users virtual gold coins to vote for their favorite product features. These exercises are enhanced by accompanying sound effects. I’ve received numerous comments from users describing MindCanvas’ exercises as “fun”. They have also commented that while they don’t understand how these exercises will help me build a better website or software interface, they still enjoyed the tasks and were pleased at the conclusion of the test.

The other online research tools I’ve reviewed offer more awkward interfaces. Sorting exercises take multiple steps or the online tasks are not intuitive and confuse research participants. I’m not interested in making my users become experts at online card sorting or other UT methods. I simply want to extract what they know or understand about a particular website or service.

According to Jess McMullin of nForm User Experience Consulting, “MindCanvas is unmatched as a remote research tool in its ability to provide creative methods for gathering data [and] engaging participants…..”

Data visualization

Another MindCanvas strength is its data output. Although you can obtain the raw data and analyze it yourself (assuming you have statistical software and know how to use it), the real benefit of MindCanvas is its easy-to-understand data visualizations, which showcase the results of your study. All my clients have received clear, easy-to-interpret answers to their research questions. The visualizations can be embedded into a PowerPoint slide or Word document, making them easily accessible. Your clients don’t have to rely on your interpretation of the data; they can interpret the data themselves if they choose. Every client who has viewed MindCanvas’ data visualizations has been impressed and wondered why it wasn’t used all along.

Weaknesses

I’ve used MindCanvas a handful of times and encountered some weaknesses:

• Study size. If you have a large client with complex, statistically rigorous research needs, MindCanvas is not for you. It has a limit of 200 users per study. Two hundred is plenty for most of my research needs, but some of my clients want to go beyond that.
• Data sorting. If you have complex user segmentation needs, MindCanvas has its limitations. It allows you to perform a single data sort to identify user sub-groups. For example, it’s easy to segment all male vs. female participants or all participants who are 21- to 50-years-old. If you need to segment 16- to 20-year-old females or men who only shop online (or any two parameters of your choice), you’ll need a different tool. There are ways around these limitations: You can create two separate research studies to deal with different users, or you can build more complex research questions to solicit the answers you need in order to sort the data required. However, these solutions have limitations of their own, so there is a trade-off.
• Pricing structure. The current pricing structure is $499 per study, with each accompanying report $99. This is adequate for quick-and-dirty research to resolve obvious user issues, but the pricing structure doesn’t scale well. For example, if you run a single study and want multiple reports for different audience segments, each $99 report adds up quickly. It can be difficult to budget up front before the research study is even developed, leaving the door open for cost increases. If a simple card sorting tool is all that you need, check out WebSort, which costs $499 for three months of unlimited use and automatically generates a dendogram. (Please note that MindCanvas offers much more than card sorting).
• Data analysis bottleneck. Some of the back-end data analysis is done by a human, who works on a schedule. All data reports are generated once a week. If you get your report order request to Uzanto by the Tuesday deadline, results will be available by Thursday. This might not work with your tight project schedule, in which case, you’re out of luck.

MindCanvas’s Workbench

MindCanvas is currently offered in self-service mode. This means that you (or your researcher) need to become familiar with the finer points of MindCanvas’ Workbench for constructing studies. The upside is that some parts are made easy, like being able to “copy” another study in order to create your own (a handy feature), or creating as many preliminary studies as you like before distributing the real thing.

The downside is that some interface elements in the study creation console are a bit mysterious. For example, under Manage Study, it’s unclear if the data has been downloaded 164 times or if there are 164 participants who have completed the study. The difference between Manage Study and Track Activity is also hazy. Manage Study allows you to specify where to send users after they have completed the study and limit the number of participants or the length of the study, while Track Activity informs you how many people have completed the study. The Download Tracking CSV gives you access to a text file with a list of all participant’s URL information and their start and stop times.

The Workbench allows access to MindCanvas’ powerful study creation module, but you can tell most of the design effort went into the end user’s interface, not the study designer’s. Luckily, there is a wiki available which answers a lot of questions and Uzanto consultants are very friendly and helpful with the occasional question.

Conclusion

The IA community can finally say that we have a tool designed for us. For so long, we’ve had to take existing tools and try to use them in ways not intended by their designers, sometime with frustrating results and having to develop clever and complicated workarounds. These issues are no longer a problem. It’s a tool for us, made by one of us. It’s about time!