Buying new clothes and looking at current fashions is usually much more interesting and exciting than digging through one’s closet or laundry hamper. However, there is a lot one can learn by stopping and taking a minute to examine one’s own clothes. Such a review can tell you history (past fashions), document what you’ve been doing (attending a ball game, as evidenced by that ketchup stain), and indicate what you need (oh, yeah, I need a shirt to go with those pants). As user experience professionals it is crucial to your success that you learn who your users are, understand their environments and business needs, determine current user interface (UI) problems with the products, and review design solutions with users.
Like most of you, I can’t wait to get into the field to observe users working with my products. Yet, I’m going to pull on the reins, and do a 180-degree turn. Don’t worry – it’s still research, but from an internal perspective. There is a wealth of information at your fingertips in your own office, and surprisingly, some of it is usability-related. You can optimize your internal resources by understanding where and how you can find UI information about your users within your own company. To provide context and practical guidelines, this article presents examples of how to mine internal resources at a large enterprise software company.
Beginning with the end in mind
Successful research initiatives begin by first considering the research questions that need to be answered, then by selecting sources and methods to obtain those answers. Many user experience (UE) questions can be answered by examining internal resources. These questions may include:
- Who are the product users? What types of industries do they represent?
Although these might sound like marketing questions, they are important for UE professionals to know as well. Defining the user base is key for research purposes, as well as for understanding the usability patterns emerging in different markets. One would expect that a hospital, for example, might use purchasing software differently than a manufacturing company. - Are usability issues already being documented within your organization?
Before running numerous usability studies and visiting customer sites, first find out what customers are already reporting. These reports can be a low-cost way to collect and summarize feedback and can direct your future research initiatives. - Where is the UI becoming an issue in your organization’s business? What are the usability hotspots?
You can ask questions to determine whether the UI is affecting:- Revenue-generating sources: new sales, upgrades, and competitive pressures
- Customer relations: implementation, training, documentation, and help desk calls
Assessing internal documentation can help the UE team react to some of these problems and make the appropriate changes before problems escalate.
Internal resources
Most companies–small, medium, and large–maintain their product, sales, and customer knowledge within some sort of information documentation system. This knowledge organization can range from Excel spreadsheets to multi-database knowledge management software solutions such as Lotus Notes (Zorn & Taylor, 2004). Every company categorizes and prioritizes information differently. Likewise, not all companies have the same resources written about in this article and many have additional sources not addressed here. At a minimum, this diversity provides directions for exploration.
PeopleSoft’s UE team has found it valuable to look at the following sources:
- Customer Database
An application that supports all customer-facing operations. This application feeds into a database that contains all customer information, including customer contacts, business details, market details, products implemented, and licenses sold. - Support Call Center
Most companies with a customer call center have a tracking system to log the issues reported by customers. Many log issues into categories, such as “usability.” - Consultancy and Implementation Services
Consultants have day-to-day experience with product implementation and customization. They document customizations made to the UI based on customer needs, such as accommodating changes to infrastructure or business processes. - Product Managers
These individuals have ongoing dialogues with customers that influence the future direction of the product. Product Managers can alert UE teams to design issues, such as a major redesign needed due to newly added functionality. - Company Survey Data
Most mid- to large-scale companies evaluate their customers’ satisfaction with their products. Usability may be one of the dimensions surveyed.
Examining and synthesizing information from your own corporate resources will allow you to have a deeper understanding of customers and users prior to conducting field and usability research, incorporate usability findings into the product release cycle phases, and leverage internal relationships for future research and design reviews.
When to assess your internal resources?
Assessing internal resources is a continuous task. Information is constantly being updated, in many cases daily. Reevaluation of data may occur quarterly, yearly, and/or at salient times during the product release schedule, such as the product planning phase.
Another key time to investigate internal resources is when you start a new job. This is a great time to understand the knowledge management tools and information collected within your organization. Identifying your key contacts and information sources as early as possible helps you be more productive, efficient, and better at decision-making. Questions you may ask yourself include: who does what, when, and how? Who has liaisons with the customers? Which interest groups have an impact on the product designs? How is customer feedback typically handled?
Findings first! What types of usability results can you expect to find by doing internal research? Table 1 below provides a description of each resource, denotes the format, and identifies key usability issues that can be uncovered.
|
Group
|
Description
|
Data Format
|
Types of UI Issues
|
|
Customer Database
|
Database with customer contact info, company stats,
product and license details, etc. |
Database
|
None
|
|
Call Support Center
|
Customers call product support staff to report specific
application-related problems. |
Database
|
Labeling issues. Error messages incorrect or ambiguous. Need to change displayed defaults. Interaction difficulties. Not able to complete task due to process. |
|
Product Managers
|
Customer-facing liaisons who provide guidance, feedback,
and strategic direction into the product. |
Interpersonal communication
|
Customer-specific feedback. Customer hotspots. Feature- and functionality-specific issues. |
|
Field Consultants
|
Consultants determine business requirements and set
up and implement products at customer location. Customizations are made on the basis of needs. |
Interpersonal communication
|
Inconsistency within products, across modules. Mismatch of terminology with real world terms. Documentation not enough, need more clarification. Technical response times too slow, batch processes run at inappropriate times. Interaction design not portrayed as customer expects or needs. |
| Corporate Surveys | Customers are asked to fill out surveys regarding products, satisfaction, and company loyalty. |
Database |
Overall sense of usability satisfaction with products. Generally, high-level information. |
Table 1: Usability issues uncovered by internal resources
Mining customer data
The role of market research in the user-centered design (UCD) process (Norman & Draper, 1980; Norman, 1988) is not explicit. Usability professionals need to be responsible for the data they collect and need to understand how their customer base is distributed to ensure that key industry customer types will be properly represented in their future research. Analyzing your customer databases, therefore, should be the first step-before running usability studies, before field research, and before design-to truly understand who your current customers are. Do not rely solely on others for this information, as most departments have their own competing priorities, but verify the information for your team.
The process of accessing customer databases may take more time than you initially expect. The raw data might be classified as company-sensitive information and, therefore, may have usage restrictions. Usability teams are not always given access to customer databases and you may have to work with the marketing or sales departments to gain data access. Furthermore, you may have to take classes to learn how to use the data application, how to run queries (commands for pulling the data you want), and understand how the data is entered into the system (how columns and rows are defined).
As you gain access to the system, you will want to cultivate relationships with key people who work with the datasets. We recommend identifying:
- The initial gatekeeper who can help you gain entry to the system. In our case, we needed buy-in from an expert user of the system to request that we be given entry-level access.
- A database expert who understands the source and classification of the data. We found that differing database queries led to result sets that had to be reconciled. (Either the queries were not matching up or the polled datasets were different.) Experts can answer questions about the data structure, queries, and problems you may run into while using the system.
- Key product experts who can verify the synthesized data that you produce. Once you identify the percentages of key industries per product, you need product managers to verify that the numbers actually reflect their products.
In the software domain, customer data can be classified by a number of dimensions. If our research is to inform our UCD initiatives, it is important to consider customers who currently use our software and to identify categories from which we can easily recruit.
To begin, from your larger database create a sample of customers for which you are interested in gathering more detailed facts. For example, our UE team initially ran a query that selected customers based on: 1) product type (e.g., inventory customers); 2) customer status (e.g. actual license holder vs. prospective customer) ; and 3) status of product usage: must have implemented the product and be a current user. How your data is classified will influence how you can filter it.
Caption: Key industries for Product XYZ.
Once the sample is defined, you can run crosstabs by key variables you would like to learn more about. In our UE group, we really wanted to know which industries were purchasing certain products. We created pie charts and lists of key customers to reflect the core industries for each product type (e.g., purchasing software, inventory software). This segmentation was important for three key reasons. First, we wanted to be able to identify various user roles within an industry. To do this, we needed to know the core industries for any given product. In our case, the core industries for Product XYZ are: healthcare (25 percent), education (15 percent), utilities (20 percent), and financial services (40 percent). Second, the industries can define the type of user for whom we are developing our software. For example, is the user base primarily expert or more self-service?
Having a picture of those for whom we are designing at an industry level helps us then select users who are more representative of our user group for additional research. This leads us to the third reason: we can use those top industries as the first places we recruit users to create user profiles and then abstracted personas. Our long-term UCD goals were to create personas for our design and development teams. Understanding the industries where our users work was the first step of this research agenda.
One side note: future sales projections or targeted markets should also be considered when interviewing individuals for user profiles. Projected information cannot be collected from the customer database but needs to be gathered from talking with product managers, sales, and marketing. This information will indicate new user groups to consider and possible changes to the distribution of potential and current users.
Advantages, disadvantages, and considerations of the data
Now what? This customer demographic and sales information alone will not increase the usability of your product. It is, however, a step toward understanding your users. This deeper understanding will inform both the recruiting for usability testing and the process of designing the product itself.
There are two challenges that you should be aware of during these database analyses:
- Customer versus User dilemma
From a usability standpoint, we design for the user (the person using the application on a regular basis), but the data collected is based on the customer (the management or business analysts who make application selections and purchases). Does this fact then make the information less useful? No, it is still useful to know the industries that we are selling to. However, we recommend ensuring that subsequent research focuses on the user and that you understand their use of and perspective on the software. - Data validity
Salespeople often enter customer information. As with any human-managed process, there is bound to be some error involved. This can be through mis-keying of data, not collecting and reporting on all key questions/fields, or through mis-categorizing or misunderstanding what the customer was saying. In addition to how data is entered, there is also the concern of how current the data is. Realize that there is some percentage of outdated information in these databases. Always validate data with the product manager and the person who entered the data.
After identifying the core industries your products serve, you will want to understand the types of user roles within each main industry, which most likely will require additional research, such as interviews with customers from those industries. From there, you’ll want to make sure that you recruit an appropriate sampling of users from the key industries and identify user roles to truly understand how each user group and/or industry is using your application.
Conclusion
The User Centered Design process (Norman & Draper, 1980) is well-accepted, endorsed, and integrated into many software corporations. Users provide context, feedback, and validation to designs using a spectrum of methodologies ranging from contextual inquiries (Holtzblatt & Beyer, 1993) to usability benchmark studies. To do this type of research and design can take considerable time, resources, and financial investment. One of the most overlooked areas to inform user interface considerations is within our own companies. Corporate resources are seriously undervalued and underused, but are at our fingertips. As a first step, we should determine where usability issues are already being documented within our organizations and where they could be used to inform our design decisions.
This article is the first of a two-part series that recommends five internal resources that should be assessed. In this article, I examined customer databases. In part 2 of this series, I will consider the following internal resources: product managers, call support centers, field consultants, and corporate surveys.
Mining customer databases is an essential first step to really identifying who user experience professionals are designing for. The information is at a high level, but extremely valuable when determining who to recruit and where to focus additional research efforts. Furthermore, this information is a level of data most other stakeholders in the company (such as marketing, sales, and strategy) can understand. It also helps in starting dialogues with these other working teams. UCD teams should take responsibility for performing, or at a minimum overseeing these analyses, to ensure accurate results. In sum, begin with the end in mind by optimizing your own internal resources.
- English, J., & Rampoldi-Hnilo, L. Remote contextual inquiry: A technique to improve enterprise software, 2004.
- Holtzblatt, K. & Beyer, H. (1993). Making customer-centered design work for teams. Communications of the ACM, 36, 93-103. http://www.incent.com/pubs/customer_des_teams.html
- Norman, D. A., & Draper, S. W. (Eds.). User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1986.
- Spradley, J.P. The ethnographic interview. New York, NY: Holt, Rinehart and Winston, 1979.
- Wood, L. E. Semi-structured interviewing for user centered design. Interactions, March + April 1997, 48 – 61.
- Zorn, T., & Taylor. Knowledge management and/as organizational communication. In D Rourish & O. Hargie, Ed., Key Issues in Organizational Communication. Routledge, in press, 2004.
John Stickley (Illustrator) is an Information Designer with over 10 years experience translating complex systems, concepts, and experiences into accessible visual solutions. His site Visual Vocabulary shows a complete overview of his work.
Excellent ideas!
Great ideas