Bringing Holistic Awareness to Your Design

Written by: Joseph Selbie

Gone, thankfully, are the days when the user experience and the user interface were an afterthought in the website design process, to be added on when programming was nearing completion. As our profession has increasingly gained importance, it also become increasingly specialized: information design, user experience design, interaction design, user research, persona development, ethnographic user research, usability testing—the list goes on and on. Increased specialization, however, doesn’t always translate to increased user satisfaction.

My company conducted a best practice study to examine the development practices of in-house teams designing web applications—across multiple industries, in companies large and small. Some teams were large and highly specialized, while others were small and required a single team member to perform multiple roles. We identified and validated best practices by measuring user satisfaction levels for the applications each team had designed; the higher the user satisfaction scores for the application, the more value we attributed to the practices of the team that designed it.

We did not find any correlation with user satisfaction and those teams with the most specialized team members, one way or the other: some teams with the most specialization did well, and some teams did poorly. What we did consistently observe among teams that had high user satisfaction scores, was one characteristic that stood out above all the others—what we began to call shared, holistic understanding. Those teams that achieved the highest degree of shared, holistic understanding consistently designed the best web applications. The more each team member understood the business goals, the user needs, and the capabilities and limitations of the IT environment—a holistic view—the more successful the project. In contrast, the more each team member was “siloed” into knowing just their piece of the whole, the less successful the project.

All of the members of the best teams could tell us, with relative ease, the top five business goals of their application, the top five user types the application was to serve, and the top five platform capabilities and limitations they had to work within. And, when questioned more deeply, each team member revealed an appreciation and understanding of the challenges and goals of their teammates almost as well as their own.

The members of the teams that performed less well not only tended not to understand the application as a whole, they saw no need to understand it as a whole: programmers did not care to know about users, user researchers did not care to know about programming limitations, and stakeholders were uninvolved and were considered “clueless” by the rest of the development team. These are blunt assessments of unfortunate team member attitudes, but we were surprised how often we found them to be present.

We also observed that the best teams fell into similar organizational patterns—even though there was a blizzard of differing titles—in order to explore and understand the information derived from business, users and IT. We summarized the organization pattern in the diagram below. We chose generic/descriptive titles to simplify the picture of what we observed. In many cases there were several people composing a small team such as the “UI Developer(s)” or the “User Representative(s)” often with differing titles. Also fairly common were very small teams where the same person performed multiple roles.

Holistic Awareness Fig 1

Fig. 1 — Teams tend to organize in similar patterns in response to the information domains they need to explore and understand

Even this simplified view of the development team reveals the inherent complexity of the development process. The best team leaders managed to not only encourage and manage the flow of good information from each information domain, but they also facilitated thorough communication of quality information across all the team members regardless of their domain. Here’s how they did it.

Five Key Ways to Promote Shared, Holistic Understanding

1. All team members—all—conduct at least some user research

Jared Spool once wrote that having someone conduct user research for you is like having someone take a vacation for you—it doesn’t have the desired effect! On the best teams, everyone, from programmers to stakeholders, participate to some degree in user research. A specialist on the team often organizes and schedules the process, provides scenario scripts or other aids to the process, but everyone on the team participates in the research process and thus has direct contact with actual users. There is no substitute for direct contact with users. Interviewing living breathing users, ideally in their own home or work place, makes a deeper impression than any amount of documentation can duplicate.

2. Team members participate in work and task flow workshops

Designing applications to support the preferred work and task flow of the users—providing the right information, in the right features, at the right time—is one of the hallmarks of applications that get high user satisfaction scores. The best teams devote enough whiteboard-style collaborative workshop time to explore work and task flow (including in the sessions actual users when possible), until all team members truly understand all the steps, loops and potential failure paths of their users.

3. Team members share and discuss information as a team

A simple practice, but one which is often overlooked, is taking the time to share and discuss findings and decisions with the entire team. Too often teams communicate information of significant importance to the project through documentation alone or through hurried summaries. Even if it is not possible for the team to participate in all user research or in mapping out all work and task flow on a whiteboard together, at a minimum, the team should go through the results of these processes in sufficient detail and with sufficient time to discuss and understand what has been learned.

Direct participation is the most effective way to learn and understand. Full and relaxed discussion with team members is the second best. Reading documentation only is the least effective way for team members to retain and understand project information.

4. Team members prioritize information as a team

Not only is it important that all team members be familiar with information from all three domains (business goals, user needs, and IT capabilities), but it is especially significant that they understand the relative importance of the information—its priority.

My company developed what we call a “Features and Activity Matrix”, based on our own experience designing applications, and from the practices we observed in our study. The features and activity matrix accomplishes two things:

  1. It forces teams to translate business goals, user input and IT capabilities into specific proposed features or activities that a user would actually see and use at the interface level. We have found that if an identified user need (for example, the need to know currency conversion values) isn’t proposed as a specific feature (say, a pop-up javascript-enabled converter, tied to a 3rd party database) then a potentially important user need either gets lost, or is too vague to design into the application.
  2. The features and activities matrix allows team members to prioritize the proposed features and activities from the perspective of their own domain through a process of numeric ranking. Business ranks according to the importance to the business, IT ranks according to “doability” (measuring budget, resources and schedule against each proposed feature and activity), and the user representatives rank according to their assessment of what will make users most satisfied.

The numeric ranking for each feature or activity, from each domain, is then averaged to arrive at a consensus prioritization of every feature and activity proposed for the application. If, as is usually the case, the team is already aware that they cannot do everything that has been requested or proposed, this is a very effective way to determine which features and activities are not going to make the cut and which ones have the highest importance.

Holistic Awareness Fig 2

Fig.2 — Features and Activities Matrix. Note that we used a ranking scale of 1-5, 5 being the most important.

5. Team members design together in collaborative workshops

Once information from all three domains is gathered, analyzed, shared and prioritized, the remaining—and most powerful—practice to promote a shared, holistic understanding is to conduct wireframe-level, whiteboard-style, collaborative design sessions. Your session participants should include a solid representation of users, business and IT but not exceed roughly 12 people. In these workshops teams can work out together the basic layout, features and activities for most of the core screens needed for a project. These sessions can, and should, require multiple days. We have found that between 10 and 20 core screens can be considered, discussed, iterated and designed in 4-5 days of workshops.

Make sure everyone is fully engaged: business cannot be half committed, and IT cannot say that they will determine later if the screen can be built. All team members should be prepared to make real-time decisions in the collaborative design session. Inevitably there will be a few things that simply cannot be decided during the session, but the greater the shared, holistic understanding that already exists, the fewer things that will require processing and final decisions outside the session.

A well prepared collaborative design session both promotes and leverages the team’s shared, holistic understanding of the project. Even though they are time consuming, collaborative workshops eventually save time because the team ends up needing less iterations to refine and finish the design. Collaborative workshops also insure higher quality; there are fewer missteps and errors down the line because of the shared understanding of the design. Finally, collaborative workshops create great buy-in from business and IT. It is far less likely that some—or all—of the project will undergo unexpected or last minute changes if the goals and priority of features is clearly established during the design process.

Whatever the size and structure of your team, and no matter how many or how few specialists it has, your outcomes will be better if everyone shares a holistic understanding of the work at hand. Taking the extra time as a team to develop a shared holistic understanding pays off in greater efficiency in the long run—and most importantly—in greater user satisfaction and overall success!

Collaboration Sessions: How to Lead Multidisciplinary Teams, Generate Buy-In, and Create Unified Design Views in Compressed Timeframes

Written by: Sasha Verhage
“…The benefits of this tool include increased participation, increased understanding of the value of each discipline, and consequently increased buy-in from the team.”

I have participated in, led, and suffered major website redesign efforts. Whether at process-heavy consultancies, notable product companies, or design studios, all teams experience the same points of pain: late feedback, lack of common design vision, and complaints that individuals or teams didn’t have enough input.

No matter what your design process is during the early phases, most interaction designers and IAs complain that requirements aren’t clear or specific enough. Product managers are anxious to get started and have high hopes–expecting product innovation, timely delivery, and no negative impact on the site. Engineers are frustrated because they are not solicited for input. Finally, to complicate matters, the approval process becomes mired in team dynamics and politics.

The traditional software development “waterfall” process was first described in 1970 in a paper by Winston Royce. He postulated that as the project moved from requirements gathering to implementation, specialized disciplines would create specific deliverables and then pass them off to the next discipline. For example, interface designers would hand off low fidelity schematics to visual designers, who would design the look and feel before delivering to web developers.

fig1.gif
Figure 1. A frequent and frustrating consequence of the waterfall process is late feedback.

Over the years, I have developed a framework that I call “Collaboration Sessions.” Collaboration Sessions encourage multidisciplinary collaboration while creating a unified design vision–all within a compressed time frame. The benefits of this tool include increased participation, increased understanding of the value of each discipline, and consequently increased buy-in from the team.

collab.gif

What are Collaboration Sessions?

Collaboration Sessions are highly interactive meetings (or more accurately, work sessions) with representation from each discipline. These meetings address everything from strategic planning to the design of site sections and page details. For example, a team working on the Travel section of our site used this technique to brainstorm a new line of business and then used it to help design page details. This method is most helpful for redesigns, new features, and controversial or strategic sections of a site (e.g. the home page). Typically, an interaction designer or product manager leads the meeting at the beginning of the Design phase.

fig2.gif
Figure 2: Collaboration Sessions are excellent tools, especially during the design phase of a project.

A project manager is designated as the central point person, responsible for inviting the required and optional representatives from Sales, Marketing, Product Management, Interaction Design, Visual Design, Web Development, and Engineering (or whatever your particular departments/functions are). The meeting request should include an agenda and describe the purpose of the meeting. The request might also encourage the team to send in ideas and innovations or ask them to present their new ideas to the group at the beginning of the meeting. If it makes sense, prior to the Collaboration Session, the interaction designer conducts due diligence and a heuristic evaluation of the current site. For a new feature, the designers might look at competitors or research best practices.

Collaboration Sessions are most effective when attendees understand the project goals, the design problem to be solved, the roles and responsibilities of individual team members, and the business context. The facilitator starts the meeting by explaining the purpose of the meeting, the type of contributions encouraged, the ground rules for the session, and the desired outcome. The facilitator also assigns roles: time keepers, notetakers, final arbiters, and scope police (who call attention to scope creep). It might be helpful to assign these roles before the meeting. The product manager (or participant with the most domain knowledge) should provide the team with the historical, business, and competitive context. Scope boundaries can also be identified. For example, when a page will be completely redesigned and rearchitected, the product manager will state the purpose of the pages and the current design challenges.

With a blank wireframe template pasted on the wall, the facilitator turns the team’s attention to page level design. The focus is high level, not pixel level. Figure 3 is an example of “Search Results” with an accompanying “Notes” page. Advertising requirements are identified in pink. The primary focus of the page is the search result set, noted in purple. The secondary call to action, identified in blue, is “Modify Search.” The Post-It notes provide general placement for the page components but do not dictate design. However, some design decisions may be requirements; for example, the Sales department may specify that all ad units have to be above the fold.

fig3.jpgfig4.gif

Figure 3: Next to each page or site section, the Notetaker documents issues, to do’s, questions, assumptions, and major decisions.

As issues and questions arise, they are captured in real time by the notetaker on a “Notes” page taped next to the page being designed. At the end of the meeting, the To Do items are assigned to specific people. After the Collaboration Session, the interaction designer distributes the Notes. The team can then use the session output to start working on a more detailed design, creating wireframes that use the accompanying Notes documentation. When answers are found, the Notes page is updated; Notes pages should be living documents, shared by team members.

Before the Meeting
  • Identify key constituents from Sales, Marketing, Product Management, Interaction Design,Visual Design, Web Development, and Engineering (you can easily map these functional areas to your company’s organization)
  • Send out agenda to participants (assigning any prep work)
  • Facilitator conducts due diligence
  • Prep the room with supporting material (e.g. screen shots, marketing research, supplies)
Collaboration Session Agenda
  • Purpose of meeting (facilitator)
  • Ground rules and roles/responsibilities (facilitator)
  • Scope boundaries (what’s in/out of bounds) (product manager)
  • Walk through pages (facilitator)
  • State purpose of each page (product manager)
  • Discuss current design problems or challenges
  • Brainstorm page solutions. Use Post-It notes to document primary and secondary calls to action and other page elements. (facilitator)
  • Capture issues, to do’s (with owners), assumptions, major decisions, and design rationales on “Notes" page (notetaker)
  • Document any major product innovations (notetaker)

What Collaboration Sessions are Not

The process should not be “design by committee” but rather design for common understanding. When the designers provide solutions during the meeting, these are meant to foster discussion, not offer a final solution The focus should not be on granular elements like interface widgets or pixels. If you are debating whether something should be a drop-down or not, then you are wasting people’s time. Remember, if you have representation from each discipline, these are costly meetings. The facilitator should not try to provide real-time solutions or answers. Collaboration Sessions identify the requirements (i.e. what needs to change, the current design’s problem areas, current traffic pattern, what competitors are doing). The designers should take this information and ultimately present solutions to the problems. The session output serves as the foundation for design rationales.

Benefits
Collaboration Sessions enable teams to compress the development cycle by accomplishing the following:

  • Establish a common design vision. Encourage early collaboration.
  • Generate ideas and innovations for release and/or inform product roadmap.
  • Identify questions, issues, and to do’s.
  • Encourage full team contributions and subsequently ensure team buy-in.
  • Document assumptions, design rationales, and changes.
  • Enable parallel development (i.e. mobilize each discipline).
  • Improve scoping, estimating, and planning (e.g. number of impacted pages, new pages).
  • Inform process flows and page level design.
  • Can improve credibility (by identifying design problems and having designer recommend solutions). Can change perception of user experience design from “pair of hands" to more "strategic partner."
  • Define the design problem within its historical context.
  • Establish regular check-in meetings.
  • Allow User Experience team to present recommendations, design solutions, and design rationale as a unified voice.
  • Identify opportunities and innovations at a strategic level.

Best Practices
After running Collaboration Sessions over the past couple of years, here are some best practices I recommend:

  • Prep the meeting room the day before.
  • Encourage one representative from each appropriate discipline to attend. Cross-functional representation deepens the understanding of the problems, encourages more holistic solutions, and fosters collaboration across the team.
  • Start each meeting by allowing the business owner or the person with the most experience to provide context on the competitive landscape, current design, or any useful and relevant information. Also, identify roles: timekeepers, final arbiters, scope police, product manager as hub, design manager as process filter.
  • Make adaptations or customize the format of Collaboration Session process after each session if needed. Encourage the team to be flexible, nimble, and adaptive.
  • Summarize, capture, and distribute the notes from each Collaboration Session within 24 hours.
  • Involve key stakeholders before the session, and have them buy in to the participants, agenda, and meeting objectives. If relevant, send out any homework, such as links to competitors, before the meeting.
  • Send out “Issues and To Do’s” immediately after meeting.
  • For major innovations, package ideas and “sell up” to key decision makers afterwards.

Testimonials

After the first one:

“I just wanted to say I thought the first collaboration session went really well overall. Lots of focused participation from different disciplines. Discussing and agreeing on goals and priorities at a page/template level is really going to help communication throughout the design process. It’s often a big leap from presenting documents to one another to solving problems together and getting design work done in real time.”

Contract content strategist with 8 years experience.

After project was done:

“Super proud [of the redesign]. Collaboration sessions were pain in the ass to attend, but extremely valuable to the process” “Very proud and pleased with the end result.”

Product Manager.

“Best process at Yahoo I’ve seen, in 5 years. Smoothest redesign to date I’ve worked on.”

Senior Engineer.

Invitation from another team

“Our goal of this meeting is to have you lead us through a Collaboration Session…. I remember how helpful this process was in designing pages for our previous product….I think our team could really benefit from applying this tool.”

Senior Product Manager

Sasha Verhage is a Senior Design Manager at Yahoo with over 10 years experience in software, internet and professional services industries. He is passionate about leading fast paced and high performing teams to successful outcomes. In his not so spare time, he makes award winning wine under the Eno label.

Tackling Maintenance Projects

Written by: Dan Saffer
“Maintenance projects are distinguished by their (relative) brevity: for a majority of them, the design period will be two weeks or less, and very often only a day or two.”Most of what one reads about information architecture and interaction design work assumes several things: that you are creating the project from scratch, that you have the ability to alter anything on the screen, and that the project is free of technical or political constraints. None of these assumptions are true when you are dealing with maintenance projects.

Defining maintenance projects
Very soon after the first website went up, the first maintenance project began.

By maintenance project, I’m referring to altering an existing website or application, one that you likely had nothing to do with creating that comes loaded with technical and organizational baggage. It can be as small as adding a link to a page, or as large as restructuring the entire site. For in-house IAs like myself, these can make up the majority of work of you do. But more and more, consultants (since the collapse of the New Economy and the dearth of new web projects) are being asked to deal with them as well.

Most maintenance projects involve one of the following:

  • Adding functionality or content.
  • Removing functionality or content.
  • Revising architecture — moving content, pages, or whole sections to a different location.
  • Changing interaction — altering the way an existing piece of functionality works.

Maintenance projects are distinguished by their (relative) brevity: for a majority of them (excepting major restructuring work), the design period will be two weeks or less, and very often only a day or two. They generally require a quick assessment of the situation and decisive judgment, not to mention immediate action: rapid prototyping, wireframes, and site maps. Few maintenance projects will allow the luxury of a full exploration and design process.

A typical maintenance project goes something like this: someone has a new piece of functionality or content they want to put up on the website. The IA’s job: find the best place for it, both in the site overall and on the specific page chosen.

Server logs or a usability test (or common sense) can trigger a maintenance project by revealing that a site or an application has been structured wrong, and users aren’t able to complete their tasks (finding information, buying things, etc.). The IA’s job: fix it.

Two types of maintenance projects
There are two types of maintenance projects that I’ve taken to whimsically calling “Atomic” and “Neutron.” Each has pitfalls and challenges. I’ll discuss the rarer of the two first: Atomic.

Atomic Maintenance Projects (AMPs) are those projects that allow you to completely change how the page, section, site, or application works. AMPs usually occur when a business requirement causes you to rethink the whole “environment” around the requested change. Like an atomic blast, it can destroy things on all three tiers of development — front end, middleware, and databases. AMPs are actually regular projects in the guise of maintenance, and if you are lucky enough to convince your team to tackle one, you will likely be able to do a fuller design process than you would have otherwise.

“Often, IAs are left to wrestle with decisions that were made long before their involvement with the project; decisions that people in the company may not even like but are stuck with.”For example, let’s say an existing online form needs to capture some new sort of data (e.g., a phone number). Typically, this might just involve adding another form field. But, on examining the page where the form resides, you discover that the whole thing should be reworked: the format is too confusing, the required fields aren’t indicated, and just adding a field would make the visual design even more cramped.

This type of discovery, I should add, is not rare. I guarantee you’ll come across all sorts of things that need to be improved when you take a critical look at many web pages. What is rare is being able to convince your clients and the rest of the team that more redesign needs to be done than just adding in a form field, as everyone expected.

This, in project management terms, is called “scope creep,” and now you are the advocate of it. Sometimes this is welcomed, because a small change can reveal how poorly something is built, or show possibilities no one has thought of. But be prepared for pushback! If time is of the essence (and when is it not?), the rest of the team will want you to just pipe down and make the less-drastic change they were expecting. In other words, they will want you to make the project Neutron.

If Atomic Maintenance Projects level everything and start from scratch, Neutron Maintenance Projects (NMPs) leave the “buildings” (a.k.a. the three development tiers) mostly intact. These are much more common, because, as we’ve seen, even projects that should be Atomic aren’t because of the difficulty in convincing people that they need a more thorough treatment. Instead, the IA is left with making the best choice possible given the existing conditions.

And there’s the rub. Often, IAs are left to wrestle with decisions that were made long before their involvement with the project; decisions that people in the company may not even like but are stuck with. Or decisions they have a vested stake in keeping intact. Or decisions various forces in the company (IT, Marketing, and Legal being typical stakeholders) insist must remain, for reasons the IA may or may not agree with.

To illustrate, let’s look at the online form example. The new requirement is to collect a phone number from the user. Since the phone number could be international, you decide to create one long text field. Simple, right? Yes, until the database manager tells you that there are already two fields in the database: one for international phone numbers and one for U.S. phone numbers — and she’s not about to change the whole database structure to combine them. The Java developer doesn’t have time to write a script to send U.S. numbers to one field, and international to another. And the graphic designer mentions that by adding two separate fields, it will knock the page “below the fold,” and that’s against company policy. Oy.

It’s often frustrating to deal with NMPs, especially when you know they should really be AMPs. But there are ways to cope.

Maintaining your sanity during maintenance projects
Hardly anyone likes to do maintenance projects. They are seemingly unglamorous, janitorial chores to be dealt with in-between the bigger and better projects. And while I am not going to argue otherwise, I can provide some hard-won hints that might make them easier the next time you’re handed one.

Think inside the box. The first thing you need to do is review the requirements and figure out the best place at a high level for the change you are being asked to make. Many requirements are pretty specific about this (“We need to capture the phone number on the registration form”), others much less so (“We need to put the bios of our board members on the site”). If you have a map of the site or application, it will be handy. If not, sketch one, briefly. Find (or create) a logical space for the change.

Know the page (Part I). Once you’ve found a place for the change (if it is an existing page or screen), examine the page closely, click links, push buttons. This may seem basic, but pieces of functionality and content don’t often work the way you think they will (or the way they should). Recently, on a Neutron project, I blithely assumed that underlined items were links to related pages of “help” content. Not so: they revealed a hidden DHTML layer with a brief description of the item. You’ll often come across things that make you want to make the project Atomic during this investigation.

Know the page (Part II). If you have any technical savvy at all (and you should), find out about how the environment was constructed. Is it part of a content management system? Is any of the content dynamic? What is the application built in, Flash? C++? Java? Look at the source code if you can. Talk to the developers, the people who built it. This will allow you to know what is possible given the technical constraints, as well as alert you to any hidden problems before you start your wireframes.

Beware of hidden traps. Often, there are secret conditions and corporate political issues that you may be unaware of. These, of course, won’t be written down anywhere. Your clients just know them. The best way for you to know them as well is to cultivate a friendship with the people who know this information: the members of the development team, and especially the project manager. She can tell you that the CEO hates the word “corporate” or that Marketing is opposed to underlining links.

Pick your battles. When you do discover serious problems with a page, pause before sounding the cry to get it changed. Most importantly, consider how much impact there will be for the user and weigh it against the battle you will have to wage to get it fixed. If it will substantially make the user’s experience better and only requires you send a nice email to the Visual Basic programmer, then, by all means, proceed. If it will make a slight difference, but requires you to argue your case to the President and CEO for the next month, maybe it isn’t worth it. Here, consultants have the edge over in-house IAs. “Innies” have to measure their own reservoir of corporate goodwill, built up on the projects that have gone before and estimate how much they’ll need for projects coming in the future. “Outties” have less to lose, other than annoying those people who have hired them in the first place (and could get them more work in the future).

Down ‘n’ dirty wireframes. Unless you’ve managed to talk your way into an Atomic Maintenance Project or are creating a new page or screen, it rarely makes sense to reconstruct an existing page as a wireframe. You’ll waste a lot of time doing that, because typically you are only affecting one area of the screen. Instead, take a screenshot of the existing page, move it to Photoshop or Illustrator and make the changes in the image. Then export it as an image into your wireframe template where you can layer your annotations on top of it. These “wireframes” will be easier for the developers to understand, too.

Be considerate of visual design. On new projects, IAs often have the luxury of doing their work prior to visual design. But on maintenance projects, specifically NMPs, you are often adding things to existing pages, so even more care than usual needs to be paid to the visual design. One large, ugly button, ill-placed, can ruin a look and feel that might have taken weeks to get right and get approved. It may go against what usability gurus have traditionally been screaming at us, but I contend that an ugly solution that is functionally correct is worse than a handsome one with less-than-ideal functionality.

While handling maintenance projects are few people’s idea of a dream job, they can help sharpen your IA skills. They make you more aware of what happens in development after your designs are approved. They demonstrate how sites evolve over time, and can help you better design things that account for that reality. They make you aware of the business and technical forces that ultimately affect the design. And, best of all, they make you appreciate non-maintenance projects more.

Dan Saffer is a senior interaction designer at Ameritrade. Over the last seven years of web work, he has worked with a diverse set of clients, from Lucent Technologies to the World Wrestling Federation. He is contractually obligated to say that he won a fellowship in 1998 from the Mid-Atlantic Arts Council. He lives and works in New Jersey and is not writing a book about IA.

His portfolio and blog can be found at www.odannyboy.com.