UX Book Clubs

by:   |  Posted on

In early Nov 2008, I started to talk to a few people about the idea of a book club in Sydney to discuss User Experience (UX) books. Russ Unger and Donna Spencer encouraged me to let other people hear about it, and when I did – through the Information Architecture Institute (IAI) Members discussion list, and then through the Interaciton Design Associaton (IxDA) – many people thought it was a good idea.

And then something surprising happened, people liked the idea so much that they started doing things to make it happen. Andrew Boyd registered the “uxbookclub.org”:http://uxbookclub.org/doku.php domain, set up the wiki, and starting the content rolling. Will Evans designed a logo, wrote a whole bunch of content, set up a decent structure, and let everyone use either, or both, if they wanted. Andrew’s been in on the wiki each day tidying and gardening, making sure it doesn’t get out of control.

First one volunteer, then another, and another put their hand up and offered to organize a UX Book Club in their local area. New York City joined Sydney, Canberra, and Washington D.C. By the end of that first week over 28 cities had a local UX Book Club under way, and nearly 400 people had signed up to take part.

The first meeting was held in Silicon Valley in mid-December, followed by meetings in New York and Los Angeles, Philadelphia, Chicago, Canberra, Sydney & Austin. Through the second half of February meetings were held in Atlanta, Minnesota, Melbourne, Tel Aviv, Brisbane, Toronto, London and Chicago.

The What and the Why

UX Book Club is a fairly simple idea: get a group of people together, choose a book, and agree on meeting details. Go away and read the book. On the date set, come together and discuss the book. Talk about how you might use what you’ve read in your work; how your experiences run counter to the book; an example of how the book is spot on. Have a bloody good argument about it, then go have a drink and talk about it some more.

At a UX Book Club you have an incentive to read some of those user experience books you’ve heard about but still lays on your bookshelf. You discuss the book with other UX practitioners, which will help you get more out of the book. And you meet fellow UXers working in the same town as you.

You also hear about a lot of books that other people have read, found interesting, but aren’t suitable for discussion by the group. That may be because they’re too long, or highly specialized, or too expensive for a large group of people. Hopefully, though, you’ll be exposed to a much broader range of books than you do on mailing lists or blog posts currently.

How It Works

The Sydney meeting – held on February 3rd – seems to have been fairly typical of the experiences across the board – with local variations in terms of weather, location, and numbers. But the stories seem to have a consistent theme: great discussion, lots of energy, and a good time had by all.

“It was fairly incredible how natural — how routine — it felt. I mean, here was a group of people, many of whom had never participated in any community event, and none of whom (to my knowledge) had ever engaged in an extrinsically focused book club. The book became the medium for discussion, though the topic remained entrenched in UX and design.” 
- Jonathan S Knoll, UX Book Club NYC

UX Book Club Sydney held the February meeting at the offices of the “News Digital Media”:http://usit.com.au team in their “New York Lounge”. Their hospitality was greatly appreciated, and the space was perfect for the event.

The event was structured along the same lines as those used by New York City (thanks to Cindy Chastain) and applied successfully in Los Angeles. The meeting opened with a brief welcome and introduction, then a volunteer from the group gave a 5-minute overview of the book (in our case Bill Buxton’s Sketching User Experiences). We broke into two groups (10 and 13 people, respectively) and headed to opposite ends of the Lounge to discuss the book in detail. Cindy’s rationale for the smaller groups was that they give everyone a much better opportunity to contribute to the discussion – and this was borne out by the comments I received afterward.

After a good solid hour or so of group discussion we came back together, had a bit of a recap, thanked everyone for attending, and relocated to a nearby pub to carry on. The ‘official’ proceedings kicked off at 6pm and ended just after 8pm. The ‘after-hours’ discussions wound up around 10pm. Not bad for the first event.

The entire event was terribly uncomplicated, and I highly recommend the format. Better yet, the discussion highlighted areas of the book I hadn’t really considered important on first reading. This new information encourages me to go back and re-read those parts, armed with some real-world anecdotes to help make it more concrete.

“UX Book Club got me to finally pick up a book I had been meaning to read, and to have the chance to exercise my brain a bit. I found myself waxing philosophical with my fellow book clubbers about education and urban planning, as well as positive (and negative) user experiences we’ve had.”
– “Roz Duffy”:http://stellargirl.typepad.com/stellargirl/2009/02/my-current-muse-ux-book-club.html, UX Book Club Philadelphia

The events serve as both means and end. Reading the books being discussed is a good thing, in and of itself. You will get more out of the event having read the book, and the overall level discussion and engagement will be higher for everyone.

But reading the book isn’t required. The book acts as a starting point for a wider-ranging discussion around the topic. Each person brings not only their understanding of the book, but alsp the full breadth of their professional and educational experience to the discussion. So whilst reading the book provides everyone with a common frame of reference, the really interesting discussion arises from our differences.

“Most UX people I know are web interaction designers like me, but the book club drew developers, software UI designers, business strategists, visual designers, and various flavors of agency and in-house IAs and IxDs.”
– Sarah Mitchell, UX Book Club Los Angeles

In saying all that it’s also important to recognise that no two UX Book Clubs will be the same. The books will be different. Some groups will meet monthly; others every alternate month. Some will be small affairs with half a dozen folks and others will be big (30+); and some will be more book reviews than book discussion. And that’s OK. What’s important is that we learn something, meet some people, and enjoy ourselves in the process.

“…And that sort of set the tone for the rest of the event: high-energy, engaged conversation, a fertile middle ground between events where there is a single speaker with everyone else semi-passively engaged, and free-for-all cocktail hours, which are fun and great for networking, but lighter on substance.”
– “Anders Ramsay”:http://www.andersramsay.com/2009/01/17/taking-the-ux-book-club-to-the-edge/, UX Book Club NYC

Getting Involved In UX Book Club

There are two ways to get involved in UX Book Club. The first is to sign up to the group in your local area. A list of existing UX Book Clubs is available on the wiki at “uxbookclub.org”:http://uxbookclub.org. There are around 50 groups already listed – including some groups that are just forming. If you’re working in UX or would like to learn more about the field to help with whatever work you are doing, add your name so that you can be kept informed.

The second way to get involved is to start your own UX Book Club in your area. We’ve found that the best thing to do add your city or town to the wiki list, then post a message to the IxDA.org or IAI mailing lists (or both) letting other people know. We’ll send an announcement out via “@uxbookclub”:http://twitter.com/uxbookclub on Twitter to help spread the word. If you have a local IAI, UPA, or IxDA chapter, tell them about it at your next meeting.

I’ll leave with you a quote from Whitney Hess (UX Book Club, NYC) that echoes the sentiments of so many UX Book Club attendees:

“These books really aren’t meant to be read alone — they’re references as well as jumping off points for exploration of the practice. It was great to hear what others thought of both the content and its context in the greater body of work, book-form and otherwise, that our community has produced.

I’m really looking forward to the next event.”

Flowmaps and Frag-Grenades, Part 2

by:   |  Posted on

I’d like to talk specifics a bit. I’m sure there will be some readers at B&A who aren’t gamers, and probably even more who haven’t played Halo—so my apologies to those folks— but… describe in some detail exactly what you contributed to the finished product.

When I look at Halo 3, what ‘pieces’ of the experience did you work on?

I worked on the IA, navigation and screens for the game shell; the social design for the game for systems such as the party system, matchmaking systems and sharing systems; on rewards systems such as the stats, medals and experience ratings; and also on how that user experience extended to the web through Bungie.net. I also worked on the theater features such as film clips and screenshots, and on the Forge “in-game” UI. My compatriot David Candland handled the in-game HUD in addition to collaborating with me on the design, look and feel for the overall UI and specifically handling the visual design for the game. Aaron Lemay was the art and graphic design lead for our team, including Bungie.net. Max Hoberman was the lead for the entire multiplayer and UI team during the planning stage of the project.

The information architecture and navigation includes all of the screens and flow to support the game experience outside of the game—we refer to this as the “Game Shell” UI. With Halo 3 we started by identifying what the “core game experiences” would be for the game and grouping them into “modes”.

These modes were:

  • Campaign:The story mode where players play through an adventure either solo or cooperatively.
  • Matchmaking:Players are matched with other players over the internet based on similar skills or experience and based on game preferences to play games that are controlled by Bungie matchmaking.
  • Custom Games:Players set their own game rules and maps in a player-hosted game lobby.
  • Forge:Players can customize maps to play in Custom Games or to share with the community.
  • Theater:Players can view films from any game mode and take screenshots.

Do these modes then inform the IA of the shell?

Grouping the experiences as modes allowed us to start with a foundation for the overall player experience and a baseline for the information architecture. Each of the modes support many options within the mode, but these 5 modes have unique characteristics that support a”focused” player experiences within the mode over a period of time. With the priority that “everything is social,” each of these modes are designed to support from 4 to 16 players either locally, on System Link or over Xbox Live, so we gave each of these modes its own “lobby” where players could gather to share the experience.

In addition to focusing the core experiences in the game, this lobby system sets up the infrastructure for our party system. In Halo a “party” is a group of players that gather to play together, particularly over Xbox LIVE. The party leader is the player who makes decisions for what the party will do together, and the system allows players to stay together and do anything they want without breaking up. In Halo 2 this was termed the”virtual couch”…

Yeah, I recall that H2 was really revolutionary at the time—made it so easy to form a group and hang out for the night…

It’s like sitting on the couch together—if you decide you want to switch from one game mode to another on Xbox LIVE you can do it together just like if you were sitting on the couch with your friend. This is a very big deal on consoles because many online systems do not have this flexibility and it is not always easy to get together and stay together online.

The end result was a fairly simple information architecture for our game shell. Each mode has a lobby. Within the lobby, the specific options are contextual to the game mode. For example, in Campaign the main options are to select a level or difficulty for the story, whereas in Custom games the main options are to select a game type or map to play. The lobbies themselves are “locations” for players to gather into a party and play together and once players are together they can easily switch modes from within the lobby system to travel together to try a different mode. For example, a party of players may decide to customize a map together in Forge, then switch over to Custom Games to play on the map they just created.

The other major areas for player experience are community, personal identity, sharing, and settings. These are very much tied to a player’s personal profile and so in the information architecture these are all presented in a global menu that can be accessed anytime by pressing the “Start” button. The menu is always tied to the identity of the player who presses the button.

Regarding navigation and orientation, our goal was that the player always understands where they are in the game and that menus are in most cases only a couple of levels deep. In most cases the player is only a few clicks away from a core location. Another benefit of grouping the experiences into modes is that the main experiences for the game are easily discoverable from the main menu.

What kind of process did you follow?

The overall timeline for game development was “pre-production” where the studio teams plan what we wanted to do for the project and evaluates scope, then “production” where we execute on the design. At the end of pre-production each team submits an overall design document to the leadership group and the project features are approved. For the interface and experience this was a pretty detailed document for the overall information architecture and screens for the game. This is similar to a product requirements document, but in the games world these are design documents. Over the course of the project the design evolved in some places or was scaled back in other places. A great idea may be recognized well into production and is never discarded automatically, but anything new that is proposed during production is weighed against other features that are in development.

Regarding design process, we targeted the foundation first. The information architecture and systems that would support the different features in the game, as well as the overall guiding principles for the game. This allowed us to understand where everything fit.

Then we tackled the major features based on scope and dependencies. Each of these “major features” would cover many areas of the game. For example, the lobby system would provide the foundation for many other features and was also a dependency in supporting the overall IA for the project. It included the “shell” for the interface, the player roster that shows who is in your game lobby and the core navigation for the information architecture. For each major “feature” set, I would put together a proposal for the feature using screen flow “posters” that outlined flow and also detailed screen requirements. We would then review these proposals with the team members that had an interest in the UI. From there we would refine and build out detailed design documents to support the development. Once the feature was built and in the game we would verify that the features were working according to specification through in-game testing.

We also had great support from the Microsoft usability lab. User researchers were part of our review process and provided heuristic analysis of the proposed designs, and also supported usability testing for both the early “prototype” ideas and later with the actual game.

Would your design artifacts look totally familiar to most practicing Interaction Designers? Wireframes, flows, that kinda thing?

Absolutely. The format I found most useful were poster flows.These are large format posters with detailed wireframe screens, navigation and flow decisions for a feature area. These would include detailed specs and use cases for specific features near the screen or decision point on the poster where they were relevant. I would print these out and post them on the wall near the UI pit, and also post them internally as pdf documents.

The posters allowed everyone in the studio to get an overview of the feature by reviewing the printed poster on the wall, and the engineers and QA team would use the pdf version as the spec while developing and testing the feature. I preferred this format because it was a format that outlined “the big picture” graphically, so it was easy to collaborate and refine as a team. It was also easier to update than a detailed 50 page word document. In many cases, the poster on the wall would be the “most up to date” spec because—as we were developing the feature—our team collaborated to work through issues together using the printed posters, and we would update the poster specification with markers as we refined the direction. The QA team calledthe poster wall the “wall of truth”.

I also put together design documents for the main feature areas such as matchmaking, the party system, sign-in and profile, etc. These were word documents with detailed specs, or in some cases excel spreadsheets. The word documents started with an IA diagram and overview of how the feature worked in context with the core shell UI and that then outlined specific specifications for each feature. Early in the project I also had wireframe”prototypes” in power point to walk through certain use cases to explain an idea and get feedback.

Did you do any prototyping of concepts? And how about tools in general? Does Bungie have proprietary tools for screen design and prototyping?

We conducted rough prototyping during planning to test our concepts in a usability lab or to get feedback on concepts, and we also put together a polished director demo to present the final interface proposal to the team at the end of the pre-production phase.

On the rough prototyping we worked with Randy Pagulayan and John Hopson from the Microsoft Game Studios User Research group to test the concepts in the usability lab. We put together a script for the prototype, then I created wireframe screens in illustrator and John coded the screens into a prototype so that test subjects could use an Xbox 360 controller to navigate the prototype. Randy and John and our team spent about three weeks running the prototype through tests and then rapidly iterating on ideas with matchmaking, the core game shell interface and the party system.

The content was all fakery, I think we called the game in the prototype “Mecha”, but it was designed to confirm the fundamental direction for our user experience. The lab setup and process was top notch and I really have to give props to the Microsoft usability team. The process helped us to refine our thinking and have confidence in the information architecture and core navigation. In fact, the final prototype for those sessions is very close to what we shipped in the final game.

David Candland, Max Hoberman and I then put together a polished demo in Director that was scripted to run through the main use cases for our proposed interface direction. We used this to present our proposed direction to the team and Max and the leads used this to evaluate the direction, gather feedback and reach consensus on feature sets and final direction as we moved into production.

Thanks, Colm!

Note: shortly after Halo 3 shipped, Colm left Bungie to work with Max
Hoberman at Certain Affinity, a game design and development company
based in Austin, TX.

Flowmaps and Frag-Grenades, Part 1

by:   |  Posted on

By any measure, Halo 3 is one of the most wildly-successful consumer software interfaces in recent memory: more than 1 million players played the game in its first 24 hours on Xbox Live; over 8 million copies sold to date; and “over 100,000 pieces of user generated content being uploaded daily […] 30 percent higher than YouTube on a daily basis.” It’s probably safe to say that more cumulative man-hours have already been spent in Halo gaming lobbies than in Microsoft Word! But H3 is distinguished for another reason, too. It’s one of the earliest—and definitely one of the highest-profile—mass-market video games to benefit from the contributions of a dedicated interaction designer.

Colm Nelson was the interaction designer for Halo 3 and has been a working UX designer since 2000. Before joining Bungie (the Studio that produces the Halo series), Colm’s background was largely in Internet consumer applications, with a heavy bent toward entertainment software. Colm’s experience is unique, but it’s part of a growing trend in the gaming industry toward employing UX professionals. Colm would like to see this trend continue, and was gracious enough to speak about it with us, and share some insight into the intersection between his ‘traditional’ UX background and his job duties at Bungie.

Hi Colm—I’d like to thank you for taking the time out to speak to the B&A community. Given the audience here, I thought this emerging trend—this matriculation of interaction designers into the gaming world—is something that folks would want to know more about…

Online systems that facilitate player experiences around social interaction, custom content sharing and online communities have received a lot of attention by both the gaming press and fans and is definitely a hot trend in gaming. The gaming press has even begun to draw comparisons with these features to You Tube, My Space and Facebook. My observation is that developers that are offering more features in [the] user experience around the game are seeing more of a need to specialize and fill roles specifically around user experience and interface design.

Games with success in these areas have generally done a good job developing a solid feature set and matching the social goals of gameplay with the accessibility and usability of the features. Ultimately these features add to the longevity of a game’s popularity, which translates directly to sales. I think as a result there are more opportunities for traditional interaction designers in the games business.

I’ve met developers that are actively recruiting from traditional software interaction design to take ownership of these features and if you look around you’re starting to see postings for UI designers—both Bungie and Blizzard are actively recruiting interaction designers and experience designers. There are also studios that are championing player experience research and design such as XEOdesign, Inc.

But I also think that if you look around you’ll see that it’s not as clearly defined role in all game companies as it is in traditional software so I think as a trend it’s fairly early. My impression is that in many game companies the interface and experience design in games is handled by either designers or artists that are also responsible for the overall game design. The good news is that if you are an interface designer with a passion for games, there are definitely opportunities out there.

Let’s start at the beginning. I actually remember seeing the job req. at Bungie that you filled … it even used the term ‘Interaction Designer.’ My jaw almost dropped—design jobs in the gaming industry typically focus on character design, level design, gameplay and mechanics. How did Bungie ‘catch religion’ about strong interaction design? About paying attention not just to the core gaming experience, but also all of that scaffolding that gets you into the game? The experience around the game?

Yeah, I had the same reaction when I saw the posting. I’d been looking for opportunities in the games industry for some time and had not seen any positions related to interaction design, so when I saw the posting I was amazed.

The guy that hired me, Max Hoberman, was the online, UI and multiplayer design lead for Halo 2. Max and the team at Bungie are really passionate about the user experience around the game and also about usability. It’s just part of the culture of the studio. You can see the results from the design of the party system and the matchmaking system from Halo 2. Heading into Halo 3 there was plenty of ambition for the social experience and with features for the game so the team decided to hire a dedicated Interaction Designer.

And how did you get the job? 😉

As soon as I saw the position I put together a portfolio and cover letter that said I wanted to help Bungie in their quest for world domination. I managed to get a phone interview with Max, which went OK. His feedback was that he enjoyed our conversation but if we had a second conversation he expected me to be more critical with my observations about what could be improved from Halo 2 and Bungie.net. This was on a Friday. The “if” felt pretty dicey to me so I decided to be proactive.

I worked all weekend on a concept document on ideas to improve Halo 2 and fired it off on Sunday night at 3am. I wasn’t sure how it would be received but it paid off because I got a invitation to visit Bungie for an interview. I flew to Seattle to meet the team for a full day interview and was really impressed with the energy and passion that they had for design and the experience around the game. It was a lot of fun—I was also passionate and the interview felt like a series of brainstorming sessions as we discussed problems and ideas and how we might solve them. I guess it went pretty well because they offered me the job!

Describe the development team to me. I (like you, before your time at Bungie) come from a web & consumer applications background with roles like Product Manager, Project Manager, Developers, Designers, Researchers. Is game development roughly the same? How were you situated on the team?

There are similarities. It is still software design so all of the practical considerations still apply—you need to manage the project well in order to succeed and you need the resources to make it all come together. Producers, engineers, designers, researchers and QA all play a role on the team. Producers at Bungie are roughly equivalent to project managers from my previous experience, although I think the producer role varies quite a bit across studios. But at the same time you have cinematics, art, modeling and animation that are also core to the project.

There’s really not a “product manager” role, at least at Bungie. The team makes pitches for the game, the leads of the studio then decide what will be greenlit for production, and the team leads propose and drives feature sets for the project. It’s a very collaborative process and it is driven by the leads of the various disciplines. An example is that in designing the online experience and interface plans we solicited feedback, then proposed features and prototyped “proofs of concept” in order to land on the feature sets that would be developed for Halo 3.

Was there a bit of culture shock moving into the gaming world? Did folks on the team generally ‘get’ what you were brought onboard to achieve?

Yeah, there was a bit of culture shock for me. Mainly because some of the tech, process and roles on the team were new to me. As far as people getting my role, I’d say it was about the same as what an interface designer typically encounters when joining a new team. Definitely the core team responsible for interface and social design had clear goals for how the interface design process would work and understood what I was tackling—we tackled it together as a team. I was really surprised at how important interface design and usability was to the entire team—it was awesome! And at a higher level, even if all the folks didn’t get the details about process, they were supportive and as a rule folks at Bungie are really good at giving feedback on concept proposals and contributing ideas.

[Stay tuned for another installment of Colm Nelson, designer and gamer.]

Building the UX Dreamteam – Part 2

by:   |  Posted on

As we discussed in “part one”:http://www.boxesandarrows.com/view/building-the-ux, the skills in research, information architecture, interaction design, graphic design and writing define the recognized areas of User Experience design. However, there still remains much to discuss about what makes a UX team dreamy.

Each UX Dreamteam has a finely tuned mix of skills and qualities, as varied as the environments in which they operate. Part two will address whether a person has the right ‘hard’ skills and ‘soft’ qualities like communication style, creativity and leadership ability to fit your particular organizational context. We’ll also touch on the quality of an individual’s personality that may or may not complement the others on your team.


Perhaps the most important consideration in forming your Dreamteam is mixing the personalities of your superstars. As mom used to say “It’s not just about how you look, it’s what’s inside that counts.” A candidate may look ideal on paper, but until you have them in front of you, talking and interacting, you won’t know if what is inside will be a fit. Your group spends almost as much time together as apart, they need to respect and like one another to work well together. Personality typing tests hold the promise of quantifying the immeasurable, but you would be ill advised to use them as part of the interview process. Myers Briggs, DISC and plenty of others use various axes to measure the intrinsic tendencies of a person.

As cool as it sounds, the science is just not exact enough to act as the basis of any decision. This is not to say that these tests are not illuminating in their own right – they certainly foster greater understanding and empathy among teams. Generally speaking, though, people under pressure may answer personality tests as they think they should rather than honestly.

Collaboration is a big part of design best practice and the ability to work well with teammates should be of paramount concern. Selflessness indicates that a candidate is a team player as they seek to raise not only their own reputation, but equally those with whom they work. Humility, humor and empathy are virtues particularly relevant to the creative industry and should be sought after in UX professionals. Each player on the Dreamteam accepts when they’re mistaken, keeps each other creatively entertained and feels for the users they serve.

As much as any skill or quality we have already discussed and will explore in this article, finding the right personality type you need is the classic answer: ‘it depends’. It depends on the personalities of existing Dreamteamsters, the type of work they do, and on the organization into which they must fit. There is no magic formula, but there is one thing to always avoid: toxicity. Morale and productivity can be totally undermined by a “toxic person”:http://bipolar.about.com/od/support/a/070315_toxic.htm. Having one aboard can turn your Dreamteam into a nightmare.  So, do your homework to avoid inadvertently hiring them.

Screening Tips:

Look for signs of toxicity by asking about previous work places and their interactions with teammates. Did they voluntarily leave the last job? Do they mainly talk badly about their last workplace? Remember, a toxic person is often manipulative and they may seem great on the surface, so check references. If you misjudge a new hire and you realize you have a toxic person aboard, waste no time in jettisoning them, no matter how skilled they may seem.

Creative and Analytical qualities

Most jobs in the UX Dreamteam involve a level of creativity and analysis, but it’s a rare gem who is a rock-star operator in both these modes. But visionaries and analysts are equally necessary, ensuring great ideas and the ability to organize and actualize them.

A creative person doesn’t see a glass half empty or half full, but instead asks why it should be a glass at all. An ability to think laterally, meaning" to escape from a local optimum in order to move towards a more global optimum" (“Edward de Bono”:http://www.edwdebono.com/debono/lateral.htm) – is the talent from which innovation is born. A Dreamteam accesses their creativity readily and regularly to push beyond the obvious for an appropriately innovative solution. Ensure a proportion of creative genius in your Dreamteam to increase business success and thereby the team’s reputation.

Your analytical superstars can process vast amounts of information and distill it into a concise and cohesive experience for the user. They are methodical, account for every detail, and question inconsistencies. They grow solutions by breaking a system into its component parts, then creatively reassemble it in logical order. Good analysts are passionate and detail-oriented when identifying patterns in data and behavior.

Screening Tips:

Given how ideas are often difficult to credit to the interviewee, gauge creativity from the dialogue and candor during the interview. A truly imaginative person effortlessly surprises you with a fresh, off-beat approach to old problems. Responses to tangential or seemingly random questions can help illuminate this quality. If they can link the absurd back to realistic solutions coherently and with humor, you can be sure there’s creativity within. Analytical people are interested in details. Does your candidate flinch at the idea of auditing the content of large information system? If they have they done data analysis before, did they jump into it enthusiastically? How did it go?

Practitioner vs. Managerial qualities

Managerial qualities are confused with experience in most professions, and UX is no different. Experience correlates with peer respect, but respect is not all a manager must command. Peter Merholz talks of managers needing to be either "T" shaped "Bar" shaped, referring to the profile of skills they possess. "T"-shaped people have a broad and shallow knowledge of most skills and go deep in only a few.

"Bar"-shaped people do not plunge the depths of any expertise. As “he says”:http://www.peterme.com/?p=580, they are all about the connections between disciplines, and being able to articulate the power of that integration. An "I" shape would indicate deep knowledge in just one or two areas. This profile suggests an awesome specialist practitioner (yes, there is an "I" in Dreamteam!).

Good bosses are quietly also coaches, therapists, facilitators, communicators, organizers and politicians. As leaders, they are comfortable in setting an agenda for others to fulfill while inspiring the Dreamteam to meet or beat that agenda. Your luminary leader provides ‘air cover’, also known as ‘running interference’. Making space for their reports to work by fending off interfering people or tasks, the manager ensures the Dreamteam is focused, not randomized. 

People who find less satisfaction in helping others to be effective are better placed as well-compensated senior practitioners. To presume that someone senior should be promoted into a management position is misguided. A manager’s UX skills are less important than their ability to co-ordinate a group of individuals and spot what your organization needs from them.

Screening Tips:

When seeking managerial talent, look for someone who will revel in the Dreamteam’s success, rather than their own. How have they "run interference" in the past? New managers sourced from within a team show a tendency to get the best out of others prior to their promotion. This is known as "acting up" and makes a good task to set potential managers to test their aptitude. If you’re looking for a practitioner, be sure they’re not fixated on being a manager, lest their ambitions undermine the effectiveness of your designated leader.

Strategic vs Tactical Ability

We all know guys who stand idly by, watching others do their work and wryly commenting, "You look after the details, I’m the big picture man.” Those who strategize with ‘blue sky’ ideas can raise the ire of people slaving at everyday tasks. Tactical skills are just as valuable as strategic. Each serves their purpose in envisioning and getting things done.

Conceiving an entire system and determining what both the business and users get out of it are the domains of big picture people. It is hard to imagine success without their vision to work toward. These people can be creative or analytical but find implementation a chore. They are typically well informed of industry trends and can forecast the future through them. While vision is an awesome asset, without attentive "small picture" work, it’s an apparition. Strategists think one to five years ahead and beyond and are good at depicting a vision.

Tactical people focus on day-to-day activity and on success in the one to six month timeframe. With the exception of think tanks, the organizational balance needs to skew toward small picture people in order to achieve success. Many startups and UX teams fail because of the inverse balance.

Screening Tips:

To find the detail-oriented, look for evidence of finishing products and a personal satisfaction in seeing all loose ends tied up. A strategic thinker will show evidence of helping others to see the wider context of what they’re doing, often through conceptual and architectural diagrams. Can they show you some? Also ask questions which illuminate how they’re plugged into where your organization’s industry and the wider UX field is headed.

Innies vs. Outies

In-house teams (aka "Innies") have needs different to external agencies that provide interface building/designing services or consultancy. An in-house team is working toward increasing profitability through UX. In many cases, the nature of projects does not change over time because there’s only one type of business to support. Exceptions exist, but in general those building in-house teams should discount candidates who need variety to thrive.

The in-house Dreamteam is also better suited to agile development methodologies, which rely heavily on face-to-face contact. Unless a consultant is able to work on-site for the duration of the agile project, they will not be able to fulfill some of the tenets surrounding ‘less documentation, more talking’. Aside from communicating an absent author’s intentions, documentation is a mechanism used by agencies to cover their backside if a client claims poor diligence and won’t be abandoned willingly.

Agencies don’t make much money from staff who aren’t comfortable playing the consultant role. Working under pressure, answering expertly on all subjects related and sometimes unrelated to the job requires a certain type of communication style and self-confidence. Agency staff (aka “outies”) must be broad-skilled and part salespeople to make their expertise and company’s value obvious to clients. This isn’t to say that these qualities aren’t good to have on the in-house UX Dreamteam, but they’re less critical to business success and can be compensated for in other ways.

Screening Tips:

Stack your in-house team with stars who are tactical, for their willingness to roll up their sleeves, dig-in and get enjoyment from attacking a long-term goal is what you need. Strategic thinking is also attractive, but you may want to emphasize this in your management function where vision is expected. Beware hiring those with purely "innie" experience for "outie" roles and vice versa. Outies may find innie work mundane and innies can struggle in the faster-paced, higher-pressure outie workplace. Outies need to have political and sales savvy to navigate varied organizations and present value. Confidence, plausibility and magnetism will be obvious – you’ll want to hire them before they’ve shown you their ample skills. Though be sure they have those too!

Organizational Contexts

Hiring managers generally consider organizational context subconsciously when preparing their Dreamteam, usually feeling out the candidate with gut instinct rather than concrete comparisons.  It helps to abstract the organization into something you can test applicants for compatibility with, like a “persona”;:http://en.wikipedia.org/wiki/Persona for instance, then you can envision a compatible teammate for that persona. Size, work processes, project types, employees, industry and brand among other things influence the organization’s personality.

Some organizations are process-driven and others are more free-form. Process ensures that work complies with a to-do list prescribing smooth running and/or best practice. The less experienced use process like new bicycle riders use training wheels. Some people flourish within a controlled environment. Others feel hampered or oppressed by it. What are the processes used within your organization? What unique characteristics do individuals who operate within them need to be happy and successful?

A Dreamteam’s number will impact the duties each superstar performs. Small organizations can have tasks similar in number to their larger counterparts, but spread them among fewer people. This inevitably means one fulfilling multiple roles. The graphic designer might double as the interface-layer coder. The Information Architect may also be the researcher and writer. If you are in a small organization, a ‘gun’ specialist with all their UX skills primarily in only one area may not be a good fit.

Every workplace has a pace. Agile development or simply expeditious environments tend to be frenetic and mean working quickly. Some people don’t perform without time to pause, think, rework and perfect their work. Others will be frustrated if it takes a long time to get things done. They won’t always agree that crafting something perfectly, or documenting design thoroughly is time well spent. Sometimes perfection is expected, but timescales remain fixed. In this case, experience and coping well with stress is consequential. 

Screening Tips:

What kind of personality does your office have? Who would get along best with that person? Prepare to win the best fit by making a list of organizational attributes and qualities that will complement these. Agile methodologies should be coupled with experienced folk who are natural communicators; as should organizations without process to guide activities. A quiet consensus builder might suit a contentious office, etc. Use the example below to get you started – be creative and modify the attributes as you see fit.

Company Persona and Match

Here’s an example of how you might break down how a potential new team member might fit in with your organization:Take the time to analyze what your Dream Team needs and how well that fits potential talent.

Where do we go from here?

Hiring UX staff is rarely easy, but now you can take a structured approach to identifying the skills and personal qualities your team needs within your organizational context.  Like any craft, building the UX Dreamteam takes practice and the occasional mistake leads to growth as a hiring manager. Even when you think you’ve mastered it, there is still an element of luck to contend. You may be willing to compromise skills and qualities for someone who just feels right and your instincts shouldn’t be discounted. Allow them to inform your choices while thinking about the areas we’ve touched on to build the UX Dreamteam that will make your organization shine.

We Tried To Warn You, Part 2

by:   |  Posted on

A large but unknowable proportion of businesses fail pursuing nearly perfect strategies.

In Part I of We Tried to Warn You, three themes were developed:

# Organizations as wicked problems,
# The differences of failure leverage in small versus large organizations, and
# The description of failure points

These should be considered exploratory elements of organizational architecture, from a communications information architecture perspective. While the organizational studies literature has much to offer about organizational learning mechanisms, we find very little about failure from the perspective of product management, management processes, or organizational communications.

Researching failure is similar to researching the business strategies of firms that went out of business (e.g., Raynor, 2007). They are just not available for us to analyze, they are either covered-up embarrassments, or they become transformed over time and much expense into “successes.”

In The Strategy Paradox, Raynor describes the “survivor’s bias” of business research, pointing out that internal data is unavailable to researchers for the dark matter of the business universe, those that go under. Raynor shows how a large but unknowable proportion of businesses fail pursuing nearly perfect strategies. (Going concerns often survive because of their mediocre strategies, avoiding the hazards of extreme strategies).

A major difference in the current discussion is that organizational failure as defined here does not bring down the firm itself, at least not directly, as a risky strategy might. But it often leads to complete reorganization of divisions and large projects, which should be recognized as a significant failure at the organizational level.

One reason we are unlikely to assess the organization as having failed is the temporal difference between failure triggers and the shared experience of observable events. Any product failure will affect the organization, but some failures are truly organizational. They may be more difficult to observe.

If a prototype design fails quickly (within a single usability test period), and a project starts and fails within 6 months, and a product takes perhaps a year to determine its failure – what about an organization? We should expect a much longer cycle from originating failure event to general acknowledgement of failure, perhaps 2-5 years.

There are different timeframes to consider with organizational versus project or product failure. In this case study, the failure was not observable until after a year or so of unexpectedly weak sales, with managers and support dealing with customer resistance to the new product.

However, decisions made years earlier set the processes in place that eventuated as adoption failure. Tracing the propagation of decisions through resulting actions, we also find huge differences in temporal response between levels of hierarchy (found in all large organizations).

Failures can occur when a chain of related decisions, based on bad assumptions, propagate over time. These micro-failures may have occurred at the time as “mere” communication problems.

In our case study, product requirements were defined based on industry best practices, guided by experts and product buyers, but excluding user feedback on requirements. Requirements were managed by senior product managers and were maintained as frozen specifications so that development decisions could be managed. Requirements become treated as-if validated by their continuing existence and support by product managers. But with no evaluation by end users of embodied requirements – no process prototype was demonstrated – product managers and developers had no insight into dire future consequences of product architecture decisions.

Consider the requisite timing of user research and design decisions in almost any project. A cycle of less than a month is a typical loop for integrating design recommendations from usability results into an iterative product lifecycle.

If the design process is NOT iterative, we see the biggest temporal gaps of all. There is no way to travel back in time to revise requirements unless the tester calls a “show-stopper,” and that would be an unlikely call from an internal usability evaluator.

In a waterfall or incremental development process, which remains typical for these large-scale products – usability tests often have little meaningful impact on requirements and development. This approach is merely fine-tuning foregone conclusions.

In a waterfall or incremental development process, which remains typical for these large-scale products – usability tests often have little meaningful impact on requirements and development. This approach is merely fine-tuning foregone conclusions.

Here we find the seeds of product failure, but the organization colludes to defend the project timelines, to save face, to maintain leadership confidence. Usability colludes to ensure they have a future on the job. With massive failures, everyone is partly to blame, but nobody accepts personal responsibility.

The Roles of User Experience

Figure 1. Failure case study organization – Products and project timeframes. (View figure 1 at full-size.)

As Figure 1 shows, UX reported to development management, and was further subjected to product and project management directives.

In many firms, UX has little independence and literally no requirements authority, and in this case was a dotted-line report under three competing authorities. That being the case, by the time formal usability tests were scheduled, requirements and development were too deeply committed to consider any significant changes from user research. With the pressures of release schedules looming, usability was both rushed and controlled to ensure user feedback was restricted to issues contained within the scope of possible change and with minor schedule impact.

By the time usability testing was conducted, the scope was too narrowly defined to admit any ecologically valid results. Usability test cases were defined by product managers to test user response to individual transactions, and not the systematic processes inherent in the everyday complexity of retail, service, or financial work.

* Testing occurred in a rented facility, and not in the retail store itself.
* The context of use was defined within a job role, and not in terms of productivity or throughput.
* Individual screen views were tested in isolation, not in the context of their relationship to the demands of real work pressures – response time, database access time, ability to learn navigation and to quickly navigate between common transactions.
* Sequences of common, everyday interactions were not evaluated.

And so on.

The product team’s enthusiasm for the new and innovative may prevent listening to the users’ authentic preferences. And when taking a conventional approach to usability, such fundamental disconnects with the user domain may not even be observable.

Many well-tested products have been released only to fail in the marketplace due to widespread user preference to maintain their current, established, well-known system. This especially so if the work practice requires considerable learning and use of an earlier product over time, as happened in our retail system case. Very expensive and well-documented failures abound due to user preference for a well-established installed base, with notorious examples in air traffic control, government and security, medical / patient information systems, and transportation systems.

When UX is “embedded” as part of a large team, accountable to product or project management, the natural bias is to expect the design to succeed. When UX designers must also run the usability tests (as in this case), we cannot expect the “tester” to independently evaluate the “designer’s” work. The same person in two opposing roles, the UX team reporting to product, and restricted latitude for design change (due to impossible delivery deadlines) – we should consider this a design failure in the making.

In this situation, it appears UX was not allowed to be effective, even if the usability team understood how to work around management to make a case for the impact of its discoveries. But the UX team may not have understood the possible impact at the time, but only in retrospect after the product failed adoption.

We have no analytical or qualitative tools for predicting the degree of market adoption based on even well-designed usability evaluations. Determining the likelihood of future product adoption failure across nationwide or international markets is a judgment call, even with survey data of sufficient power to estimate the population. Because of the show-stopping impact of advancing such a judgment, it’s unlikely the low-status user experience role will push the case, even if such a case is clearly warranted from user research.

The Racket: The Organization as Self-Protection System

Modern organizations are designed to not fail. But they will fail at times when pursuing their mission in a competitive marketplace. Most large organizations that endure become resilient in their adaptation to changing market conditions. They have plenty of early warning systems built into their processes – hierarchical management, financial reports, project management and stage-gate processes. The risk of failure becomes distributed across an ever-larger number of employees, reducing risk through assumed due diligence in execution.

The social networks of people working in large companies often prevent the worst decisions from gaining traction. But the same networks also maintain poor decisions if they are big enough, are supported by management, and cannot be directly challenged.

The social networks of people working in large companies often prevent the worst decisions from gaining traction. But the same networks also maintain poor decisions if they are big enough, are supported by management, and cannot be directly challenged. Groupthink prevails when people conspire to maintain silence about bad decisions. We then convince ourselves that leadership will win out over the risks; the strategy will work if we give it time.

Argyris’ organizational learning theory shows people in large organizations are often unable to acknowledge the long-term implications of learning situations. While people are very good at learning from everyday mistakes, they don’t connect the dots back to the larger failure that everyone is accommodating.

Called “double loop learning,” the goal is learn from an outcome and reconfigure the governing variables of the situation’s pattern to avoid the problem in the future. (Single-loop learning is merely changing one’s actions in response to the outcome). Argyris’ research suggests all organizations have difficulties in double-loop learning; organizations build defenses against this learning because it requires confrontation, reflection, and change of governance, decision processes, and values-in-use. It’s much easier to just change one’s behavior.

What can UX do about it?

User experience/IA clearly plays a significant role as an early warning system for market failure. Context-sensitive user research is perhaps the best tool for available for informed judgement of potential user adoption issues.

Several common barriers to communicating this informed judgment have been discussed:

* Organizational defenses prevent anyone from advancing theories of failure before failure happens.
* UX is positioned in large organizations in a subordinate role, and may have difficulty planning and conducting the appropriate research.
* UX, reporting to product management, will have difficulty advancing cases with strategic implications, especially involving product failure.
* Groupthink – people on teams protect each other and become convinced everything will work out.
* Timing – by the time such judgments may be formed, the timeframes for realistic responsive action have disappeared.

Given the history of organizations and the typical situating of user experience roles in large organizations, what advice can we glean from the case study?

Let’s consider leveraging the implicit roles of UX, rather than the mainstream dimensions of skill and practice development.

UX serves an Influencing role – so let’s influence

UX practice must continue to develop user/field research methods sensitive to detecting nascent problems with product requirements and strategy.

User experience has the privilege of being available on the front lines of product design, research, and testing. But it does not carry substantial organizational authority. In a showdown between product management and UX, product wins every time. Product is responsible for revenue, and must live or die by the calls they make.

So UX should look to their direct internal client’s needs. UX should fit research and recommendations to the context of product requirements, adapting to the goals and language of requirements management. We (UX) must design sufficient variability into prototypes to be able to effectively test expected variances in preference and work practice differences. We must design our test practices to enable determinations from user data as to whether the product requirements fit the context of the user’s work and needs.

We should be able to determine, in effect, whether we are designing for a product, or designing the right product in the first place. Designing the right product means getting the requirements right.

Because we are closest to the end user throughout the entire product development lifecycle, UX plays a vital early warning role for product requirements and adoption issues. But since that is not an explicit role, we can only serve that function implicitly, through credibility, influence and well-timed communications.

UX practice must continue to develop user/field research methods sensitive to detecting nascent problems with product requirements and strategy.

UX is a recursive process – let’s make recursive organizations as well

User experience is highly iterative, or it fails as well. We always get more than one chance to fail, and we’ve built that into practices and standards.

Practices and processes are repeated and improved over time. But organizations are not flexible with respect to failure. They are competitive and defensive networks of people, often with multiple conflicting agendas. Our challenge is to encourage organizations to recurse (recourse?) more.

We should do this by creating a better organizational user experience. We should follow our own observations and learning of the organization as a system of internal users. Within this recursive system (in which we participate as a user), we can start by moving observations up the circle of care (or the management hierarchy if you will).

I like to think our managers do care about the organization and their shared goals. But our challenge here is to learn and perform from double-loop learning ourselves, addressing root causes and “governing variables” of issues we encounter in organizational user research. We do this by systematic reflection on patterns, and improving processes incrementally, and not just “fixing things” (single-loop learning).

We can adopt a process of socialization (Jones, 2007) rather than institutionalization, of user experience. Process socialization was developed as a more productive alternative to top-down institutionalization for the introduction of UX practices in organizations introducing UX into an intact product development process.

While there is strong theoretical support for this approach (from organizational structuration and social networks), socialization is recommended because it works better than the alternatives. Institutionalization demands that an organization establish a formal set of roles, relationships, training, and management added to the hierarchy to coordinate the new practices.

Socialization instead affirms that a longer-term, better understood, and organizationally resilient adoption of the UX process occurs when people in roles lateral to UX learn the practices through participation and gradual progression of sophistication. The practices employed in a socialization approach are nearly the opposite (in temporal order) of the institutionalization approach:

# Find a significant UX need among projects and bring rapid, lightweight methods to solve obvious problems
# Have management present the success and lessons learned
# Do not hire a senior manager for UX yet, lateral roles should come to accept and integrate the value first
# Determine UX need and applications in other projects. Provide tactical UX services as necessary, as internal consulting function.
# Develop practices within the scope of product needs. Engage customers in field and develop user and work domain models in participatory processes with other roles.
# Build an organic demand and interest in UX. Provide consulting and usability work to projects as capability expands. Demonstrate wins and lessons from field work and usability research.
# Collaborate with requirements owners (product managers) to develop user-centered requirements approach. Integrate usability interview and personas into requirements management.
# Integrate with Product Development. Determine development lifecycle decision points and user information required.
# Establish User Experience as process and organizational function
# Provide awareness training, discussion sessions, and formal education as needed to fit UX process.
# Assessment and renewal, staffing, building competency

We should create more opportunities to challenge failure points and process breakdowns. Use requirements reviews to challenge the fit to user needs. Use a heuristic evaluation to bring a customer service perspective on board. In each of those opportunities, articulate the double-loop learning point. “Yes, we’ll fix the design, but our process for reporting user feedback limits us to tactical fixes like these. Let’s report the implications of user feedback to management as well.”

We can create these opportunities by looking for issues and presenting them as UX points but in business terms, such as market dynamics, competitive landscape, feature priority (and overload), and user adoption. This will take time and patience, but then, its recursive. In the long run we’ll have made our case without major confrontations.


The best we can hope to bat is .500. If you’re getting better than that, you’re not swinging for the fences. Even Barry bonds, steroids or not, is not getting that. We need to celebrate failure.

Scott Cook, Intuit’s Founder, famously said at CHI 2006: “The best we can hope to bat is .500. If you’re getting better than that, you’re not swinging for the fences. Even Barry bonds, steroids or not, is not getting that. We need to celebrate failure.”

Intelligent managers actually celebrate failures – that’s how we learn. If we aren’t failing at anything, how do we know we’re trying? The problem is recognizing when failure is indeed an option.

How do we know when a project so large – an organizational level project – will belly-up? How can something so huge and spectacular in its impact be so hard to call, especially at the time decisions are being made that could change the priorities and prevent an eventual massive flop? The problem with massive failure is that there’s very little early warning in the development system, and almost none at the user or market level.

When product development fails to respect the user, or even the messenger of user feedback, bad decisions about interface architecture compound and push the product toward an uncertain reception in the marketplace. Early design decisions compound by determining architectures, affecting later design decisions, and so on through the lifecycle of development.

These problems can be compounded even when good usability research is performed. When user research is conducted too late in the product development cycle, and is driven by usability questions related to the product and not the work domain, development teams are fooled into believing their design will generalize to user needs across a large market in that domain. But at this point in product development, the fundamental platform, process, and design decisions have been made, constraining user research from revisiting questions that have been settled in earlier phases by marketing and product management.


Argyris, C. (1992). On organizational learning. London: Blackwell.

Howard, R. (1992). The CEO as organizational architect: an interview with Xerox’s Paul Allaire. Harvard Business Review, 70 (5), 106-121.

Jones, P.H. (2007). Socializing a Knowledge Strategy. In E. Abou-Zeid (Ed.) Knowledge Management and Business Strategies: Theoretical Frameworks and Empirical Research, pp. 134-164. Hershey, PA: Idea Group.

Raynor, M.E. The strategy paradox: Why committing to success leads to failure (and what to do about it). New York: Currency Doubleday.

Rittel, H.W.J. and Weber, M.W. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155-169.

Taleb, N.N (2007).The Black Swan: The Impact of the Highly Improbable. New York: Random House.