IA Summit 10 – Whitney Hess Keynote

by:   |  Posted on

IA Summit 2010

This year marks the 11th annual Information Architecture Summit. Our theme is meant to inspire everyone in the community—even those who aren’t presenting or volunteering—to bring their best ideas to the table.

As busy practitioners, we rarely have the chance to step back and think about the future of our field—we’re too busy resolving day-to-day issues. By gathering and sharing practical solutions for everyday challenges, we can create more breathing room to plan for what’s to come.

Subscribe to the Boxes and Arrows Podcast in iTunes or add this page to your Del.icio.us account:

iTunes     Del.icio.us     2010 IA Summit theme music generously provided by Bumper Tunes

Keynotes

| “Day 1 – Dan Roam“:http://boxesandarrows.wpengine.com/view/ia-summit-10-dan | “Day 2 – Richard Saul Wurman“:http://boxesandarrows.wpengine.com/view/ia-summit-10-richard | Day 3 – Whitney Hess |

Full Program

| Day 1 | Day 2 | Day 3 |

In her keynote closing the 2010 IA Summit, Whitney asks if our work is just our job or our passion. To really make the difference we seek, our practice needs to be our calling. The UX community is united because of a common mission:

We empower people to become self-reliant and more resourceful, organized, social, and relaxed. We don’t do it for them, they do it for themselves.

Most designers don’t make the things that people use; we’re ideas people. We’re nothing without the visual designers, developers, copywriters, and business managers with whom we work. It’s time we focus our energy outside the UX tribe, to these people that bring our ideas to fruition.

Ms. Hess implores us to stop feeling so disenfranchised and misunderstood, to stop isolating ourselves and strive for the influence to bring about the change that drives us to do what we do. Defending “perfection” will defeat us; instead, she calls us to have the audacity to fail spectacularly and then move upward from those moments.

In the end, she asks us to consider our legacy. Do we want to be a footnote in a technology textbook, or do we want to “cross the chasm” and embody the change that we talk about amongst ourselves.


Download mp3

These podcasts are sponsored by:

MAD*POW logo
At Mad*Pow, they leverage the disciplines of Human Factors, Psychology, and Visual Design to create engaging experience that maximize customer acquisition, increase attention, and reduce costs.

ASIS&T logo
The American Society of Information Science & Technology: Since 1937, ASIS&T has been THE society for information professionals leading the search for new and better theories, techniques, and technologies to improve access to information.

IA Summit 2010
The IA Summit: the premier gathering place for information architects and other user experience professionals.

The design behind the design
Boxes & Arrows: Since 2001, Boxes & Arrows has been a peer-written journal promoting contributors who want to provoke thinking, push limits, and teach a few things along the way.

Contribute as an editor or author, and get your ideas out there. boxesandarrows.com/about/participate

Transcript of Whitney Hess Closing Plenary Address of the 2010 IA Summit in Phoenix, Arizona.

Announcer: In the closing plenary of the 2010 IA Summit, Whitney Hess discusses her experiences within the IA community, and calls for greater inclusion, leadership, and the necessity to embrace failure as a fundamental aspect of our discipline’s growth, without which we cannot get a seat at the board room table.
I hope everyone enjoys the podcast. Cheers.
Whitney Hess: Before I jump in, I just want to take a moment to thank all of you, to thank each and every one of you, for being here in this room right now. I know there are a lot of other places that you could have been. I know that you could have booked your flight to leave already, and so it means a great deal to me that you’re here right now and allowing me to speak to you. So thank you.
Let’s begin at the beginning. The truth about me, just me, and all of my vulnerability. That might be what Jennifer was talking about a little bit.
I’m amazed at the position that I find myself in today. I know I’ve said this to you, each of you individually, perhaps, or you may have seen me write about this. Standing in front of you here today is quite frankly, incomprehensible to me. It’s a complete out of body experience. It doesn’t even feel like I’m living it.
When Jennifer Bombach and Livia Labate contacted me and they invited me to give this closing plenary, I just sat down on my couch and I cried, and that’s the honest truth. I just cried.
So for those of you who don’t know me, last year’s IA Summit was my first ever public presentation. Sure, I had given presentations to clients and colleagues and whatnot, and shown my deliverables in presentation form, but I’d never stood on a stage by myself until a year ago right now.
The year before that, the IA Summit in 2008 was my first ever summit. So you can get a better picture, maybe, knowing this, of why this moment is just so insane for me. These are the past few IA Summit closing plenaries. What the fuck?
[laughter]
I think it goes without saying, but I’ll say it anyway, that the people who have held this position prior to me have been considerably further along in their careers and have contributed far more to this community than I have so far.
So, for the folks who have an issue with me standing up here today, I don’t need you to tell me how green I am, okay? I’m 27 years old. I graduated from school a little over five years ago. I became self‑employed a little over a year and a half ago. So, yeah, I’m not nearly as experienced as pretty much everyone else in this audience. I’m probably not as smart as you, either. But I’m standing here, so move on with your life.
[Laughter, cheering]
Thank you.
Synchronicity brought me here and I am not unclear enough that I don’t realize this. I didn’t plan this. I didn’t even particularly want this for myself in my career. But a remarkable series of events has landed me in this position. Some of it with luck, some of it was just my personality, some of it was timing being in the right place at the right time. I realized all of those things but that is life and you never know what is going to happen so here I am.
I’ve been on this incredible journey. There’s a huge gap between who I am and who I want to be and the journey between those two points is what I consider to be my pursuit of happiness. Two and a half years ago, I identified something in myself that I didn’t like. I was closed off and my inner introvert was preventing me from living to the fullest. And I could already tell even a few years out of school that it was going to negatively effect my career. So, I reached out to this community slowly but surely and my life changed forever.
The price of sheep is boredom. The price of being a wolf is loneliness. Choose one or the other with great care. I made this conscious decision to no longer be a sheep, but I never expected that the alternative would mean being a wolf. I have felt lonely at times. I am not going to lie. I’ve gotten attention that I wasn’t really comfortable with. And it made me feel different than I had felt in my life previously. But it may have felt lonely, but I have never felt alone.
Every wolf has her pack and this is mine, the tribe, our tribe. Look around you. You are not alone. You will not find another profession in which the community of practitioners is disconnected, is this loving, is this dedicated to each other’s success and happiness. We are truly each other’s kindred spirits. There are very few egos in this community and I discovered that two years ago, I was at the interaction conference formed by AXTA in 2008. It was their first conference in Savannah.
I had booked my ticket to leave a little late so the conference would’ve ended and I was hungry and I decided to send out a tweet to see if anyone would grab a dinner. And I got a response from someone named @mediajunkie. Now, I had kind of met him at the conference but I didn’t entirely know who he was and in his reply, he said that he and @cb were going to take a walk and grab some dinner and I was more than welcome to join them.
So I said yes and then I went on Google to figure out who these people were because I felt so new and I didn’t want to embarrass myself. And so I discovered that @mediajunkie was Christian Crumlish who is the curator of the Yahoo design pattern library. And @cb is Chris Baum, the editor‑in‑chief of Boxes and Arrows. So, after I crapped myself and then pulled myself together, I was like, “Okay, I guess this is it. I am going to go out and have dinner with this people.” We took a walk around Savannah.
They asked me about me. They asked me what it was like to be involved with the community for the first time. They asked me some opinions about being a younger practitioner and things that I see in the community or things that I don’t see in the community because of that and the rest is history. They were so warm and treated me with such equality that I never expected, that it really had a profound impact on me.
Chris Baum’s not in the room today. Christian Crumlish is, and I’m sorry to single you out. I have a lot of stories that are very similar with a lot of people in this room, and I just felt like that was the beginning, and I really owed it to them to tell you all publicly.
So people have asked how I have gotten here in such a short period of time. The only answer I have is this. This is a community of giants and I have felt like I have stood on all of your shoulders, and I’m eternally grateful for that. So I have to ask, “Who’s your mentor, and who calls you their mentor?”
I grew up feeling like I didn’t really have a lot of role models. I was kind of a loner. I was very self‑reliant. I didn’t exactly respect my elders. Now I find myself surrounded by a bunch of inspirational people, and I’m lucky to call many of them my mentors.
But what’s forced me to grow even more has been through sharing my experience and by mentoring others and giving my time to other people’s growth.
It’s very rewarding and it’s very humbling, and I feel that it’s being on both ends of the equation, having the mentor and being the mentor that has given me this incredible sense of belonging.
The prolific Peter Drucker wrote, “Knowing where one belongs can transform an ordinary person, hard‑working and competent but otherwise mediocre, into an outstanding performer.” Because of this tribe, my life has become outstanding.
Everything that I do, and all of my motivations are from a single point, and that is love. Because it’s all about love, and love is the differentiator, and love is everything, and love is why I’m here right now.
This is my mantra that I live by. Do what you love in the service of people who love what you do. It was written by a man named Steve Farber in his book, “The Radical Leap.” And when I saw it on the page, it leaped up at me and I realized that this is me. This is how I want to live my life.
Everything that I do springs from love or I don’t do it, quite frankly. I serve a higher purpose than myself. It’s about affecting other people’s lives, and ensuring their appreciation, and getting that feedback, and making sure that it’s a cyclical process, and that’s why this statement means so much to me.
So I have to ask you. Let’s be honest. Is this your job, or is it your passion? Are you passionately driven to make a difference here? Is that why you’re here? Why do you care, and why are you here in the first place? There are so many other places you can be, and there are so many other professions that you can have. Why this one?
There’s work, and then there’s life’s work. I think what we do is pretty hard to call work. I think our passion is so deep and so persistent that it’s much more accurately classified as “life’s work” because frankly, this isn’t a job. It’s a life calling. I’m not doing this to please my parents or to follow in their footsteps. I’d be pretty surprised if any of you had parents that did this, and that’s why you got in the profession in the first place.
[laughter]
I’m not doing this to become rich, though that would be nice. I do this because I have this unwavering internal impulse that this is what I meant to be doing for a living. And it is not a living, it is life. This is what I meant to be doing. And yet, for many of us, the job description doesn’t begin to describe what we do or what you can do.
So, everything that you can do and what your job description says. We all have different job titles and different responsibilities but we are united for a single reason. And that is what I consider our common mission. Now, I’ve struggled with how to communicate this. It may not feel right at first but hear me out. I believe that our common mission is that we help people and has their own lives. And I chose this language very carefully. Thank you. Thank you.
We empower people to become self‑reliant, to become more resourceful, more social, more informed, more organized, more successful, more relaxed. But we don’t do it for them. They do it for themselves. We design systems that allow people to make those choices and make those changes in their life. Let us be honest. Our work isn’t that momentous. We believe in the power of the butterfly effect. That if we make small changes, that they will have enormous impact.
We need that to operate in order for us to be successful. We dream of causing this wide‑ranging positive change. I know that we all do but really, it’s because of something so much deeper and that’s we want to change the world. And I really, really believe that everyone of you is here because you want to be part of something that is bigger than yourself. You want to change the world and maybe not the world but your world whatever it is that you defined it as. I defined it as something different than you may… Each of us may define it differently but it’s about having that really wide impact.
Not just on the users that we advocate for but on a much greater purpose. It is not just about creating more usable, useful desire of products. That might be your tag line. We like to communicate that but it really is something much bigger and I think that many of you know that there is something bigger here. So, there is this line in Sister Act two that just popped into my head. If you want to be somebody, if you want to go somewhere, you better wake up and pay attention.
And I think we do have to wake up. We can’t possibly have the lights matter of fact that we dream of if we’re only relying on the other members of our tribe. The world isn’t just going to change for us because we think it should. We are going to become increasingly disappointed with our progress if we just keep doing all of these back slapping that we’ve become so accustomed to. Now, I don’t mean that as negatively as it comes across because I think that the bond that exists in this community is unparalleled. And it is important that we support each other.
But there is something much bigger going on. And I think we are missing the big picture. So, the wakeup call, real world is calling, time to pick up. It is not about you or us especially not about us. We are not the most important part of the equation here. We design systems that enable people to have more successful behaviors, but we do not make them. We do not make them. That is the dirty truth. We are not creators. We’re influencers. We don’t actually make anything. Hugh McLeod said, “Like so many brilliant people, we believe that ideas move mountains. But bulldozers move mountains. Ideas show where bulldozers should go.”
We’re ideas people, and we’re nothing, nothing without the visual designers, developers, copy writers and business managers that we work with.
[applause]
They’re the makers. They’re the ones who bring our ideas to fruition. Their creations are ultimately what our users use, not our creations. They’re not ours. And we’re really doing a really shitty job of showing them that we understand that.
So here’s my plea. I think it’s time that we need to need to focus our energy outside of the UX tribe to become recognized leaders to makers and management. We really need to grow up and stop feeling so disenfranchised and misunderstood, and recognize that the real role that we play in all of this. We need to reach out to the greater tech and business communities far more than we have been, and we have really completely isolated ourselves, and it’s time that we stopped doing that.
So what we have is influence. What we have is influence, and we need to figure out how to gain that influence over our companies, our colleagues, our clients and the larger communities that we work within. But this the hard part, and we don’t have a choice. If we want this profession to exist in the next ten years, we’re going to have to get used to doing some seriously hard work. Winston Churchill said that the price of greatness is responsibility, and I think we’ve been shirking our responsibility in a lot of ways. We’ve been shirking the responsibility that it takes to change the world.
We are exasperated by the lack of understanding and common sense of the people that we work with, the way our users are mistreated, and we are so focused on fixing these immediate problems that we recognize that we neglect the bigger picture. And we’re all responsible for that. That’s how I see us.
[laughter]
And I say “us” because I’m just as guilty of it as anyone else.
So what’s holding us back? What is holding us back?
This is the 11th year of the IA Summit, and I’d venture to guess that the folks who were here on year one, they thought we’d be much further along by now.
So what’s holding us back? I have a few ideas.
Firstly, I think we heroize the tools. And this is something that I was actually surprised to hear coming up in a lot of sessions this weekend. I wrote this beforehand, and I didn’t realize that it was something that a lot of people were feeling.
Omnigraffle. Axure. Post‑It notes. White boards, and card sorting, and Ensel models, and we use all these tools, and I use them too.
I rely on them just as much as everyone else does, but we spend so much energy promoting the tools, and so much energy promoting our use of them that we neglect to feature the people that are using them, and that’s us.
A fancy tool is just a pillar to hide behind and putting the focus on the tool acting like they are these magic items that allow us to do our job. It gives the tool more power than you. The tool doesn’t have the power. It is what you do with it. It is the understanding that you create by using it, it is your thinking not the tools capability.
Doctors don’t give this kind of praise to their stethoscopes so why are we spending so much time talking about the things that we use to do our work? Secondly, I think we limit ourselves. A good friend told me don’t judge yourself, the world will do that for you. And it’s really hard for me to internalize but it’s true. Sure, most of us don’t have MBAs, most of us don’t have MFAs, most of us don’t have any formal education in this at all. But that doesn’t mean that we don’t deserve to have a sit at the war room table.
We accommodate out of guilt. We attack out of anger and we avoid out of fear.
When we present our work, we are in absolute mess. Either we let our teams roll right over us or we slip out when they don’t understand our logic and our thought process so worst of all, we completely avoid true collaboration because we are afraid of people taking away the little power that we have. I also think we are afraid to be wrong. I think somehow we got into our heads that we always have to have the right answer and maybe because we convinced our designers and or developers that they need our help and that is where this comes from.
And we feel that with every wrong answer, our future hangs in the balance and that is simply not true. You screw up every day and everyone already knows it. But do you admit it? We are human and we are wrong all the time. Everyone is but most of us don’t own up to it. I mean how often really do you proclaim your fuck ups? I mean something that Jerry was just talking about. Do you celebrate them? Probably not. I know I don’t. So, I think it’s time that we celebrate fear.
Fear is good. Fear is great. Fear is growth. Fear means you are doing something right. If we aren’t tackling the things that scare us the most, then we aren’t striking ourselves enough. Love hurts. We are in love with what we do and we have this calling. Aren’t we lucky? But no one said it would be easy. So, if you are finding it easy, you probably aren’t doing enough. Love hurts. It’s suppose to hurt because it matters. And that’s the honest truth.
So, living pursuit of the OSM. That is “Oh Shit Moment.” This also comes from the C Fiber book that have a big impact on me called: “The Radical Leap”. Seek out the experiences that will really make you scream, “Oh, shit!” It gives you be visceral like when you take that glass elevator of the hotel from the top floor down to the lobby and it drops and you feel your stomach go, that’s the feeling that you should get. It should have magnitude and there should be a high likelihood that you’ll fail. And when you do fail, fail magnificently.
Fail big. Like really fucked up. Okay? Because only then that you are going to know that you are living. The bigger the ocean moment, the bigger the failure but at least you’ll know that you are taking your responsibility seriously. Not doing it when you know full well you had the opportunity that hurts far more than any failure and I think a lot of you can relate when I say that my biggest regrets in life are not the things that I did but the things that I didn’t do.
The intersection of failure, thinking and determination: a success layer. Who doesn’t believe that’s true? You can’t succeed without failure, and we’re a lucky bunch, because we have oodles of determination and oodles of brainpower.
Perfection doesn’t equal credibility. I think a lot of us get stuck up in this mindset that it does, and it’s just not true. It’s critical that we believe this. It’s critical that we believe that perfection doesn’t equal credibility. The more perfect you try to appear to your teams, the less they’re going to trust you.
Right now, this is my biggest “Oh, shit!” moment. The last few months have been absolute Hell for me.

[laughter]

I’ve probably given myself an ulcer, and I’ve definitely driven my boyfriend crazy. But my sense of responsibility to do this, to our shared mission, far outweighed my personal fear. That’s why I’m here.
I think another thing we really need to do is promote and include. Some people protect themselves from fear by surrounding themselves in an exclusive world. The less they have to share themselves, the less they risk. We keep trying to convince ourselves that we’re special. But, if we keep this tribe so isolated, we’re going to lose.
Recognize the destructive impact of your words in dealing with others, especially your colleagues, the makers and the builders that lift us up, who bring our ideas to fruition. Beware of the language that you use in imparting your criticisms.
Yes, they may have created something that’s hard to use, or that isn’t useful for your specific users, or is just plain weird. They’re not necessarily going to have the right answers, either. But instead of feeling the need to prove them wrong, teach them how to be right.
If you try to dominate people, you’re already defeated. We like to think of ourselves as saviors. I think we have the sense that we’re this organized and very committed group of people, and very headstrong, certainly. But we don’t always have to prove our value. We’ve been undervalued for so long that I think we’ve got this chip on our shoulder, and it’s really time to put that to bed.
We need to be assertive without being aggressive, and there’s a big difference between the two. Being assertive shows confidence. Being aggressive shows insecurity. So, be aware of not just what you say, but how you say it.
This is one of my favorite quotes by Peter Drucker, who wrote many tomes on leadership and management: “Your intellectual arrogance is causing disabling ignorance.”
What that means to me is, that by thinking we’re the most important piece of the puzzle, and by failing to gain knowledge in other areas, like programming or visual design or business management, we’re just further isolating ourselves. If we want them to better understand us, we need to do a lot more to better understand them. Because all human beings have the basic need to be recognized.
We shouldn’t think of ourselves as saviors with all the answers. We need to be better facilitators, and ultimately, leaders. We need to lead. Now, this leadership stuff is tricky. It’s very amorphous. I realize that, and I can’t tell you all to go out and be leaders, and suddenly the user‑experience field gains recognition and standing in the business world. I know that.
But I can only tell you what leadership means to me and hopefully that sparks something in you. Firstly, it means having Chutzpah. For those of you don’t know what Chutzpah means, it is essentially audacity. It means being able to walk into the room and say exactly what needs to be said. No letting fear hold you back.
We need to have the courage of our convictions and knock back down when we get pushed showing our coffins through the consistent behavior, not wavering. We need to have adaptability. We need to be able to roll at the punches and try new things and adapt to the circumstances and the people that we are surrounded by. Not having a fear of newness. And when all those fails, we have to act as if…Don’t expose the people that you need to be leading to your insecurities and your worry.
Act like the leader that you want to be and suddenly you will become it. Ultimately, the true mark of a leader is someone who inspires others to lead and that is what we need the most. Your Chutzpah, your conviction, your adaptability will become contagious and that’s when real changes going to happen. So, our legacy… It concerns me greatly. So, some say they have twenty years experience when all they really have is one year experience repeated twenty times and all that age. Each of us has to force ourselves to move beyond the supportive tribe that we found ourselves in.
To become business leaders ‑‑ in the face of fear, lead the people that we work with, the people that we collaborate with in order to achieve our common mission. We keep focusing on our site maps and our wire frames and our prototypes like that is all there is. Then we aren’t really advancing the profession as a whole. So, I have to ask you a tough question. Can your company succeed without you? And if the answer is no, are you proud of that? You have to ask yourself if your DNA, the shared belief system that we all hold so dear, the things that you have heard in the past weekend is that coursing through the veins of your company?
Are you just doing stuff or are you transforming stuff? So, what is our legacy? Is it outside this room? I love this room. But it is not enough for me. Let’s not be a footnote on technology textbook. I don’t want to look back thirty years from now and when a really talented, really brilliant, caring group of people almost change the face of technology forever but just couldn’t figure out how to cross the line. Our legacy rest on all of our shoulders. Thank you for listening.
[applause]

Research Logistics

by:   |  Posted on

With more companies today putting a stronger emphasis on gaining a deeper understanding of their customer, it’s not unusual for us to be called in for a project to find that our clients don’t have a lot of experience with research and don’t know what to expect. This article is for every designer, architect, manager, engineer, and stakeholder who wants to know more about research and is intended to provide you with the most critical tools for interacting with researchers and understanding how the work that we do can make your job easier.

This article will also outline what to expect from researchers and some ways to recognize when you’re working with a good one. These are indicators, not standards, based on what we’ve found to be effective. There are many ways to do research and every research study is different so it doesn’t mean that a researcher is incompetent if he or she doesn’t conform to these indicators. One sign of a strong researcher is that he or she will educate you throughout the process so that you know what to expect. With that in mind this article is ultimately intended to provide a useful starting point.

Recruiting

One of the most critical and time-consuming elements of test preparation is defining the right target audience and recruiting participants. Participant recruiting is usually conducted by professional recruiters who typically consult databases of potential participants. Sometimes researchers will do the recruiting themselves, but it’s usually more cost effective to use a specialist.

Recruiting will almost always take two weeks or more depending on the number of participants and the type of research, so make sure that you get started early enough for the recruiter to have enough time to find the appropriate participants for the study. Recruiting for phone interviews may take slightly less time and any kind of home visit will likely take longer (ethnography or contextual interview). Your researcher should be able to provide you with an estimate at the time of initial engagement.

A week for recruiting tends to be difficult and any less than that is pretty much unthinkable. Short-changing the recruiting could result in participants that don’t properly fit the target market segment, don’t provide quality feedback, or just don’t show up at all. All of these can have a negative impact on the data. Even if it is possible to get participants faster, it’s usually better to take the time to ensure that you are getting the right people. Your researcher should know all of this and recruiting participants is where he or she will start after getting a basic understanding of your product and schedule.

A recruiter will need a screener to get started. A screener is a description of the target user with open and close-ended questions about the participant that will help the recruiter to select the right people. What you can do to smooth the process along is to have a prepared concept of your target user. This does not need to be a full market research report—just an outline of the types of users that will use your product.

Your researcher should dig deep with questions that include more than demographic information by asking behavioral questions. Behavioral questions can include such topics as TV watching behavior, purchasing behavior, internet use, etc. Typically behavioral questions will give you a stronger understanding of those who are being recruited than demographics alone. These are important elements of market segmentation that are sometimes organized into profiles called personas.

Personas are useful because they create a consistent concept of the intended market segment that can guide the design process through multiple iterations. Personas can also be adjusted following deeper discovery research, such as in-depth interviews, as more information about the intended user comes to light. Within a few days, the researcher should present a screener that includes behavioral questions as well as demographics.

Scheduling

When creating a schedule for data collection, the researcher should know that you cannot run participants back to back. It’s generally not feasible to squeeze in 8 one-hour sessions in a single day, because of all of the activity that must occur between sessions. In an eight hour day, a researcher can perform four (maybe five) one-hour sessions but any more than that will take more time. Here are the reasons why:

One-hour sessions rarely go exactly one hour, some are shorter and quite a few will run longer. This can be due to a variety of reasons such as the product malfunctioning, the participant arriving late, or the participant providing lots of feedback. My rule of thumb is to allocate 50% of the session length as a buffer between sessions to allow for overrun, not including time needed to set up for the next session.

For sessions at an office or lab, some participants will arrive 10-20 minutes early, at which time they will need to use the restroom, sign NDAs and consent forms, and generally get comfortable. Comfortable participants give useful feedback, while uncomfortable participants tend to clam up and provide short, unemotional responses.
The researcher needs to set up and get ready. For usability or experience testing, the test will need to be reset, notes and documents need to be filed and new ones prepared. For any kind of home or location visit, the researcher will need to pack up all equipment and travel to the new location and set up equipment again.

Thus for every one-hour usability or experience testing session, there’s forty-five minutes to an hour of buffer and setup time. Home visits can take much longer.

Test Plan

A test plan should take no more than a week to develop and the researcher should give it to you for review and approval before being finalized. The test plan should specify the research and business goals associated with the project. During this period the researcher will need a significant amount of time with the product, either with a prototype or available concepts, while writing and checking the test plan. The better the researcher understands the intended final product, the more valuable the information he or she can get from the participant.

For usability or experience testing, the researcher will test the tasks with the product prior to a pilot test. He or she will need to make sure that there are no glitches, no unexpected areas under construction, and nothing giving away future tasks when performing each of the tasks with the product. With that in mind, it’s important to give the researcher a stable product or prototype and avoid drastic changes to the product prior to the test.

You should receive a well-written and organized test plan that details each research question and how it will be addressed. For usability testing this will include a list of tasks, what each task is intended to examine, approximate wording for the task (avoiding leading language), and detail on how each task will be scored or evaluated. For discovery research, it will include a list of topics to be addressed such as processes, environment and context, and expected pain points and needs.

Data Collection

When the data collection starts, it’s important to let the moderator work. During this time, the participant should feel comfortable enough to open up and provide honest feedback. In order to do this, it’s important to try to minimize observer impact during the testing session.

If you don’t have a separate place to watch the session (e.g. behind a two-way mirror or through a video feed), don’t make it obvious that you are paying close attention. Think about bringing in a laptop during the session to make it look like you’re doing other work. One way of doing this is telling the participant that you are also a researcher but you’re just going to be taking notes.

When you’re observing, remain objective and don’t make judgments based on one or two participants. It’s not uncommon to see a couple participants have a completely opposite reaction to a product compared to ten other participants. The researcher’s job is to sort through all the noise and report the real trends in the research. Take what you see with a grain of salt and listen to your researcher.

At the same time, it’s important to try to observe as many sessions as possible and give your researcher feedback between sessions if there are certain aspects of the user experience you want to know more about. The researcher should put the participant at ease and extract a great deal of information, including details that might have been overlooked or emotions that the person experiences. Different researchers will tend to achieve this in different ways as everyone has their own style, but you’ll notice by paying attention to the participant and seeing if they feel relaxed or nervous throughout testing.

Findings

Frequently, stakeholders will want to make immediate changes to a design, product, or prototype and won’t have the time to wait for the researcher’s final report. People have schedules that need to be met so it’s understandable that a project can’t always wait for the final report but the researcher should be able to provide you with quick findings within 24 hours of the last session.

For usability research, these quick findings should consist of a couple of short paragraphs including problems in the interface, possible solutions to these problems, and participants’ general reactions to the product, its look and feel, and expected usage. For ethnography or other forms of discovery research quick findings will tend to consist of expected usage of the product, expected value, high and low value features, and general trends about the intended user. Quick findings aren’t comprehensive and come before the researcher can get a complete look at the data, but it will provide you with the overall themes from the study.

When you do get the final report, make sure you take a look at it. It will tell you two things:
* Detailed findings regarding the interface, product, features, and intended user
* The quality and clarity of the report will tell you quite a bit about the quality of your researcher.

There’s one other thing to keep in mind when you are processing the findings from a usability test. The participants will tend to focus on the more obvious problems with a product or interface. There could be other, smaller or more abstract problems that are not identified in the first pass of usability testing. It’s usually a good idea to perform another test on the product after making changes to ensure that the changes you made were effective and identify any additional issues.

Summary

In summary, here are the most important points for non-researchers to know about the research process:
* Recruiting will almost always take two weeks or more.
* For every one-hour usability or experience testing session, there’s forty-five minutes to an hour of buffer and setup time, home visits can take much longer.
* The researcher will need a significant amount of time with the product (prototypes or concepts) while writing and checking the test plan.
* Try to minimize your impact during the testing session.
* Remain objective and don’t make judgments based on one or two participants.
* Ask your researcher to provide you with quick findings within 24 hours of the last session.

Any comments, feedback, or suggestions are very much appreciated.

IA Summit 09 – Plenary

by:   |  Posted on

Download   Watch the video

iTunes     Del.icio.us     IA Summit theme music created and provided by BumperTunes™

IA Summit 2009 logo

IA Summit 2009 Podcasts

The IA Summit was held in Memphis, TN from March 20-22. Boxes and Arrows captured many of the main conference sessions (see schedule).

| Preview | Keynote | Day 1 | Day 2 | Day 3 | Closing Plenary |

The IA Summit Closing Plenary

Jesse James Garrett delivers a passionate closing plenary at the 2009 IA Summit in Memphis, TN.Jesse James Garrett is a noted figure in the IA community, not only for his ground breaking book Elements of User Experience, but for the essay that galvanized the community in 2002, IA Recon.

In this IA Summit Closing Plenary, given without slides while wandering amidst the audience, Jesse examines what he has learned at the conference, he thoughts on the nature of the discipline and the practitioner, and gives bold, perhaps even shocking advice for the future direction of information architecture.

The following is an outline of some of his key points; please download the audio or watch the video for the complete experience.

Looking Back

Jesse revisits the turbulence of the first IA Summit in Boston, lamenting that he does not see this same turbulence in the IA community right now. Warning that “the opposite of turbulence is stagnation,” he looks back at the Great Depression and compares our grandparent’s feelings of scarcity to the community’s continued reliance on categorization in its various guises (e.g. taxonomy, thesauri, etc.) for its identity.

Moving On

Thanking IA leaders and the organizations that have nurtured Information Architecture, he declares that it is time to move on from the past. Leaders in IA, including himself, are notable based upon what they say about their work, not by their actual work and asks, “Do you know good IA when you see it?”

He is surprised that we don’t have schools of thought around IA. We have many ways to talk about our processes, but not about the “product of our work, a language of critique.” Until we can talk about the qualities of IA, we cannot judge the quality of the work.

No Information Architects

One of the desires of the IA community is to command respect. However, the overall value will take time to manifest itself, only reaching critical mass when “someone from this room” ascends to be CEO of an organization and creates a culture that respects the user to decimate the competition.

Jesse then puts forth his declaration that Information Architects and Interaction Designers do not exist. “There are, and only ever have been, User Experience Designers.”

He continues by breaking down UxD, examining how each element implied in the title illuminate his hypothesis – that the ephemeral and insubstantial CAN be designed independent of medium and across media. The web is just clay, he implores, and we can use many materials to create experiences.

Synthesis & Cohesion

Engagement is paramount, within any medium and across mediums. “Designing with human experience as an explicit outcome and human engagement as a specific goal is unique in human history.”

The varieties of engagement (e.g. the senses, mind, heart, and body) and other elements that influence the experience (e.g. capabilities, context, constraints) create the environment in which we work. UxD produces experiences that cross all of these elements, and mapping these experiences is incredibly challenging. The main goal is to synthesize them and create cohesive experiences that honor them.

Discovery, not Invention

With perception covered by visual designers, sound designers, and industrial designers, cognition and emotion are the manifest destiny of IA. User experience is not about information, rather, it is always about people and how they relate to information.

By structuring the information, User Experience Designers structure the tools that humanity uses. And, as a result, we influence how people think and feel. The final result is that those tools, in turn, shape humanity. We should embrace that responsibility.

Jesse predicts that UxD will take it’s place among fundamental human crafts. He posits that we are discovering the realities of people, their tools, and experiences rather than inventing them. With only ten years under our belts, we’ve only just begun that discovery, and he hopes that there will always be more.

 

Transcript of the closing plenary address delivered March 22, 2009 at ASIS&T IA Summit 2009 in Memphis, TN.

This address was written to be read aloud. I encourage you to listen to audio or watch video of the address if possible.

I recognize that being chosen to deliver the closing plenary is an honor, and I do not intend to repay that kindness by giving you a product demo.

I will not be participating in five-minute madness this year. You may consider this my 45-minute madness.

This is a different kind of talk for me. First of all, I have no slides! I kind of feel like I’m working without a net here. I can’t throw in the occasional visual pun to keep you guys paying attention. Secondly, I have no idea how long this talk is. I just finished it just before this began, so basically when I’m out of things to say, I’ll stop talking. Hopefully that will be sooner than you expected, and not later. Third, I’ve decided not to take questions at the end of this talk. My preference would be that if you have questions, don’t pose them to me. Pose them to each other. Publicly, if you can.

So if I run short, we’ll just go straight into five-minute madness and then we’ll all get to the bar that little bit sooner.

Okay, now: first-timers, please stand up.

[audience applauds]

I don’t think we do enough to recognize the importance of new voices in this community, and at this event. Those of you who were here last year may recall my comments from five-minute madness last year, where it seemed like maybe I was a little bit too hard on the first-timers for not being more active participants. What I was really trying to do was scold the old-timers for not doing more to make the first-timers feel welcome, and so I hope that those of you who are first-timers this year have been made to feel welcome by this community.

Now, before you sit down, I want to apologize to all of you, because there’s a great big chunk of this talk that is not going to mean very much to you — because I’m a ten-timer and I’ve got some things to say to my fellow ten-timers. So I’ll just get that out of the way. I hope you’ve enjoyed the rest of the conference — and now you can sit down.

So yeah, in case you guys haven’t heard, this is the tenth IA Summit. I don’t know if word got around about that. This is my tenth IA Summit. Anyone who was at that first Summit will recount for you the strange energy in that room: academics and practitioners eyeing each other warily, skeptical of what the other had to contribute. There was turbulence. (Hi Peter!) But it was productive turbulence.

I can’t say I’ve seen much turbulence at these events since then. Which ought to make all of us nervous, because the opposite of turbulence is stagnation.

In his opening keynote, Michael Wesch quoted Marshall McLuhan: “We march backward into the future.” When I saw this quote, it reminded me of the old quip that generals are always fighting the last war — which is why I think we’ve been stagnating. What war is the field of information architecture fighting?

The war we still seem to be fighting is the war against information architecture itself as a valid concept, as a meaningful part of design practices.

Almost everything you see about the IA community and IA practices — the mailing lists, the conferences, the professional organizations, the process models, the best practice patterns — they’re all optimized to answer two questions: Is this stuff for real? And is it valuable? And the answer to both questions is always, invariably, an emphatic “yes”.

IA is real. And IA is good. And that’s what we all agree on: some IA is better than no IA. But is there such a thing as “bad IA”? I mean, is it possible for an information architecture professional to do a thorough, responsible job, following all the agreed-upon best practices, and still come up with a bad solution?

I don’t think anybody knows the answer to this question. Because we’re still fighting the last war. We’re still trying to defend the answer to that question: is IA good? Is IA valuable?

Now, if you are about my age (and most of you seem to be, which I’ll come back to in a minute), your grandparents grew up in the Depression. And if your grandparents are like mine, this was an experience that shaped their behavior for the rest of their lives. They save everything: any little bit of leftover food, or a loose scrap of fabric, or a button or a screw. They save everything, because the notion of scarcity was deeply imprinted on them when they were young and became such a fundamental part of their worldview that decades later they’re still hoarding all this stuff even though the Depression’s been over… well, it took a break anyway.

Here are some of the most common terms from past IA Summit programs: taxonomy, thesaurus, controlled vocabulary, metadata, faceted classification, navigation, content management — and then there was that one year with all the talks about tagging. Like my grandparents, we cling to these things because they are what saved us. They are the tools by which we proved that yes, IA is real, and it is valuable. But that war is over. We won. And now it’s time to move on, because those comfortable, familiar things represent only part of what information architecture can be.

So it’s time to leave the nest. Thank you, Lou and Peter. Thank you, library science. For getting us off to a great start. For giving us the tools and knowledge to win a place for IA in the world. There will still be a place for library science in IA, but it’s only a part of our larger destiny.

Thank you to ASIST. Thank you to Dick Hill, and Vanessa and Jan and Carlene. This field would not be where it is without your efforts at these events, year after year. But I’m curious — show of hands: who here has ever been to any ASIST event other than an IA summit? [audience raises hands] Who here is an ASIST member? [audience raises hands] A smattering at best. ASIST has been sort of a benevolent host organism for the incubation of IA, but the relationship between ASIST and IA beyond IA Summit hasn’t really gone anywhere.

Okay, I’m debating how to do this… Name the five best-known information architects. [audience calls out various names] Now: name a work of information architecture created by one of these people. [silence] Is that a sign of a mature profession?

The names you know are notable for what they say about their work, not for the work itself. They’re not known for the quality of their work (and I’m including myself in this category).

Moreover, do you know good IA when you see it? And can different people have different ideas about the qualities of a good solution or a bad one, based on their philosophical approach to their work?

One thing I’m really surprised we don’t have yet, that I had expected to see long before now, is the emergence of schools of thought about information architecture.

Will there ever be a controversial work of information architecture? Something we argue about the merits of? A work that has admirers and detractors alike?

We have lots of ways of talking about our processes. In fact, if you look back at these ten years of the IA Summit, the talks are almost all about process. And to the extent that we’ve had controversy, it’s been over questions of process: Is documentation necessary? If so, how much? Which deliverables are the right ones? Personas, absolutely essential, or big waste of time?

What we don’t have are ways of talking about the product of our work. We don’t have a language of critique. Until we have ways to describe the qualities of an information architecture, we won’t be able to tell good IA from bad IA. All we’ll ever be able to do is judge processes.

Another thing that you’ll notice from looking back over ten years of the Summit is that talks are ephemeral. I was at all those summits, and I remember maybe a tenth of what I saw — and I saw less than half of what was on the program. I’m known for being down on academia a lot of the time, but they do have one thing right: you have to publish in order to create a body of knowledge.

I think I’m pretty good at what I do. But you guys are going to have to take my word for it. Because you don’t know my work. You only know what I say about my work.

I think I’m pretty good at what I do. I hope I’m getting better. I hope that my best work is still ahead of me. But I’m not sure. And I’m not sure how I would know. I’ve been coming to the Summit for ten years, and I’ve been doing this work, in some form or another, for close to 15. And as I’ve watched my professional peers settle down, get married, start families, become managers, I’ve found myself wondering about creative peaks.

In the field of mathematics, they say that if you haven’t made a significant contribution by the age of 30, you never will. It’s a young person’s game. 33 is young to be publishing your first novel, but it’s old to be recording your first album.

When do information architects hit their creative peaks? Let’s assume that I’m at about the median age for this group. Just assume most of you are my age, and there are about as many older than me as younger than me.

Now, if I’m at about the median age for an information architect now, when will that change? Will the median age keep going up, as this group of people ages? Presumably, at some point I’ll be one of the oldest guys in the room.

Alternately, what if information architecture is something that you don’t really get good at until you’ve been doing it for 20 years? Then we really have something to look forward to, don’t we?

Here’s another thing I thought we’d be hearing more about by the time of the tenth IA Summit:

You guys heard of this thing called neuromarketing? Man, this stuff is cool. They take people, they hook them up to MRIs — you know, brainwave scanners — and then they show them TV commercials. And they look at what parts of their brains light up when they watch these TV commercials. Then they do a little bit of A/B testing, and they can figure out how to craft a TV commercial that will elicit things like a feeling of safety. Or trust. Or desire.

So yeah, my first reaction when I saw this stuff was: Wow, I gotta get my hands on some of that! We’ve only just scratched the surface of what we can do with eyetracking and the marketers have already moved on to braintracking! But then my second reaction was: Wait a minute. What are we talking about here? A process designed to elicit specific patterns of neural activity in users? Back in the 50s, they called that “mind control”!

Now in a lot of ways, we’re already in the mind control business. Information architecture and interaction design both seek to reward and reinforce certain patterns of thought and behavior. (Just ask anybody who’s tried to wrestle any 37signals app into functioning the way they want to work, instead of the way Jason Fried thinks they ought to be working.)

So there’s always been an ethical dimension to our work. But who’s talking about this stuff? Who’s taking it seriously?

I don’t hear anybody talking about these things. Instead, what everybody wants to talk about is power, authority, respect. “Where’s our seat at the table?” Well, you know, there are people who make the decisions you want to be making. They’re called product managers. You want that authority? Go get that job. Don’t ask them to give that authority to you.

“When are we going to get the respect we deserve?” I’ll tell you how it’s going to happen. Somebody in this room, right now, at some point in the future is going to be the CEO of some company other than a design firm. They’ll develop all of those right political and managerial skills to rise to that level of power. And they will institute a culture in their organization that respects user experience. And then they’re just going to start kicking their competitors’ asses. And then gradually it will happen in industry after industry after industry. That’s how it will happen. But it will take time.

I had the thought at one of these summits a few years ago that we would know we had really arrived as a profession when there were people who wanted to sell us stuff. Because, you see, I grew up in the United States, where you don’t exist unless you are a target market.

And here at this event this year we have companies like TechSmith and Axure and Access Innovations and Optimal Workshop. And we thank them for their support. But where’s Microsoft? Where’s Adobe? Where’s Omni?

We aren’t a target market for any but the smallest companies. The big ones still don’t understand who we are. We’re still a small community, struggling to define itself.

In 2002, in the wake of the last bubble burst, I wrote an essay called “ia/recon”. In that essay, I tried to chart what I saw as a way forward for the field out of the endless debate over definitions. In the essay, I drew a distinction between the discipline of information architecture and the role of the information architect, and I argued that one need not be defined by the other.

Seven years later, I can see that I was wrong. The discipline of information architecture and the role of the information architect will always be defined in conjunction with one another. As long as you have information architects, what they do will always be information architecture. Seems pretty obvious, right? Only took me seven years to figure out.

But that’s okay, because what is clear to me now is that there is no such thing as an information architect.

Information architecture does not exist as a profession. As an area of interest and inquiry? Sure. As your favorite part of your job? Absolutely. But it’s not a profession.

Now, you IxDA folks should hold off for a moment before Twittering your victory speeches — because there’s no such thing as an interaction designer either. Not as a profession. Anyone who claims to specialize in one or the other is a fool or a liar. The fools are fooling themselves into thinking that one aspect of their work is somehow paramount. And the liars seek to align themselves with a tribe that will convey upon them status and power.

There are no information architects. There are no interaction designers. There are only, and only ever have been, user experience designers.

I’d like to talk about each of these three words, in reverse order, starting with “design”. Now, this is a word that I have personally had a long and difficult history with. I didn’t like this word being applied to our work for many years. I thought it placed us in a tradition — graphic design, industrial design, interface design — where our work did not belong. I also saw the dogmatism endemic to design education as poisonous and destructive to a field as young as ours. I still find the tendency of “designers” to view all human creative endeavor through the narrow lens of their own training and experience to be contemptible and appallingly short-sighted.

But I’m ready to give up fighting against this word, if only because it’s easily understood by those outside our field. And anything that enables us to be more easily understood is something we desperately need.

Now, let’s talk about that word “experience”. A lot of people have trouble with this word, especially paired with the word “design”. “You can’t call it experience design!” they say. “How can you possibly control someone else’s experience?” they demand.

Well, wait a minute — who said anything about control? Treating design as synonymous with control, and the designer as the all-powerful controller, says something more about the way these designers think of themselves and their relationship to their work than it does about the notion of experience design.

“Experience is too ephemeral,” they say, “too insubstantial to be designed.” You mean insubstantial the way music is insubstantial? Or a dance routine? Or a football play? Yet all of these things are designed.

The entire hypothesis of experience design (and it is a hypothesis at this point) is that the ephemeral and insubstantial can be designed. And that there is a kind of design that can be practiced independent of medium and across media.

Now, this part makes a lot of people uncomfortable because they’re committed to the design tradition of a particular medium. So they dismiss experience design as simply best practices. “What you call experience design,” they say, “is really nothing more than good industrial design.” Or good graphic design. Or good interface design.

This “mediumism” resists the idea that design can be practiced in a medium-independent or cross-media way. Because that implies that there may be something these mediumist design traditions have been missing all along.

If our work simply recapitulates what has been best practice in all these fields all along, why are the experiences they deliver so astonishingly bad? And let’s face it, they are really bad.

One big reason for it has to do with this last word, one which I think has been unfairly maligned: the word “user”. You guys know the joke, right? There are only two industries in the world that refer to their customers as users. One is the technology business and the other is drug dealers. Ha ha, get it? Our work is just as dehumanizing as selling people deadly, addictive chemicals that will destroy their lives and eventually kill them! Get it? It’s funny because it’s true.

No, it’s not. I’m here to reclaim “user”. Because “user” connotes use, and use matters! We don’t make things for those most passive of entities, consumers. We don’t even make things for audiences, which at least connotes some level of appreciation. The things we make get used! They become a part of people’s lives! That’s important work. It touches people in ways most of them could never even identify. But it’s real.

Okay, time for another show of hands: who here has “information architect” or “information architecture” in your title, on your business card? Raise your hand. [audience raises hands] Almost as many as we had ASIST members.

Okay, now let me see those hands again. Keep your hand up if there is also someone in your organization with “interaction design” or “interaction designer” in the title.

[hands go down]

Almost every hand went down. I see one hand, two hands. Three, four… five.

This is what the interaction design community recognizes — and what the leadership of the IxDA recognizes in particular — that the IA community does not.

In the marketplace, this is a zero-sum game. Every job req created for an “interaction designer” is one less job req for an “information architect” and vice versa. And the more “interaction designers” there are, the more status and authority and influence and power accrues to the IxDA and its leadership.

They get this, and you can see it play out in everything they do, including refusing offers of support and cooperation from groups they see as competitors, and throwing temper tantrums about how other groups schedule their conferences. Meanwhile, the IAs are so busy declaring peace that they don’t even realize that they’ve already lost the war.

This territorialism cannot go on, and I hope the IxDA leadership sees an opportunity here for positive change. These organizations should be sponsoring each other’s events, reaching out to each other’s membership, working together to raise the tide for everyone.

There is no us and them. We are not information architects. We are not interaction designers. We are user experience designers. This is the identity we must embrace. Any other will only hold back the progress of the field by marginalizing an important dimension of our work and misleading those outside our field about what is most important and valuable about what we do. Because it’s not information, and it’s not interaction.

We’re in the experience business. User experience. We create things that people use.

To use something is to engage with it. And engagement is what it’s all about.

Our work exists to be engaged with. In some sense, if no one engages with our work it doesn’t exist.

It reminds me of an artist named J.S.G. Boggs. He hand-draws these meticulously detailed near-replicas of U.S. currency. It’s gotten him in trouble with the Secret Service a couple of times. They’re near-replicas — they’re not exact, they’re obviously fake. They’re fascinating and they’re delightful, in and of themselves, as objects.

But here’s the catch: For Boggs, the work isn’t complete until he gets someone to accept the object as currency. The transaction is the artwork, not the object that changes hands. As he sees it, his work is not about creating things that look like currency it’s about using art as currency. It’s the use — the human engagement — that matters.

Designing with human experience as an explicit outcome and human engagement as an explicit goal is different from the kinds of design that have gone before. It can be practiced in any medium, and across media.

Show of hands: Who here is involved in creating digital experiences? [audience raises hands] Okay, hands down. Now: who’s involved in creating non-digital experiences? [audience raises hands] More hands than I thought.

Now, do we really believe that this is the boundary of our profession? And if we don’t, why are there so many talks about websites at conferences like this one?

Don’t get me wrong, I love the web. I hope to be working with the web in 10 years, in 20 years. But the web is just a canvas. Or perhaps a better metaphor is clay — raw material that we shape into experiences for people.

But there are lots of materials — media — we can use to shape experiences. Saying user experience design is about digital media is rather like saying that sculpture is about the properties of clay.

That’s not to say that an individual sculptor can’t dedicate themselves to really mastering clay. They can, and they do — just like many of you will always be really great at creating user experiences for the web.

But that does not define the boundary of user experience design. Where it really gets interesting is when you start looking at experiences that involve multiple media, multiple channels. Because there’s a whole lot more to orchestrating a multi-channel experience than simply making sure that the carpet matches the drapes.

We’ve always said we were in the multimedia business. Let’s put some weight behind that. Expanding our horizons in this way does not dilute our influence. It strengthens it.

So if we’re all user experience designers, and there are no more information architects, but there is still such a thing as information architecture, what does it look like?

Well, let’s take a closer look at engagement, and think about the ways we can engage people. What are the varieties of human engagement?

We can engage people’s senses. We can stimulate them through visuals, through sound, through touch and smell and taste. This is the domain of the traditional creative arts: painting, music, fashion, cooking.

We can engage their minds, get them thinking, reasoning, analyzing, synthesizing. This is where fields like scholarship and rhetoric have something to teach us.

We can engage their hearts, provoke them in feelings of joy and sadness and wonder and rage. (I’ve seen a lot of rage.) The folks who know about this stuff are the storytellers, the filmmakers, and yes, even the marketers.

And we can engage their bodies. We can compel them to act. This is the closest to what we’ve traditionally done studying and trying to influence human behavior.

And that’s really about it. Or at least, that’s all that I’ve been able to think of: Perception, engaging the senses. Cognition, engaging the mind. Emotion, engaging the heart. And action, engaging the body.

Mapping out the interrelationships between these turns out to be a surprisingly deep problem. Every part influences every other part in unexpected ways. In particular, thinking and feeling are so tangled up together that we practically need a new word for it: “thinkfeel”.

There are a few other factors, sort of orthogonal to these, that influence experience:

There are our capabilities: the properties of our bodies, the acuity of our senses, the sharpness and flexibility of our minds, the size of our hearts. Our capabilities determine what we can do.

Then there are our constraints, which define what we can’t do. The limits on our abilities, whether permanent — someone who’s having a hard time reading because they have dyslexia — or temporary — someone who’s having a hard time reading because they’ve had five bourbons.

Finally, we have context. And I have to admit that I’m cheating a bit on this one because I’m packing a lot of different factors up into this one category. There’s the context of the moment: babies crying, dogs barking, phones ringing. (Calgon, take me away!) Then there’s personal context: the history, associations, beliefs, personality traits of that individual. And there’s the broad context: social, cultural, economic, technological.

But these three — capabilities, constraints, and context — are really just cofactors, shaping and influencing experience in those big four categories: perception, cognition, emotion, and action.

Our role, as user experience designers, is to synthesize and orchestrate elements in all of these areas to create a holistic, cohesive, engaging experience.

So how do we create user experiences that engage across all of these areas? Where can we look to for expertise? Where’s the insight? Where are the areas for further inquiry?

Perception is already pretty well covered. We’ve got visual designers and, sometimes, animators. In some cases we’ve got sound designers. We’ve got industrial designers, working on the tactile aspects of the products we create.

Action, again, is pretty much what we were doing already. I defined action as engagement of the body, which may sound strange to many of you when I say that we’ve really been doing this all along. But if you think about our work, when we talk about behavior, we are always talking about some physical manifestation of a user’s intention — even when that manifestation is as small as a click. (And the interaction designers claim to own behavior anyway so I say let them have it.)

Because the real action is in these last two areas, cognition and emotion. This, to my mind, is the manifest destiny for information architecture. We may not have fully recognized it before because the phrase “information architecture” puts the emphasis on the wrong thing.

It’s never been about information. It’s always been about people: how they relate to that information, how that information makes them think, how it makes them feel, and how the structure of that information influences both things. This is huge, unexplored territory.

We must acknowledge that as user experience designers we have a broader place in the world than simply delivering value to businesses. We must embrace our role as a cultural force.

Here’s Michael Wesch quoting Marshall McLuhan again: “We shape our tools, and then our tools shape us.” Think about that for a second. “We shape our tools, and then our tools shape us.” When McLuhan said “we”, and when he said “us”, he was talking about the entire human race. But not everybody’s a shaper, right? The shapers are the people in this room, the people in this field. We shape those tools and then, the experiences that those tools create shape humanity itself. Think about the responsibility that entails.

I believe that when we embrace that role as a cultural force, and we embrace that responsibility, this work — user experience design — will take its place among the most fundamental and important human crafts, alongside engineering and architecture and all kinds of creative expression and creative problem solving disciplines.

At last year’s five-minute madness, I said that the experts who give talks at events like this one were making it up as they went along. But, I said, that’s okay, because we all are.

I take that back. We aren’t making it up as we go along. This is not a process of invention. This is a process of discovery.

What we are uncovering about people, about tools and their use, about experiences — it’s always been there. We just didn’t know how to see it.

This discovery phase is far from over. Ten years isn’t nearly enough time. There’s more that we can’t see than is apparent to us right now.

For my part, and for you as well, I hope there’s always more for us to discover together.

Thank you all very much.


Video by Chris Pallé and “The UX Workshop”:http://theuxworkshop.tv/
photo by “Jorge Arango”:http://www.flickr.com/photos/jarango/3382137521/
Thanks to Chris and Jorge.

These podcasts are sponsored by:

ASIS&T logo
The “American Society of Information Science & Technology”:http://asist.org/: Since 1937, ASIS&T has been THE society for information professionals leading the search for new and better theories, techniques, and technologies to improve access to information.

IA Summit 2009 logo
The “IA Summit”:http://www.iasummit.org: the premier gathering place for information architects and other user experience professionals.

The theme of the event this year, Expanding Our Horizons, inspired peers and industry experts to come together to speak about a wide range of topics. This included information as wide ranging as practical techniques & tools to evolving practices to create better user experiences.

The design behind the design
“Boxes & Arrows”:http://www.boxesandarrows.com: Since 2001, Boxes & Arrows has been a peer-written journal promoting contributors who want to provoke thinking, push limits, and teach a few things along the way.

Contribute as an editor or author, and get your ideas out there. “boxesandarrows.com/about/participate”:http://www.boxesandarrows.com/about/participate

Bringing Holistic Awareness to Your Design

by:   |  Posted on

Gone, thankfully, are the days when the user experience and the user interface were an afterthought in the website design process, to be added on when programming was nearing completion. As our profession has increasingly gained importance, it also become increasingly specialized: information design, user experience design, interaction design, user research, persona development, ethnographic user research, usability testing—the list goes on and on. Increased specialization, however, doesn’t always translate to increased user satisfaction.

My company conducted a best practice study to examine the development practices of in-house teams designing web applications—across multiple industries, in companies large and small. Some teams were large and highly specialized, while others were small and required a single team member to perform multiple roles. We identified and validated best practices by measuring user satisfaction levels for the applications each team had designed; the higher the user satisfaction scores for the application, the more value we attributed to the practices of the team that designed it. Continue reading Bringing Holistic Awareness to Your Design

Why We Call Them Participants

by:   |  Posted on

It was not an easy recruit. Directors of IT are busy people. Oddly, they’re hard to get hold of. They don’t answer calls from strangers. They don’t answer ads on web sites. The ones who do answer ads on web sites we had to double-check on by calling their company HR departments to verify they had the titles they said they did.

And now this.

“Hi! So we have some executives coming in tomorrow to observe the test sessions.” This was the researcher phoning. He was pretty pleased that his work was finally getting some attention from management. I would have been, too. But. He continued, “I need you to [oh yeah, the Phrase of Danger] call up the participants and move some of them around. We really want to see the experienced guy and the novice back-to-back because Bob [the head of marketing] can only come at 11:30 and has to leave at 1:00.”

“Sure,” I say, “we can see if the participants can flex. But your sessions are each an hour long. And they’re scheduled at 9:00, 10:30, 12:00, and 2:00. So I’m not quite clear about what you’re asking us to do.”

“I’m telling you to move the sessions,” the researcher says, “so the experienced guy is at 11:30 and the novice is at 12:30. Do whatever else you have to do to make it work.”

“Okay, let me check the availability right now while we’re on the phone,” I say. I pull up the spreadsheet of participant data. I can see that the experienced guy was only available at 9:00 am. “When we talked with Greg, the experienced guy, the only time he could come in was 9:00 am. He’s getting on a plane at 12:30 to go to New York.”

“Find another experienced guy then.” What?!

Five signs that you’re dissing your participants

You shake hands. You pay them. There’s more to respecting participants? These are some of the symptoms of treating user research participants like lab rats:

They seem interchangeable to you.


If you’re just seeing cells in a spreadsheet, consider taking a step back to think about the purpose and goals of your study.

You’re focused on the demographics or psychographics.


If it’s about segmentation, consider that unless you’re running a really large study, you don’t have representative sample, anyway. Loosen up.

Participants are just a way to deliver data.


You’ve become a usability testing factory, and putting participants through the mill is just part of your life as a cog in the company machine.

You don’t think about the effort it takes for a person to show up in your lab.


Taking part in your session is a serious investment. The session is only an hour. But you ask participants to come early. Most do. You might go over time a little bit. Sometimes. It’ll take at least a half hour for the participant to get to you from wherever she’s coming from. It’ll take another half hour for her to get wherever she’s going afterward. That’s actually more than 2 hours all together. Think about that and the price of gas.

You don’t consider that these people are your customers and this is part of their customer experience.

You and your study make another touch point between the customer and the organization that most customers don’t get the honor of experiencing. Don’t you want it to be especially good?

They’re “study participants” not “test subjects.”

Don’t forget that you couldn’t do what you do without interacting with the people who use (or might use) your organization’s products and services. When you meet with them in a field study or bring them into a usability lab, they are doing you a massive favor.

Although you conduct the session, the participant is your partner in exploration, discovery, and validation. That is why we call them “participants” and not “test subjects.” There’s a reason it’s called “usability testing” and not “user testing.” As we so often say in the introductions to our sessions, “We’re not testing you. You’re helping us evaluate the .”

Throw away your screener: Tips on recruiting humans

I’m not kidding. Get rid of your screener and have a friendly chat with your market research people. Tell them you’re not going to recruit to match the market segments anymore. Why not? Because they usually don’t matter for what you’re doing. In a usability test, you focus on behavior and performance, right? So recruit for that.

Focus on behavior, not demographics


Why, if you’re testing a web site for movie buffs, will selecting for household income matter? What you want to know is whether they download movies regularly. That’s all. Visualize what you will be doing in the session, and what you want to learn from participants. This should help you define what you absolutely require.

Limit the number of qualifiers


Think about whether you’re going to compare groups. Are you really going to compare task success between people from different sized companies, or who have multiple rental properties versus only one, or different education levels? You might if you’re doing a summative test, but if most of your studies are formative, then it’s unlikely that selecting for multiple attributes will make a big difference when you’re testing 5 or 6 people in each audience cell.

Ask open-ended questions


Thought you covered everything in the screener, but fakers still got into your study? Asking multiple-choice questions forces people to choose the answer that best fits. And smart people can game the questionnaire to get into the study because they can guess what you’re seeking. Instead, ask open-ended questions: Tell me about the last time you went on a trip. Where did you go? Where did you stay? Who made the arrangements? You’ll learn more than if you ask, Of your last three trips taken domestically, how many times did you stay in a hotel?

Learn from the answers


You get “free” research data when you pay attention to the answers given to open-ended screening questions because now people have volunteered information about their lives, giving you more information about context in which you can make decisions about your study design and the resulting data.

Flex on the mix


If you use an outside recruiting firm, ask to review a list of candidates and their data before anyone gets scheduled. You know the domain better than the recruiters do. You should know who will be in the testing room (besides you). You should make the trade-offs when there’s a question about how closely someone meets the selection criteria.

Remember, we’re all human, even your participants. These steps will help you respect the human who is helping you help your company design better experiences for customers.

Building the UX Dreamteam – Part 2

by:   |  Posted on

As we discussed in “part one”:http://www.boxesandarrows.com/view/building-the-ux, the skills in research, information architecture, interaction design, graphic design and writing define the recognized areas of User Experience design. However, there still remains much to discuss about what makes a UX team dreamy.

Each UX Dreamteam has a finely tuned mix of skills and qualities, as varied as the environments in which they operate. Part two will address whether a person has the right ‘hard’ skills and ‘soft’ qualities like communication style, creativity and leadership ability to fit your particular organizational context. We’ll also touch on the quality of an individual’s personality that may or may not complement the others on your team.

Personality

Perhaps the most important consideration in forming your Dreamteam is mixing the personalities of your superstars. As mom used to say “It’s not just about how you look, it’s what’s inside that counts.” A candidate may look ideal on paper, but until you have them in front of you, talking and interacting, you won’t know if what is inside will be a fit. Your group spends almost as much time together as apart, they need to respect and like one another to work well together. Personality typing tests hold the promise of quantifying the immeasurable, but you would be ill advised to use them as part of the interview process. Myers Briggs, DISC and plenty of others use various axes to measure the intrinsic tendencies of a person.

As cool as it sounds, the science is just not exact enough to act as the basis of any decision. This is not to say that these tests are not illuminating in their own right – they certainly foster greater understanding and empathy among teams. Generally speaking, though, people under pressure may answer personality tests as they think they should rather than honestly.

Collaboration is a big part of design best practice and the ability to work well with teammates should be of paramount concern. Selflessness indicates that a candidate is a team player as they seek to raise not only their own reputation, but equally those with whom they work. Humility, humor and empathy are virtues particularly relevant to the creative industry and should be sought after in UX professionals. Each player on the Dreamteam accepts when they’re mistaken, keeps each other creatively entertained and feels for the users they serve.

As much as any skill or quality we have already discussed and will explore in this article, finding the right personality type you need is the classic answer: ‘it depends’. It depends on the personalities of existing Dreamteamsters, the type of work they do, and on the organization into which they must fit. There is no magic formula, but there is one thing to always avoid: toxicity. Morale and productivity can be totally undermined by a “toxic person”:http://bipolar.about.com/od/support/a/070315_toxic.htm. Having one aboard can turn your Dreamteam into a nightmare.  So, do your homework to avoid inadvertently hiring them.

Screening Tips:

Look for signs of toxicity by asking about previous work places and their interactions with teammates. Did they voluntarily leave the last job? Do they mainly talk badly about their last workplace? Remember, a toxic person is often manipulative and they may seem great on the surface, so check references. If you misjudge a new hire and you realize you have a toxic person aboard, waste no time in jettisoning them, no matter how skilled they may seem.

Creative and Analytical qualities

Most jobs in the UX Dreamteam involve a level of creativity and analysis, but it’s a rare gem who is a rock-star operator in both these modes. But visionaries and analysts are equally necessary, ensuring great ideas and the ability to organize and actualize them.

A creative person doesn’t see a glass half empty or half full, but instead asks why it should be a glass at all. An ability to think laterally, meaning" to escape from a local optimum in order to move towards a more global optimum" (“Edward de Bono”:http://www.edwdebono.com/debono/lateral.htm) – is the talent from which innovation is born. A Dreamteam accesses their creativity readily and regularly to push beyond the obvious for an appropriately innovative solution. Ensure a proportion of creative genius in your Dreamteam to increase business success and thereby the team’s reputation.

Your analytical superstars can process vast amounts of information and distill it into a concise and cohesive experience for the user. They are methodical, account for every detail, and question inconsistencies. They grow solutions by breaking a system into its component parts, then creatively reassemble it in logical order. Good analysts are passionate and detail-oriented when identifying patterns in data and behavior.

Screening Tips:

Given how ideas are often difficult to credit to the interviewee, gauge creativity from the dialogue and candor during the interview. A truly imaginative person effortlessly surprises you with a fresh, off-beat approach to old problems. Responses to tangential or seemingly random questions can help illuminate this quality. If they can link the absurd back to realistic solutions coherently and with humor, you can be sure there’s creativity within. Analytical people are interested in details. Does your candidate flinch at the idea of auditing the content of large information system? If they have they done data analysis before, did they jump into it enthusiastically? How did it go?

Practitioner vs. Managerial qualities

Managerial qualities are confused with experience in most professions, and UX is no different. Experience correlates with peer respect, but respect is not all a manager must command. Peter Merholz talks of managers needing to be either "T" shaped "Bar" shaped, referring to the profile of skills they possess. "T"-shaped people have a broad and shallow knowledge of most skills and go deep in only a few.

"Bar"-shaped people do not plunge the depths of any expertise. As “he says”:http://www.peterme.com/?p=580, they are all about the connections between disciplines, and being able to articulate the power of that integration. An "I" shape would indicate deep knowledge in just one or two areas. This profile suggests an awesome specialist practitioner (yes, there is an "I" in Dreamteam!).

Good bosses are quietly also coaches, therapists, facilitators, communicators, organizers and politicians. As leaders, they are comfortable in setting an agenda for others to fulfill while inspiring the Dreamteam to meet or beat that agenda. Your luminary leader provides ‘air cover’, also known as ‘running interference’. Making space for their reports to work by fending off interfering people or tasks, the manager ensures the Dreamteam is focused, not randomized. 

People who find less satisfaction in helping others to be effective are better placed as well-compensated senior practitioners. To presume that someone senior should be promoted into a management position is misguided. A manager’s UX skills are less important than their ability to co-ordinate a group of individuals and spot what your organization needs from them.

Screening Tips:

When seeking managerial talent, look for someone who will revel in the Dreamteam’s success, rather than their own. How have they "run interference" in the past? New managers sourced from within a team show a tendency to get the best out of others prior to their promotion. This is known as "acting up" and makes a good task to set potential managers to test their aptitude. If you’re looking for a practitioner, be sure they’re not fixated on being a manager, lest their ambitions undermine the effectiveness of your designated leader.

Strategic vs Tactical Ability

We all know guys who stand idly by, watching others do their work and wryly commenting, "You look after the details, I’m the big picture man.” Those who strategize with ‘blue sky’ ideas can raise the ire of people slaving at everyday tasks. Tactical skills are just as valuable as strategic. Each serves their purpose in envisioning and getting things done.

Conceiving an entire system and determining what both the business and users get out of it are the domains of big picture people. It is hard to imagine success without their vision to work toward. These people can be creative or analytical but find implementation a chore. They are typically well informed of industry trends and can forecast the future through them. While vision is an awesome asset, without attentive "small picture" work, it’s an apparition. Strategists think one to five years ahead and beyond and are good at depicting a vision.

Tactical people focus on day-to-day activity and on success in the one to six month timeframe. With the exception of think tanks, the organizational balance needs to skew toward small picture people in order to achieve success. Many startups and UX teams fail because of the inverse balance.

Screening Tips:

To find the detail-oriented, look for evidence of finishing products and a personal satisfaction in seeing all loose ends tied up. A strategic thinker will show evidence of helping others to see the wider context of what they’re doing, often through conceptual and architectural diagrams. Can they show you some? Also ask questions which illuminate how they’re plugged into where your organization’s industry and the wider UX field is headed.

Innies vs. Outies

In-house teams (aka "Innies") have needs different to external agencies that provide interface building/designing services or consultancy. An in-house team is working toward increasing profitability through UX. In many cases, the nature of projects does not change over time because there’s only one type of business to support. Exceptions exist, but in general those building in-house teams should discount candidates who need variety to thrive.

The in-house Dreamteam is also better suited to agile development methodologies, which rely heavily on face-to-face contact. Unless a consultant is able to work on-site for the duration of the agile project, they will not be able to fulfill some of the tenets surrounding ‘less documentation, more talking’. Aside from communicating an absent author’s intentions, documentation is a mechanism used by agencies to cover their backside if a client claims poor diligence and won’t be abandoned willingly.

Agencies don’t make much money from staff who aren’t comfortable playing the consultant role. Working under pressure, answering expertly on all subjects related and sometimes unrelated to the job requires a certain type of communication style and self-confidence. Agency staff (aka “outies”) must be broad-skilled and part salespeople to make their expertise and company’s value obvious to clients. This isn’t to say that these qualities aren’t good to have on the in-house UX Dreamteam, but they’re less critical to business success and can be compensated for in other ways.

Screening Tips:

Stack your in-house team with stars who are tactical, for their willingness to roll up their sleeves, dig-in and get enjoyment from attacking a long-term goal is what you need. Strategic thinking is also attractive, but you may want to emphasize this in your management function where vision is expected. Beware hiring those with purely "innie" experience for "outie" roles and vice versa. Outies may find innie work mundane and innies can struggle in the faster-paced, higher-pressure outie workplace. Outies need to have political and sales savvy to navigate varied organizations and present value. Confidence, plausibility and magnetism will be obvious – you’ll want to hire them before they’ve shown you their ample skills. Though be sure they have those too!

Organizational Contexts

Hiring managers generally consider organizational context subconsciously when preparing their Dreamteam, usually feeling out the candidate with gut instinct rather than concrete comparisons.  It helps to abstract the organization into something you can test applicants for compatibility with, like a “persona”;:http://en.wikipedia.org/wiki/Persona for instance, then you can envision a compatible teammate for that persona. Size, work processes, project types, employees, industry and brand among other things influence the organization’s personality.

Some organizations are process-driven and others are more free-form. Process ensures that work complies with a to-do list prescribing smooth running and/or best practice. The less experienced use process like new bicycle riders use training wheels. Some people flourish within a controlled environment. Others feel hampered or oppressed by it. What are the processes used within your organization? What unique characteristics do individuals who operate within them need to be happy and successful?

A Dreamteam’s number will impact the duties each superstar performs. Small organizations can have tasks similar in number to their larger counterparts, but spread them among fewer people. This inevitably means one fulfilling multiple roles. The graphic designer might double as the interface-layer coder. The Information Architect may also be the researcher and writer. If you are in a small organization, a ‘gun’ specialist with all their UX skills primarily in only one area may not be a good fit.

Every workplace has a pace. Agile development or simply expeditious environments tend to be frenetic and mean working quickly. Some people don’t perform without time to pause, think, rework and perfect their work. Others will be frustrated if it takes a long time to get things done. They won’t always agree that crafting something perfectly, or documenting design thoroughly is time well spent. Sometimes perfection is expected, but timescales remain fixed. In this case, experience and coping well with stress is consequential. 

Screening Tips:

What kind of personality does your office have? Who would get along best with that person? Prepare to win the best fit by making a list of organizational attributes and qualities that will complement these. Agile methodologies should be coupled with experienced folk who are natural communicators; as should organizations without process to guide activities. A quiet consensus builder might suit a contentious office, etc. Use the example below to get you started – be creative and modify the attributes as you see fit.

Company Persona and Match

Here’s an example of how you might break down how a potential new team member might fit in with your organization:Take the time to analyze what your Dream Team needs and how well that fits potential talent.

Where do we go from here?

Hiring UX staff is rarely easy, but now you can take a structured approach to identifying the skills and personal qualities your team needs within your organizational context.  Like any craft, building the UX Dreamteam takes practice and the occasional mistake leads to growth as a hiring manager. Even when you think you’ve mastered it, there is still an element of luck to contend. You may be willing to compromise skills and qualities for someone who just feels right and your instincts shouldn’t be discounted. Allow them to inform your choices while thinking about the areas we’ve touched on to build the UX Dreamteam that will make your organization shine.

We Tried To Warn You, Part 2

by:   |  Posted on

A large but unknowable proportion of businesses fail pursuing nearly perfect strategies.

In part I of We Tried to Warn You, three themes were developed:

  • Organizations as wicked problems,
  • The differences of failure leverage in small versus large organizations, and
  • The description of failure points

These should be considered exploratory elements of organizational architecture, from a communications information architecture perspective. While the organizational studies literature has much to offer about organizational learning mechanisms, we find very little about failure from the perspective of product management, management processes, or organizational communications.

Researching failure is similar to researching the business strategies of firms that went out of business (e.g., Raynor, 2007). They are just not available for us to analyze, they are either covered-up embarrassments, or they become transformed over time and much expense into “successes.”

In The Strategy Paradox, Raynor describes the “survivor’s bias” of business research, pointing out that internal data is unavailable to researchers for the dark matter of the business universe, those that go under. Raynor shows how a large but unknowable proportion of businesses fail pursuing nearly perfect strategies. (Going concerns often survive because of their mediocre strategies, avoiding the hazards of extreme strategies).

A major difference in the current discussion is that organizational failure as defined here does not bring down the firm itself, at least not directly, as a risky strategy might. But it often leads to complete reorganization of divisions and large projects, which should be recognized as a significant failure at the organizational level.

One reason we are unlikely to assess the organization as having failed is the temporal difference between failure triggers and the shared experience of observable events. Any product failure will affect the organization, but some failures are truly organizational. They may be more difficult to observe.

If a prototype design fails quickly (within a single usability test period), and a project starts and fails within 6 months, and a product takes perhaps a year to determine its failure – what about an organization? We should expect a much longer cycle from originating failure event to general acknowledgement of failure, perhaps 2-5 years.

There are different timeframes to consider with organizational versus project or product failure. In this case study, the failure was not observable until after a year or so of unexpectedly weak sales, with managers and support dealing with customer resistance to the new product.

However, decisions made years earlier set the processes in place that eventuated as adoption failure. Tracing the propagation of decisions through resulting actions, we also find huge differences in temporal response between levels of hierarchy (found in all large organizations).

Failures can occur when a chain of related decisions, based on bad assumptions, propagate over time. These micro-failures may have occurred at the time as “mere” communication problems.

In our case study, product requirements were defined based on industry best practices, guided by experts and product buyers, but excluding user feedback on requirements. Requirements were managed by senior product managers and were maintained as frozen specifications so that development decisions could be managed. Requirements become treated as-if validated by their continuing existence and support by product managers. But with no evaluation by end users of embodied requirements – no process prototype was demonstrated – product managers and developers had no insight into dire future consequences of product architecture decisions.

Consider the requisite timing of user research and design decisions in almost any project. A cycle of less than a month is a typical loop for integrating design recommendations from usability results into an iterative product lifecycle.

If the design process is NOT iterative, we see the biggest temporal gaps of all. There is no way to travel back in time to revise requirements unless the tester calls a “show-stopper,” and that would be an unlikely call from an internal usability evaluator.

In a waterfall or incremental development process, which remains typical for these large-scale products – usability tests often have little meaningful impact on requirements and development. This approach is merely fine-tuning foregone conclusions.

In a waterfall or incremental development process, which remains typical for these large-scale products – usability tests often have little meaningful impact on requirements and development. This approach is merely fine-tuning foregone conclusions.

Here we find the seeds of product failure, but the organization colludes to defend the project timelines, to save face, to maintain leadership confidence. Usability colludes to ensure they have a future on the job. With massive failures, everyone is partly to blame, but nobody accepts personal responsibility.

The roles of user experience


Figure 1. Failure case study organization – Products and project timeframes. (View figure 1 at full-size.)

As Figure 1 shows, UX reported to development management, and was further subjected to product and project management directives.

In many firms, UX has little independence and literally no requirements authority, and in this case was a dotted-line report under three competing authorities. That being the case, by the time formal usability tests were scheduled, requirements and development were too deeply committed to consider any significant changes from user research. With the pressures of release schedules looming, usability was both rushed and controlled to ensure user feedback was restricted to issues contained within the scope of possible change and with minor schedule impact.

By the time usability testing was conducted, the scope was too narrowly defined to admit any ecologically valid results. Usability test cases were defined by product managers to test user response to individual transactions, and not the systematic processes inherent in the everyday complexity of retail, service, or financial work.

  • Testing occurred in a rented facility, and not in the retail store itself.
  • The context of use was defined within a job role, and not in terms of productivity or throughput.
  • Individual screen views were tested in isolation, not in the context of their relationship to the demands of real work pressures – response time, database access time, ability to learn navigation and to quickly navigate between common transactions.
  • Sequences of common, everyday interactions were not evaluated.

And so on.

The product team’s enthusiasm for the new and innovative may prevent listening to the users’ authentic preferences. And when taking a conventional approach to usability, such fundamental disconnects with the user domain may not even be observable.

Many well-tested products have been released only to fail in the marketplace due to widespread user preference to maintain their current, established, well-known system. This especially so if the work practice requires considerable learning and use of an earlier product over time, as happened in our retail system case. Very expensive and well-documented failures abound due to user preference for a well-established installed base, with notorious examples in air traffic control, government and security, medical / patient information systems, and transportation systems.

When UX is “embedded” as part of a large team, accountable to product or project management, the natural bias is to expect the design to succeed. When UX designers must also run the usability tests (as in this case), we cannot expect the “tester” to independently evaluate the “designer’s” work. The same person in two opposing roles, the UX team reporting to product, and restricted latitude for design change (due to impossible delivery deadlines) – we should consider this a design failure in the making.

In this situation, it appears UX was not allowed to be effective, even if the usability team understood how to work around management to make a case for the impact of its discoveries. But the UX team may not have understood the possible impact at the time, but only in retrospect after the product failed adoption.

We have no analytical or qualitative tools for predicting the degree of market adoption based on even well-designed usability evaluations. Determining the likelihood of future product adoption failure across nationwide or international markets is a judgment call, even with survey data of sufficient power to estimate the population. Because of the show-stopping impact of advancing such a judgment, it’s unlikely the low-status user experience role will push the case, even if such a case is clearly warranted from user research.

The racket: The organization as self-protection system

Modern organizations are designed to not fail. But they will fail at times when pursuing their mission in a competitive marketplace. Most large organizations that endure become resilient in their adaptation to changing market conditions. They have plenty of early warning systems built into their processes – hierarchical management, financial reports, project management and stage-gate processes. The risk of failure becomes distributed across an ever-larger number of employees, reducing risk through assumed due diligence in execution.

The social networks of people working in large companies often prevent the worst decisions from gaining traction. But the same networks also maintain poor decisions if they are big enough, are supported by management, and cannot be directly challenged.

The social networks of people working in large companies often prevent the worst decisions from gaining traction. But the same networks also maintain poor decisions if they are big enough, are supported by management, and cannot be directly challenged. Groupthink prevails when people conspire to maintain silence about bad decisions. We then convince ourselves that leadership will win out over the risks; the strategy will work if we give it time.

Argyris’ organizational learning theory shows people in large organizations are often unable to acknowledge the long-term implications of learning situations. While people are very good at learning from everyday mistakes, they don’t connect the dots back to the larger failure that everyone is accommodating.

Called “double loop learning,” the goal is learn from an outcome and reconfigure the governing variables of the situation’s pattern to avoid the problem in the future. (Single-loop learning is merely changing one’s actions in response to the outcome). Argyris’ research suggests all organizations have difficulties in double-loop learning; organizations build defenses against this learning because it requires confrontation, reflection, and change of governance, decision processes, and values-in-use. It’s much easier to just change one’s behavior.

What can UX do about it?

User experience/IA clearly plays a significant role as an early warning system for market failure. Context-sensitive user research is perhaps the best tool for available for informed judgement of potential user adoption issues.

Several common barriers to communicating this informed judgment have been discussed:

  • Organizational defenses prevent anyone from advancing theories of failure before failure happens.
  • UX is positioned in large organizations in a subordinate role, and may have difficulty planning and conducting the appropriate research.
  • UX, reporting to product management, will have difficulty advancing cases with strategic implications, especially involving product failure.
  • Groupthink – people on teams protect each other and become convinced everything will work out.
  • Timing – by the time such judgments may be formed, the timeframes for realistic responsive action have disappeared.

Given the history of organizations and the typical situating of user experience roles in large organizations, what advice can we glean from the case study?

Let’s consider leveraging the implicit roles of UX, rather than the mainstream dimensions of skill and practice development.

UX serves an influencing role – so let’s influence

UX practice must continue to develop user/field research methods sensitive to detecting nascent problems with product requirements and strategy.

User experience has the privilege of being available on the front lines of product design, research, and testing. But it does not carry substantial organizational authority. In a showdown between product management and UX, product wins every time. Product is responsible for revenue, and must live or die by the calls they make.

So UX should look to their direct internal client’s needs. UX should fit research and recommendations to the context of product requirements, adapting to the goals and language of requirements management. We (UX) must design sufficient variability into prototypes to be able to effectively test expected variances in preference and work practice differences. We must design our test practices to enable determinations from user data as to whether the product requirements fit the context of the user’s work and needs.

We should be able to determine, in effect, whether we are designing for a product, or designing the right product in the first place. Designing the right product means getting the requirements right.

Because we are closest to the end user throughout the entire product development lifecycle, UX plays a vital early warning role for product requirements and adoption issues. But since that is not an explicit role, we can only serve that function implicitly, through credibility, influence and well-timed communications.

UX practice must continue to develop user/field research methods sensitive to detecting nascent problems with product requirements and strategy.

UX is a recursive process – let’s make recursive organizations as well

User experience is highly iterative, or it fails as well. We always get more than one chance to fail, and we’ve built that into practices and standards.

Practices and processes are repeated and improved over time. But organizations are not flexible with respect to failure. They are competitive and defensive networks of people, often with multiple conflicting agendas. Our challenge is to encourage organizations to recurse (recourse?) more.

We should do this by creating a better organizational user experience. We should follow our own observations and learning of the organization as a system of internal users. Within this recursive system (in which we participate as a user), we can start by moving observations up the circle of care (or the management hierarchy if you will).

I like to think our managers do care about the organization and their shared goals. But our challenge here is to learn and perform from double-loop learning ourselves, addressing root causes and “governing variables” of issues we encounter in organizational user research. We do this by systematic reflection on patterns, and improving processes incrementally, and not just “fixing things” (single-loop learning).

We can adopt a process of socialization (Jones, 2007) rather than institutionalization, of user experience. Process socialization was developed as a more productive alternative to top-down institutionalization for the introduction of UX practices in organizations introducing UX into an intact product development process.

While there is strong theoretical support for this approach (from organizational structuration and social networks), socialization is recommended because it works better than the alternatives. Institutionalization demands that an organization establish a formal set of roles, relationships, training, and management added to the hierarchy to coordinate the new practices.

Socialization instead affirms that a longer-term, better understood, and organizationally resilient adoption of the UX process occurs when people in roles lateral to UX learn the practices through participation and gradual progression of sophistication. The practices employed in a socialization approach are nearly the opposite (in temporal order) of the institutionalization approach:

  • Find a significant UX need among projects and bring rapid, lightweight methods to solve obvious problems
  • Have management present the success and lessons learned
  • Do not hire a senior manager for UX yet, lateral roles should come to accept and integrate the value first
  • Determine UX need and applications in other projects. Provide tactical UX services as necessary, as internal consulting function.
  • Develop practices within the scope of product needs. Engage customers in field and develop user and work domain models in participatory processes with other roles.
  • Build an organic demand and interest in UX. Provide consulting and usability work to projects as capability expands. Demonstrate wins and lessons from field work and usability research.
  • Collaborate with requirements owners (product managers) to develop user-centered requirements approach. Integrate usability interview and personas into requirements management.
  • Integrate with product development. Determine development lifecycle decision points and user information required.
  • Establish user experience as process and organizational function
  • Provide awareness training, discussion sessions, and formal education as needed to fit UX process.
  • Assessment and renewal, staffing, building competency

We should create more opportunities to challenge failure points and process breakdowns. Use requirements reviews to challenge the fit to user needs. Use a heuristic evaluation to bring a customer service perspective on board. In each of those opportunities, articulate the double-loop learning point. “Yes, we’ll fix the design, but our process for reporting user feedback limits us to tactical fixes like these. Let’s report the implications of user feedback to management as well.”

We can create these opportunities by looking for issues and presenting them as UX points but in business terms, such as market dynamics, competitive landscape, feature priority (and overload), and user adoption. This will take time and patience, but then, its recursive. In the long run we’ll have made our case without major confrontations.

Conclusions

The best we can hope to bat is .500. If you’re getting better than that, you’re not swinging for the fences. Even Barry bonds, steroids or not, is not getting that. We need to celebrate failure.

Scott Cook, Intuit’s Founder, famously said at CHI 2006: “The best we can hope to bat is .500. If you’re getting better than that, you’re not swinging for the fences. Even Barry bonds, steroids or not, is not getting that. We need to celebrate failure.”

Intelligent managers actually celebrate failures – that’s how we learn. If we aren’t failing at anything, how do we know we’re trying? The problem is recognizing when failure is indeed an option.

How do we know when a project so large – an organizational level project – will belly-up? How can something so huge and spectacular in its impact be so hard to call, especially at the time decisions are being made that could change the priorities and prevent an eventual massive flop? The problem with massive failure is that there’s very little early warning in the development system, and almost none at the user or market level.

When product development fails to respect the user, or even the messenger of user feedback, bad decisions about interface architecture compound and push the product toward an uncertain reception in the marketplace. Early design decisions compound by determining architectures, affecting later design decisions, and so on through the lifecycle of development.

These problems can be compounded even when good usability research is performed. When user research is conducted too late in the product development cycle, and is driven by usability questions related to the product and not the work domain, development teams are fooled into believing their design will generalize to user needs across a large market in that domain. But at this point in product development, the fundamental platform, process, and design decisions have been made, constraining user research from revisiting questions that have been settled in earlier phases by marketing and product management.

References

Argyris, C. (1992). On organizational learning. London: Blackwell.

Howard, R. (1992). The CEO as organizational architect: an interview with Xerox’s Paul Allaire. Harvard Business Review, 70 (5), 106-121.

Jones, P.H. (2007). Socializing a Knowledge Strategy. In E. Abou-Zeid (Ed.) Knowledge Management and Business Strategies: Theoretical Frameworks and Empirical Research, pp. 134-164. Hershey, PA: Idea Group.

Raynor, M.E. The strategy paradox: Why committing to success leads to failure (and what to do about it). New York: Currency Doubleday.

Rittel, H.W.J. and Weber, M.W. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155-169.

Taleb, N.N (2007).The Black Swan: The Impact of the Highly Improbable. New York: Random House.

The Trouble With Web 2.0

by:   |  Posted on

Today, there is a lot of buzz around a number of topics labeled as "Web 2.0“. Consultants jump on the "Web 2.0" bandwagon and IT vendors desperately struggle to add “Web 2.0” features to their products. But the term is still unclear and nobody has a good definition of what “Web 2.0” is and what it is not. The term was originally coined by Tim O’Reilly in an article describing the changes in business processes and models that have been triggered by new and creative combinations of already existing technologies.

Other “social networking” services (like wikis and blogs) have been added to the Web 2.0 “genre”, generating a new end user experience on the Internet. Many large enterprises are now starting “Web 2.0” projects, their IT departments seemingly eager to show their technological abilities. The “new” is strangely attractive and everyone wants to be on board.

But what is the real core of this new phenomenon? According to the conclusions of Tim O’Reilly the core is not the technology (which has been there for some time) but the emergence of new “patterns” – new or changed business processes and a new concept of the user. These patterns have been put to good use in the World Wide Web. But can they be transferred to a corporate environment, at the enterprise level, as well? Let’s have a closer look on them.

  • the Web 2.0 platform breaks down borders between services
  • Web 2.0 utilizes the collective intelligence of its users
  • Web 2.0 cannot control the process of knowledge creation
  • Web 2.0 is constantly linking knowledge, thereby not protecting intellectual property

The web as platform.

Or the Web 2.0 platform breaks down borders between services

One pattern identified by Tim O’Reilly is a change in the business model of software suppliers. In “Web 1.0” times the web was used only as a transport medium, delivering predefined information (e.g. static HTML pages) to client based software products (e.g. the Netscape browser). In Web 2.0 companies use the web as a platform, using the enhanced technology offering to create web-based applications or services that do not require any installation on the user’s machine (e.g. the Google Search Engine). If the client does not install a program that allows tracking of usage then charging for usage become difficult.

Business models that rely on the sale of personalized or concurrent software licenses will not work anymore. The suppliers will be forced to find another way to charge for their services. To generate income they may use advertisements (like Google) or additional service or content offerings (like “land” purchases in Second Life). Internal IT departments however may have to rethink their funding models, especially if they are funded by various company departments for their services. They will not be able to allocate operating costs per license if they want to be Web 2.0 .

Let’s assume a departmentally funded IT department offers a blog or wiki as a new service. No client installation is necessary, but access can be restricted to the members of those departments who pay for the service. Restricted access, as we will see later, will affect the overall value of a collaboration service and yet is not a fair sharing model – it opens up all sorts of claims for charge reduction on the grounds that the other departments use the service more intensively or in a different manner. A per-capita cost allocation may be a solution, but then the departments might try to overextend service usage as they don’t pay extra for more intensive use. The means of measuring usage and paying for it will have to change in order for the IT department to get paid for its services.

But this new platform pattern implies more than just a change in software distribution mechanisms. Not being bound to a client means these new web services typically do not rely on a single source of data, like a classical application with its own database, but combine data from different sources, enabled by open source protocols and standards. The large number of mash-ups (data combined and displayed in a new and usually creative manner) that have emerged around the Google map function are a good example. With these new service offerings the core competency of the service provider shifts from software development to database management and the ability to combine data that is available on the Web to create meaningful information.

Many internal IT departments face a different situation where access to data and the means to combine them are restricted by departmental boundaries, technological incompatibilities or data security and protection rules. Person-related HR or sales data are among the most protected in a corporate environment, and their usage is highly restricted for good reason. Data security and protection rules will also be a showstopper for externally hosted services. According to Article 25 of the European Data Protection Guideline, issued by the European Commission, personal data may only be transferred to third countries (all non-EU countries) if the receiving country provides an adequate level of protection. US organizations have to qualify under the “safe harbor” program to be able to receive data from a European organization.

But even then the idea of having your company’s most valuable data in an externally hosted application (e.g. the Salesforce CRM), with data transfer over the Internet will be the ultimate nightmare of every company’s IT security officer. The required security prohibits the clever combination and reinterpretation of data that we see on Google maps or elsewhere.

Web 2.0 cannot control the process of knowledge creation.

Or utilizing and enabling collective intelligence

We have seen that the new Web 2.0 services are powered by the ability to combine data from different sources. But where does the data come from and who creates it? The Web 2.0 pattern of “collective intelligence” shifts the task of creating and maintaining data and content from centralized resources to a dispersed user community. The eBay selling platform would be useless without the activities of the millions of sellers and buyers, who are creating the content and a critical mass of offerings that attract other users into using the service. Wikipedia would be a completely empty shell without its users creating and maintaining the content. In his article Tim O’Reilly states that the value of the service is related to the scale and dynamism of the data it provides. The larger the number of articles in Wikipedia the more users will use it as reference, therefore the service gets better the more people use it and contribute to it. Tim O’Reilly calls this the “LOW” principle – “let other’s do the work”. This works perfectly in the Web with its huge user base (according to the statistics already more than one billion people) but will this pattern also work in a corporate environment?

The corporate user base, even in the largest enterprises, is smaller than the user base on the Web which limits the number of potential contributors. If the quality of the service depends on the number of users then companies have a disadvantage compared to the Web-based services. In many interviews end users said that they often prefer to research on the Web instead of using their corporate knowledge resources because they are confused by the complexity of internal search and because the information they can find on the web seems to be more recent. However the reliability of Web-based information that can be edited by everybody is in doubt when comes to hard legal or medical use. Would you be willing to bet your career on a Wikipedia article?

If the quality of the service is improved by the amount of user commitment it will only be successful if a critical mass of users can be attracted. While Web services like eBay, Wikipedia or Flickr have been soaring over the last years, driven by user commitment, corporate services often have a contribution pattern like this:

web2_knowledgecurve.gif

But what attracts users to donate their time and energy to contribute to Web services like Wikipedia or Flickr while not doing so to corporate services? Psychology and economics teach us that there is no such thing as altruism – whatever people do will create a personal return of value for them. This personal value is measured by individual criteria. Respect and prestige, personal reputation, political beliefs or desires, and of course monetary incentives influence the decision as to whether their contribution creates this value. People create an article in Wikipedia because they believe that the topic is interesting or important or because they want to see their name in print, and put pictures on Flickr because they want to share them with others, thereby influencing how they are perceived by others. The value of contributing must be higher than doing something else (e.g. watching a sports game on TV or adding to the corporate knowledge base).

In a corporate environment this might be different as a different set of values becomes dominant. Company vision, goals, or instructions will be added on top of the personal value criteria, together with given priorities changing the decision of what creates value. Of course one big driver for such decisions is the need for an employee to be externally “chargeable”, a typical situation in the consultancy business. If a consultant has to decide whether he spends his time generating direct revenue for the company (and therefore for himself) by working for a client; or having to explain to his supervisor why he choose to improve the internal knowledge base instead he or she will opt for the first alternative. As long as contribution is regarded as less important topic in the corporate hierarchy the priority of knowledge initiatives will go down over time simply because people start to value other things as more important.

So contribution is rational for people if there is a reward. But there is another rationale for people, especially in large organisations. Cooperation within the Web communities is mainly driven by non-monetary values as the contributors don’t receive any money for their input. These communities are networks of specialists who rely on each others knowledge. Investing time to create and contribute knowledge will pay off here because there is no direct competition and other people’s knowledge might help you tomorrow. Small consultant companies may be examples of those tight communities. If there is direct competition people normally tend to change their behaviour. They try to acquire and to protect their special knowledge. The German language even has a special term for this kind of behaviour: “Herrschaftswissen”, which means superiority through withheld or not communicated information or knowledge. Many corporations have answered these issues by centralizing knowledge management into special departments . But Web 2.0 requires involving the largest number of users possible and a centralized approach might not be the right answer anymore. Corporate projects that focus on providing Web 2.0 technologies alone will fail if the companies do not change their rewarding schemes and knowledge management processes. Such projects will need to rethink incentives for participators and to create the time slots necessary for people to contribute. If the corporation wants the genuine benefit from Web 2.0 then they must not underestimate the effort it takes to produce it. Finding a balance between corporate or proprietary knowledge and the free-for-all idea exchange of Web 2.0 services is critical if a corporate IT department wants be benefit from Web 2.0 style services.

Web 2.0 cannot control the process of knowledge creation.

Or the uncontrollable wisdom of the “blogosphere”

Judging by the hype they have created; wikis and web logs (blogs) seem to be an important part of the Web 2.0 patterns. As blogs have spread through the web like wildfire vendors of content management systems are struggling to add this functionality into their applications. Already the term “blogosphere” has emerged as a collective term encompassing all blogs and their interconnections. It is the perception that blogs exist together as an extended connected community or a complex social system. However it is common knowledge that a system composed from several component parts will act differently than its individual isolated components. An ant colony develops complex social behaviour and erects structures – a task a single ant could never perform. While a single nerve cell is only able to transfer electrical impulses the enormous network of synaptical references and trackbacks of the human forebrain enables conscious thought.

Tim O’Reilly states that the blogosphere creates a structure that resembles the human brain. Expressing an idea in a single blog might not change the world, but if this idea is picked up, discussed and commented in a large number of blogs it not only gets the attention of many people – it might get enhanced, developed, refined, challenged and eventually transformed evolutionary into something that might influence the way of the world. Like in the anthill or the human brain this process is not controlled by a single instance – it is driven by the participation and cooperation of many individuals with their individual motives. This absence of a controlling instance allows for creativity, progress of ideas and the expression of individual opinions. The old saying that the whole is more than the sum of the parts is true here. However it is a self-organized process that follows its own rules – forcing or securing this is not currently possible nor probably desirable.

The various discussions on the attempts of some nation states to restrict Internet access within their borders show that organizations that rely on its inhabitants to keep within the existing structures will only tolerate an “uncontrollable” environment up to a certain level and will try to erect restrictions if this environment starts to thread organizational foundations. Using the discussions within a blogosphere to enable the development of new consulting solutions might be welcome to a corporation while critical comments on the latest corporate policies and procedures might not. In some corporate areas where following predefined procedures and processes (e.g. accounting standards) is necessary or where secrets have to be protected an open exchange among employees might not be allowed at all to protect the company interests. And companies who start allowing employees to blog will experience that enforcing control on the blog or wiki contents will be detected and will create strong opposition. Companies have to be aware that open service offerings like blogs and wikis cannot be removed afterwards or restricted in scope without losing employee loyalty or making themselves look like fools in the process. And worse – as the Internet provides a means to get around those blockages that the enterprise might believe are comforting: people might take their internal discussions public.

One other aspect – the strength of a wiki is the presentation of content in a loosely structured, intuitive way, creating a hypertext structure resembling creative human thought processes. As there is no visible hierarchical structure in a wiki retrieving content mainly relies on search. That is why Wikipedia or Google show a large search field on their main pages. This unstructured organisation of content fails if the content itself is highly structured. A law commentary contains the text of the law structured by paragraphs, with some comments or additional materials for each paragraph, sometimes for each sentence. Having a search might help if materials related to a special topic are needed, but if an experienced lawyer needs the latest comment on a certain paragraph he needs a different navigation – he will prefer to select the paragraph directly from a list, browsing through the hierarchy of paragraphs and comments. So wikis are a great tool but not a cure-all.

The Web 2.0 is constantly linking knowledge, thereby not protecting intellectual property.

Or the perpetual beta, mash-ups and new intellectual property

Another pattern Tim O’Reilly points to – most of the emerging 2.0 services will (or should) forever wear a “beta” sticker on their homepages. As the role of the user moves from passive consumer to active participant; the quick and continuous implementation of user driven enhancements becomes a driver for the service provider, especially in a competitive market environment.

Release cycles tend to diminish as deployment does not count for Web based services – users will always get the latest version of the service when (re-)loading the site. While development cycles shrink to days or hours and pressure to continuously implement new features rises quality assurance procedures becomes less important – bug-fixes can be deployed instantly and as long as the users don’t pay for the service they will tolerate errors or will be willing to learn new features by themselves. In a corporate environment this might be different. Deployment delays do play a role, especially when permanent online access is not given or network traffic is limited. Quality of the service and at least a certain operational stability will be more important than speed of delivery, especially if software errors will cause financial damage or threaten the company in other ways. And as corporate applications tend to be more complex than Web services training efforts becomes an important part. So corporate applications cannot be developed and deployed using a “perpetual beta” mode.

One other thing is the focus – while most of the Web 2.0 companies focus on a single product or a small suite of similar services; an internal corporate IT service provider will have hundreds, sometimes thousands of services (and applications) to provide. Release management and portfolio management will be needed to ensure maximum value, which means that development resources might not be able to work on one service the whole time.

Another pattern that follows from this development mode is the emergence of so called mash-ups. As the rapid development relies on “lightweight programming models” (another pattern described by Tim O’Reilly), like scripting languages the security of the code is minimal, exposing the software to the users who are able to use or even “hijack” the existing interfaces to create their own solutions and mash-ups on the existing platforms, combining data from different sources to create new information and knowledge. In some cases (e.g. Google Maps) this is welcome to the service provider as it increases the spread of the service and the data quality it provides – all the users of Google Maps do provide Google with an enormous amount of location data, enabling Google to create the most detailed worldwide Yellow Pages ever seen. However if access to or re-use of the data should be limited (e.g. to paying customers) the Web 2.0 technology might not be safe enough. In business to business relations and also in a corporate environment data protection, security and the protection of intellectual property are issues of huge importance, so an open technology platform will be out of scope. On the other hand this limits companies in leveraging the know-how and creativity of its users. Even the internal use of existing Internet web-based services might cause issues as the company cannot control the service. What if the service provider decides to change, charge for or even discontinue an external service the company has come to rely on? Replacing the service will again create additional efforts to adapt the internal applications, which might outweigh the savings created by the free use of the service.

What now?

We have seen that there are differences between the Web and corporate environments. While the Web is a deregulated environment, with millions of users contributing and easy access to data, corporations have to restrict their users for many reasons, thereby limiting the potential of the Web 2.0 patterns. While Web 2.0 patterns work well in the Web there might be obstacles and issues when they are implemented in a corporate environment without adaptation. “Might” because every company is an individual organization and there are no easy, “one-size-fits-all” solution. On the other hand the Web 2.0 patterns have been proven to be too successful to be ignored.

There is no ready-made solution, only some good advice. The most important and most simple is that corporate behaviours and processes are not changed just by implementing a new IT service. Installing a blog in a formal and hierarchical corporate culture will not change the company in an open and informal community. Web 2.0 patterns will only work if the corporate and even national culture is already responsive to more collaboration and participation or if the implementation is accompanied by other measures to support cultural change. Creating and holding up motivation of users to contribute, seemingly no problem in the WWW with a billion users will be one of the success factors. So corporate Web 2.0 implementation projects have to broaden their scope, have to add structural and cultural change or process redesign to their charter. And those “soft” topics tend not to have easy solutions. So when your IT department or an external consultant excitedly tells you about how they are adding “Web 2.0” to the corporate computing environment: be prepared for a difficult birthing process and adjusting your expectations.

I would be happy to hear about your experiences.

Building the UX Dreamteam

by:   |  Posted on

Part one of a two-part article.

Finding the right person to compliment your User Experience team is part art and part luck. Though good interviewing can limit the risk of a bad hire, you need to carefully analyze your current organizational context, before you can know what you need. Herein lies the art. Since you can’t truly know a candidate from an interview, you gamble that their personality and skills are what they seem. Aimed at managers and those involved in the hiring decision process, this article looks at the facets of UX staff and offers ways to identify the skills and influence that will tune your team to deliver winning results.

The Art

There are many pieces to the User Experience puzzle. The art of fitting the roles together to compliment each other and your particular situation requires a bit of luck and intuition. Try as we might, it is nearly impossible to find someone who is highly skilled in all areas, so you will want to find either a "Jack of all trades" or a specialist. First, lets explore some loose definitions of various skills that make up the User Experience Team.

Skills are measurable. Anybody can learn new skills through education or apprenticeship. They are the capital built over the course of a career, making the applicant more saleable. Categories of research, information architecture, interaction design, graphic design and writing help us communicate and understand the part each skill plays in defining user experience. Not to be confused with roles – which define the activities of any member on the team – staff employ skills to do the work.

Let’s look at skills in a sequential order, as they’re typically utilized when practicing User Centered Design. We’ll begin with research.

Research Skills

Research is interwoven into all user experience roles – the inspiration and validation of ideas and designs greatly enhances the chance of success in meeting your design objectives.

This skill, as it relates to UX, is about asking questions and illuminating a subject area in unobvious ways. Knowledge of psychology, sociology and anthropology are used to tease out intelligence from users, market data and academia. In this regard, Interaction Designers and Information Architects must use research skills to inform the strategic aspects of their job. Even a cursory study of a potential product’s competitive landscape requires an essential research component.

The researcher in us takes a scientific approach to the study of humanity and uses quantitative and qualitative studies to inform the design process. Roles on the UX Dreamteam employ techniques such as:

  • Contextual inquiry – field research that involves interviewing users “in context” i.e. as they perform familiar tasks in their normal environment
  • Surveys – one questionnaire answered by many respondents, statistically analyzed for trends direct us toward a user’s requirements
  • Usability testing – key for highlighting UI and system design flaws as well as opportunities
  • Card sorting – used by IAs to test categorization ideas
  • Emotional response testing – great value to graphic designers seeking direction on the impact of their visuals

Research skills punctuate the UX professionals’ work agenda.

Being good at research is key, but disseminating the results for maximum impact to ensure findings are used is equally important. A lack of attention to this can undermine valuable work. Good communicators reap the benefits of clearly, poignantly presenting facts and theories.

A researcher, whether dedicated to this role or filling it temporarily, needs to be pragmatic. Remaining objective – interpreting findings only from collected data – is often a challenge when we are invested in a particular idea or direction. Researchers should be inquisitive and analytical with an empathetic instinct to dig beneath the surface of things.

Screening tips: Look for some evidence that a candidate understands scientific method with regard to research. They should also be able to separate themselves from an emotional attachment to their own ideas. Not to say they should be dispassionate about finding the right answer, but their personal biases should not taint this effort. Probe their ability to analyze data. Test to see if their nature is exploratory (good) or if they are just as happy to make general assumptions (not so much). See how they have creatively engaged the team with research findings by threading them in to the day’s work.

Information Architecture Skills

Information Architecture entails designing an information system and the users’ pathways through it. The IA’s goal is to create a system that will provide useful information to suit the user’s context. System structure, inputs and outputs of information, semantic analysis and accommodating changes in the user’s context are in the information architect’s domain.

Frequently Information Architecture (IA) and Interaction Design (IxD) skills are confused. Job titles of one or the other do not neatly describe the skills at work and it’s common for an “IA” to use IxD skills and visa versa. Jesse James Garrett in his book The Elements of User Experience differentiates IA and IxD by the type of system being designed. He asserts that Information Architecture fits a model of the web as a hypertext system, rather than a software interface. Johnathan Korman from Cooper delves into the distinction in his article The Web, Information Architecture and Interaction Design – “IA means defining information structures to answer the question "how does a user find the information they want?… IxD means defining system behaviors to answer the question "how does a user take the action they want?”…”

IA and IxD roles can work in tandem. The IA defines what data needs to appear and the IxD crafts the UI and user flow. Primarily IxDs in this setup are focused on the nuances of the functionality of the system, and IAs on the data that drives it or is manipulated through it. This is a good strategy for large scale, data-centric projects such as defining a content management system. For smaller projects, one person may perform both roles more efficiently. What type of systems does your team work on? How much of your work is about “content” and searching and how much is about software UI?

IA activities fall into two categories. Big IA includes creating ontology, categorization and metadata design. Little IA is labeling, auditing content, creating sitemaps and wireframes. Do you know which of these you really need?

Richard Saul Wurman – an architect and graphic designer – coined the term “Information Architecture” about 30 years ago. He laid out the domain of what’s now more commonly thought of as broadly “information design” with an emphasis on systemic design. The practice of IA we see today was matured by those in the field of information and library sciences, such as Peter Morville. An IA is an analytical, left-brained beast with a detailed eye for modeling content, metadata and information retrieval systems. They are tireless completers, auditing seemingly endless quantities of information, carefully filtering it and finding the patterns within.

Screening tips: Look for patience, attention to detail and a comfort with language, particularly vocabulary, synonyms and definitions. Pattern analysis and capacity for cataloging and organizing information such as content types, article topics, genres, authors, dates, etc, is essential. Conclusions should not all be derived from their own organizational prowess ­– are they inclined to gain inspiration or test ideas with users? The difference between administrative, intrinsic and descriptive metadata should not be foreign, after all, they revel in semantics!

Interaction Design Skills

The Interaction Designer is a story-weaver – scripting the narrative between man and machine – the dialogue of system response to user action. Goals, behavior and flow are significant strategic concerns, but this skill goes beyond making interfaces relevant and usable. IxD marries personality with each interaction story, creating a system with which users make an emotional connection. Interaction Design and Visual Communication work together to breathe life into software UI. IxD defines the way the user manipulates the interface and Visual Communication determines how that looks in concert with all the other visual elements on screen. Blending analysis and creativity – working between artistry and engineering – Interaction Design concepts muster team consensus around what to build via the user interface layer.

Scenarios, flow diagrams, interaction models, prototypes and wireframes are typical deliverables of interaction design. They capture the desired user experience that is translated into a functional specification.

Because interaction design is primarily about creating intuitive interfaces, a measure of empathy produces the best results. This skill is not a precise science, so humility and resilience in the face of criticism (or sometimes failure) is also good.

Screening tips: Look for an interest in and aptitude for psychology; passion for making things work intuitively; enthusiasm for the difference between good and great interactions. Do they understand how to brand an interaction? Good IxDs make stories; can they hold your interest? The world is full of interaction – they should draw their inspiration widely. They must be comfortable with research and usability concepts too.

Graphic Design Skills

Graphic design (also known as Visual Communication, Information Design or Visual Design) is primarily concerned with clearly communicating the aesthetic, personality and function of a system and to invoke feeling. Strategically, an understanding of branding on a level deeper than visual identity, delving into messaging, semiotics and interaction is important. It is here that they work closely with writers and Interaction Designers on software or with an IA on hypertext systems. Tactically, Visual Communicators ensure that the UI layer is lucid, communicates visual hierarchy and represents the brand in ways that appeal to the end user. Inherently creative, Visual Communicators demonstrate a passion for the marriage of beauty and function.

In collaboration with other disciplines, graphic designers translate concepts visually to persuade stakeholders. They produce ‘comps’ (short for composite or comprehensive) of the UI, advertisements, illustrations and corporate identity treatments. Some companies like to have their graphic designers produce CSS, thereby ensuring that every detail is captured in the finished product. When a graphic designer must compromise their design for technical reasons, an acceptable solution is arrived at more quickly with no friction between development and design. It’s helpful if your graphic desinger can converse in the terms of your technology.

The wider field of graphic design has its share of passionate folk. However, most that have moved to the technology sector have since matured of “artist’s ego”. A lot of compromise typically comes with crafting the surface layer of technology so only those who are flexible survive. Evoking emotional response, passion, flair and patience for refining details are hallmarks of the graphic designer.

Screening tips: Test for an understanding of branding beyond the visual, moving into interaction and messaging. Be sure they embrace usability concepts and processes and are as concerned with user comprehension as beauty. Gather evidence of “willingness to compromise”. Do they value what other UX disciplines bring to the team? Ensure they understand CSS or the constraints of your particular interface technology. How concerned are they with engaging the emotions of the user?

Writing Skills

Good writers can effortlessly guide users through an interface with concise instructional copy. They have the ability to create memorable taglines, deduce complex concepts into layman’s terms and author well-researched and thoughtful articles. Great writers have honed their skills well beyond what we learned in high-school English.

Steve Calde from Cooper says in his article Technical Writing and Interaction Design, writers have a pivotal role to play in the interaction design process: “As the first people actually trying to explain how the product works in users’ terms, technical writers are in a unique position to spot problems.” He is speaking from the technical writing perspective.

When we talk about writing to express a brand, there is a synergy between all disciplines committed to creating a strong voice. A writer’s ability to express the brand through phraseology is key not only for creating associative messages for the customer, but also for driving home a subtle Interaction or Visual Design personality.

Other than manuals or help files, instructions, labels, advertising headlines and copy, a deliverable missing from many UX teams is a style guide that details how concepts are to be expressed. Do you currently have a clearly articulated and documented voice and style?

Writing requires patience. Language allows us to express ourselves in many different ways and it can be a contentious area for stakeholders concerned with the message sent to readers. Therefore, subjective rework can happen, especially with highly visible projects. Empathetic people make good technical writers since they can quickly learn to speak the language of an audience who needs them to be clear. Equally, those exhibiting flair and wit often craft great marketing material.

Screening Tips: Are they comfortable with language? Can they demonstrate a command of the language to explain or sell ideas. Can they demonstrate how you create a ‘Brand Voice’ and keep it consistent?

While skills are important, less tangible qualities are arguably more so. With time skills are developed, but people who are creative or analytical, strategic or tactical, directive or hands-on are like this by nature. It behooves the hiring manager to identify which of these qualities are needed. In the next part of this article, we will look at some of the less tangible qualities of UX Dreamteam members and organizational contexts that determine which skills you really need.

Getting Hired

by:   |  Posted on

During a heated discussion on the difference between an Information Architect (IA) and an Interaction Designer (IxD), I suggested that what we do is more important than what we call ourselves. The response was that a label is an alias that carries a set of meanings. Yes, but what happens when there are two aliases that are very closely aligned? We can choose the alias we feel fits us best, but what do employers think?

As the User Experience Network (UXnet) local ambassador for the D.C. Metro area, one of my responsibilities is supporting local UX-related groups. Austin Govella, an IA colleague, thought UXnet should help get some answers to the question of what matters to employers, so we began to work on an event to gather professionals and employers to help us figure this out.

The ensuing event, titled IA Round-up, was a discussion panel and workshop where IAs, IxDs, usability professionals, and their employers came together to discuss what employers care about and what the perfect resume should look like.

The panel included three individuals representing three different types of employers: the agency, the corporation, and the small business. On the agency side, Dan Brown, principal of EightShapes, gave us a clear understanding of the agency perspective. On the corporate side, Livia Labate, senior manager of information architecture and usability at Comcast, outlined the best strategy to get a job with a large corporation. On the small business side, Michele Marut, human factors specialist at Respironics, Inc., described what she looks for. And I, Olga Howard, MC’d the event.

At the IA Round-up we found two reasons why employees and their potential employers may not find the right match:

  1. The terms used by professionals and employers sometimes mean different things.
  2. Resumes and portfolios may not sufficiently explain the work involved, or there may not be enough samples of work–wireframes, taxonomies, etc.

What Employers Care About

Employers have very specific needs and won’t spend much time trying to figure out the difference between an IA and an IxD. They just want their position filled. So while IAs and IxDs are having heated debates, employers pay attention to our resumes – that’s where semantics matter. The following key areas show how we can improve our resumes.

Paint a picture with your documentation:
Accurately describing documentation is difficult, if not impossible. It’s simpler just to show the documents themselves–they tell the story of where we started, where we ended, and how we got there. Unfortunately, we live in a Non-Disclosure Agreement (NDA) world that usually prevents us from showing our documentation. Regardless, according to our panelists, they’d rather see a highly censored document than no document at all.

Include only what employers ask for:
This is a tricky one. Most resumes tend to include what employers ask for, but some of us add other qualifications because we’re concerned the employer won’t see the breadth of our experience.

Present a sense of purpose:
This is the number one issue we heard from our panelists. When we put everything on the resume, the perspective on what’s important is lost.

Include a job history:
Every employer wants to know what jobs we’ve had, what we’ve accomplished, and how we accomplished it. Employers are also looking for employment gaps: if there are any, say why.

Be truthful and promote yourself:
A truthful resume is not the same thing as a factual resume. When we are part of a team we should say which areas of the project we were responsible for.

Create a straightforward resume:
Personality should not be part of the resume. Instead, focus on factual information. If our experience describes the kind of skills and knowledge the employer is looking for, they will want to see examples of our work—our portfolios.

Have a portfolio online:
Although we are bound by NDA rules, we can censor as much as necessary. As our panelists said, they’d rather see a highly censored document than no document at all.

Formalize your UX portfolio:
Lack of formality in presenting a portfolio is like a photographer showing you her photographs in a pile rather than neatly stored, each in a plastic sheet, ready for easy viewing.

What employers are looking for in portfolios is HOW we like to do our work. This is really where your personality shows.

  • Are you attentive to detail?
  • Do you communicate clearly?
  • Do you spend time only on the important aspects of the job?

Unfortunately, the portfolio is where most of us lack clarity. In your portfolio, you should include scans of sketches, drawings, and anything else you use to do your job.

Some people include odes to their heroes, and that’s ok in the portfolio. It speaks to their work and values.

Changing Careers

UX is so new that universities have just begun to offer degree programs. Although many of us actually started in another line of work, there are established communities of practice that new UX professionals should turn to, get involved in, and learn from.

Transferable skills:
A number of skills from other fields transfer to IA, but the only clear way to understand what these skills are is to read about IA, IxD, and usability and start volunteering to do projects. The IA Institute offers a mentorship program, and UXnet is always looking for volunteers.

Once you begin working in the field, you’ll know what strengths you can present to employers. Being new sometimes makes it difficult to have an opinion about the UX conversation going on, but you have a unique perspective and that’s what matters, so have an opinion.

The question of age:
What a nervous experience it must be to be older than your UX peers and compete for the same job. If you are this person, you have years of experience behind you. You have strengths younger UXers probably don’t have, so pay close attention to the job description and play to your strengths. One example is the person who has been a manager for many years. This person can play to their managerial strengths and speak to supporting the UX team in UX work. Employers are usually willing to build roles around your strengths.

One issue raised is that some older people are set in their ways. That is to say, set in the ways and processes that were in place during their tenure. These days, things change so fast that it’s hard to keep up with new thoughts and ideas, so older folks looking to work in UX need to be extremely flexible and adaptable to different processes and cultures.

Two questions you can ask yourself before moving to UX are:

  1. Why are you interested?
  2. Given that culture is a large aspect of work, will you add to the culture?

Next Steps

How much are you worth?
Find out how much other UX professionals are getting paid. This will give you a good idea of what salary you should ask for. The IA Institute Salary Survey and the Aquent Survey of Design Salaries will be helpful.

Where can you find job listings?
You can find great job listings on several websites including here in the Boxes and Arrows jobs section, the IA Institute job board, and the IxDA jobs section.

How can you get help with your resume?
If you need more help, the IA Institute’s mentoring program is a good place to start. Even if you don’t find a mentor in your area, you’ll find very friendly IAI members who will help you out. You can also contact your UXnet Local Ambassador and host your own IA Round-up. This will help give you context as to what local UX employers are looking for.

For formatting direction also try using Livia’s resume template below.

First Last
123 Name St, City, ST

(000) 000-0000 | first.last@firstlast.com | http://firstlast.com/portfolio

High-Level Summary/Goals as an IA: where you see yourself as an IA, what you like to do

Month YY to Month YY: My Title, Company Name, Location
– Two or three sentences describing responsibilities go here.
– Your favorite, proudest accomplishment goes here
– Your second greatest accomplishment goes here
– Your third relevant accomplishment goes here

(Repeat for as many relevant jobs as you want to show.)

Degree Title, YYY, Institution Degree Title, YYY, Institution

You can find Livia’s direction and template at Livlab.com.