IA Summit 10 – Dan Roam Keynote

IA Summit 2010

This year marks the 11th annual Information Architecture Summit. Our theme is meant to inspire everyone in the community—even those who aren’t presenting or volunteering—to bring their best ideas to the table.

As busy practitioners, we rarely have the chance to step back and think about the future of our field—we’re too busy resolving day-to-day issues. By gathering and sharing practical solutions for everyday challenges, we can create more breathing room to plan for what’s to come.

Subscribe to the Boxes and Arrows Podcast in iTunes or add this page to your Del.icio.us account:

iTunes     Del.icio.us     2010 IA Summit theme music generously provided by Bumper Tunes


| Day 1 – Dan Roam | “Day 2 – Richard Saul Wurman“:http://boxesandarrows.wpengine.com/view/ia-summit-10-richard | “Day 3 – Whitney Hess“:http://boxesandarrows.wpengine.com/view/ia-summit-10-whitney |

Full Program

| Day 1 | Day 2 | Day 3 |

Day 1 Keynote – Dan Roam

Dan Roam shares his unique visual-thinking approach that helps solve complex problems.

In his day one keynote from the 2010 IA Summit, Dan Roam—founder of Digital Roam Inc and author of the best-selling Back of the Napkin: Solving Problems and Selling Ideas with Pictures—shares his unique visual-thinking approach with a receptive crowd in Phoenix. Transcending language barriers, his approach helps solve complex problems through visual thinking, and has helped resolve challenges at many businesses: Microsoft, Wal-Mart, and eBay to name a few.

Note: As you might imagine, This presentation is VERY visual. As a result, the best way to view this presentation is to download it “with the visuals”:http://files.boxesandarrows.com/podcasts/Dan_Roam.m4a or subscribe to the B&A “iTunes feed”:http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=275459507.

Download mp3 (audio only)
Download m4a (with visuals, requires iTunes, Quicktime, or similar)

These podcasts are sponsored by:

MAD*POW logo
At Mad*Pow, they leverage the disciplines of Human Factors, Psychology, and Visual Design to create engaging experience that maximize customer acquisition, increase attention, and reduce costs.

ASIS&T logo
The American Society of Information Science & Technology: Since 1937, ASIS&T has been THE society for information professionals leading the search for new and better theories, techniques, and technologies to improve access to information.

IA Summit 2010
The IA Summit: the premier gathering place for information architects and other user experience professionals.

The design behind the design
Boxes & Arrows: Since 2001, Boxes & Arrows has been a peer-written journal promoting contributors who want to provoke thinking, push limits, and teach a few things along the way.

Contribute as an editor or author, and get your ideas out there. boxesandarrows.com/about/participate

Transcript of Dan Roam Keynote from Day 1 of the 2010 IA Summit in Phoenix, Arizona.

Announcer: In this day one keynote address from the 2010 IA Summit, Dan Roam, founder of Digital Roam, Inc. and author of the best selling book, “Back of the Napkin: Solving Problems and Selling Ideas with Pictures,” shares his unique visual thinking approach. Transcending language barriers, his approach helps solve complex problems through visual thinking and has helped resolve challenges at many businesses including Microsoft, Wal Mart and eBay. I hope everyone enjoys the broadcast. Cheers.
Jennifer:  Our keynote speaker today, Dan Roam, has inspired a revolution in sketching. Sketching is a technique that allows our hand to help our brain think, making our technology more about humans, and taking back design and communication from machines. Going straight to the computer or the slide deck, locks in our thinking. We need to set our minds free. This is important for us because we have complex problems to solve in our work and we can do this with pictures. Please give Dan Roam a warm welcome.
Dan Roam: Thank you. Thank you, Jennifer. Thank you very much. Thank you all for coming. You know I always say that, I always thank everybody for coming, but the reality is, I want to thank all of you for inviting me to come and share some of my ideas with you. I wanted to start this morning, with a little bit of a story. About four years ago, as a matter of fact, I was checking in my calendar, four years ago, almost to the day, I was working as an IA and a user experience lead at a company out in San Francisco, at Razor Fish out in San Francisco, and one day I had just a horrible meeting with the sales team of the company. I don’t mean to point a finger at Razor Fish, it was a wonderful company, but I had a really horrible meeting. And I thought, “I’ve had enough of this. I’m going to go do something else. I’m going to go write that book that I’ve been thinking about for so long.”
But the fact is, you know, I’ve got a family, I’ve got two kids, I’ve just moved to San Francisco, things cost a lot of money, I have no idea how to write a book. So I thought, “Well, I’m going to call a couple of friends of mine, colleagues of mine who have written books and find out what it takes.” So I called a guy named Steve Krug, who wrote a book called, “Don’t Make Me Think,” I’m sure everybody’s familiar with “Don’t Make Me Think.” “Don’t Make Me Think” is without a doubt, the best book on web usability ever, but I also think it’s one of the best books ever just on thinking.
And I called Steve and I said, “So what do you do to write a book?” And he gave me a lot of advice, he told me about agents, publishers, proposals. A whole bunch of good insights. And then he said, “There’s this other guy that you should call who a few years ago co-wrote a book which is the book on information architecture.” He said, “You should call my colleague Lou Rosenfeld, because Lou will be able to give you a whole lot more information about what it takes to actually write a book.” So I had never spoken to Lou, I called him up and I said, “Lou, you know, I want to work on this book.” And Lou was full of all kinds of ideas again, about agents, good or bad, publishers, good or bad, how do you do it.
So the fact is, here we are now, four years later, and I went ahead and I did write that book and the book has been very successful. It’s been very exciting, “The Back of the Napkin,” has done really well, which is wonderful, but the reality of it is, the success of the book is largely due enormous credit back to Lou and the information architect community because this is where I come from.
So about three or four months ago, Lou sent me an email asking if I would be interested in giving a talk at the Information Architect Summit and I said, “Absolutely.” I mean, this to me is like one of the most perfect opportunities to share this idea because in a way, I spend a lot of time talking to organizations that I don’t know anything about. And it’s kind of a scary thing, and we’ll go through several examples of that, so it’s very nice to be able to come and talk to a group of people where I at least like to think that we share an essential base of information of where we come from and where we’re starting from.
And that is not often the case when, I’ll, let me put it this way. The best part about writing a book, I know there are probably a lot of people in here who have done books. How many people here have written books and had them published? I know there’s a lot of people I’ve been meeting. Well I want to give all of you an enormous hand [applause] because I know what’s involved. It is an extraordinarily difficult thing. Writing the book itself, in my opinion is no fun at all. Writing the book is you’re alone, and you’re in your room with your computer or your drawing or whatever, it’s a very lonely process. But the best part about it is after you’ve done the book, then you get invited to go give talks. And then you get to share your idea with all kinds of companies and organizations.
So over the last, three years now, I really have had this extraordinary opportunity to share these ideas with this really incredibly array of businesses and organizations. And as I was mentioning before, in most cases I’ll go in and I don’t know a whole lot about these companies when I show up. So one recent example, this was already two years ago. I am no aeronautical engineer, but I had to go, a chance to go and give a workshop at Boeing, up in Seattle. And it was phenomenal for me because what I ended up doing was being able to spend about half a day with the project managers that are building, that are behind the 787, the new dreamliner that just had its first flight a couple of months ago.
It was magnificent because they explained to me, how do you build what is arguably the most sophisticated, advanced, meticulous machine that has ever been conceived and the amazing part is that it’s being built simultaneously in something like 23 different countries at the same time. And in 16 different languages. How do you build something that is both that big, that new, is built down to tolerances of less than hundreds of thousandths of an inch in 16 different languages? Well the answer is very simple, you do everything with pictures. Everything is being done with pictures. And I thought, “That was really interesting.”
More recently, another organization, one that I know absolutely nothing about but I had a chance to go in and address was the United States Senate. So the Senate, the New Policy Committee of the United States Senate, about a year ago, gosh, it’s a little more than a year ago now, asked me to come out and give a similar workshop. And I don’t have a background in politics, I think, I’d like to think that I have a vague understanding of how Washington, DC works. I know there are a bunch of people here from Washington, DC and I think you’ll agree with me that nobody really knows how Washington, DC works. I came out of this after, it was a wonderful session, I learned a tremendous amount, I’ll admit at the end, I still have no idea of what the Senate really does, but again, the motivator there was: Is it possible to find ways to communicate issues about complex policy through this use of simple pictures?
And I think that the answer is yes. And I think many of the people in the Senate now think that the answer is yes, too. So in the end, what I wanted to share with you is that I have a very simple proposition that I make to all of these different businesses and organizations. And it goes like this. No matter how good everything may be in our lives, or in our work, there is something that we all do have in common which is that not everything is perfect. I mean we all do have some problems.
Well the proposition that I’d like to make is very simple, and it’s this: Whatever our problems are, we can solve our problems with pictures. I mean this completely, it’s a very simple statement but I know it to be absolutely true. We can solve our problems with pictures. Now the reason I can say that as superficial as that sounds, and say it with such incredible conviction because I know it is true is because I have never seen this process not work.
That is to say, every single time people are working together on something, on addressing some problem or challenge or trying to understand a concept, and someone starts drawing out what the other people are talking about, every single time it helps everybody get together on understanding what the problem is. And more often than not, by virtue of creating that simple picture, everybody starts to then see, not what the problem is anymore, but already begins to see what the solution is going to be. It’s already inherent in the picture that you’re creating. We’ll talk more about this in detail.
But I also recognize, and I’m willing to guess that with an audience like this, I’m just going to go out on a limb and guess that probably for most of you what I’m saying right now is not really a surprise or is probably not very new. I’m going to guess that in a room full of information architects, if my experience is the same as yours, we probably are the people who spend the greatest amount of time of just about anybody trying to understand what is the nature of this big problem that the client has brought to us and we do it more often than not by really drawing things out.
I’m not talking about drawing beautiful pictures. I’m talking about maps, schematics, concept models, mind maps. How do I get all these ideas together in a way where I can see them? But that is not the nature of the audiences, what I’ve just said, is not the nature of the audiences that I’m usually talking to. Project managers, financial executives, CEOs. I’ll say, “We can solve problems with pictures,” and they look at me cross‑eyed, they say, “What are you completely out of your mind?”
What they’ll often to say, if they think that through, the really clever people will say “Dan, OK, I’m going to play with you for a moment. Let’s assume you’re right. We can solve problems with pictures. Let’s break that down into three component questions. Which problems are we talking about? Which pictures are we talking about?” And then the third most contentious of all, “Which people are we talking about?” You know, namely, who is going to do this, “Because let’s face it, you know, I’m not visual.” I like those three questions, and in fact, those three questions are really going to be the underpinning of everything we’ll talk about for the next hour or so.
And I’m just going to run through them. Here’s the Cliff’s Notes, super fast, executive summary answer to all three of them. “Which problems are we talking about?” Any problem. Think about it like this: Any problem that we have the ability to articulate at all, we have the ability to articulate infinitely more clearly through the use of pictures, which brings us to question number two. So “Which pictures are we talking about?” I mean, if these pictures are going to help us solve any problems we can conceive of, they must be really sophisticated pictures, right? That must require at a minimum years of training, and probably some really sophisticated and expensive computer software to create, right? Well, you know where I’m going. The answer is absolute not. The pictures are bone-headedly simple.
Now, back to what I had said a little bit earlier. Everybody should be sitting on a napkin. If you’re not, look under your bum and see if you can find a napkin, there should be a napkin somewhere around there. What I’d like you to do, does everyone have, we asked this once before, is there anybody in here that doesn’t have a writing instrument? OK, volunteers. Lou? You don’t have a pen. That’s excellent. Yay for the information architects.
All right, if anybody doesn’t have a pen, we have volunteers who will happily give you a pen. What I’d like you to do for a warm up exercise, we’re going to really work out this napkin, we’re going to use it several times.
So, just, the pictures are if you can draw a square, and how many of you don’t know this, and you can draw a circle, and you can draw an arrow connecting them, and the most challenge of all, of course, I’d like everyone to try, draw a little stick figure. Make it a happy stick figure. If you can draw those things, you can draw every picture that we’re going to talk about, which automatically answers this third question. “Now, who’s going to do this, because I’m not visual.” Yes, you are. Everybody is going to be able to do this. Let me just throw out a couple of data points right from the beginning for anybody who might still be a holdout against this idea that pictures can help us solve problems.
Of all the neurons in our brain that are processing incoming sensory information, so that is to say the entire capacity that we have for understanding the world around us through all of our senses for bringing the information in. Let’s do it with a picture. This is our entire sensory capacity. What percent of that is visual? Three quarters of those neurons is focused on vision. It is arguable and there are neuroscientists who do argue this that it could be said that if you take all of the capacity of our brain to do anything, the one category of stuff that we have the greatest capacity to do of anything is to see.
More of our brain is dedicated to that than any other single thing that we do. Vision is fundamentally what we’re about. I mean, for people who might still be holding out and saying “Oh I’m not visual,” let’s keep the bar really low. If you’re visual enough to walk into the room and sit down without falling down, you’re visual enough, because the process of doing that, the extraordinary process of doing that already tells us how amazing this system is that we have. So there you have it. Any problem, simple pictures, everybody.
Now, one of the things that I have learned, and this was not, this is not in my original book. This is in the unfolding book, the second book that came out, because this was something I learned in giving this talk or talks like this many many times, is I’ve been looking, and people have been bringing to me these underlying reasons, these unwritten rules of why visual problem solving really does work. And I’m going to take you, I’ve identified four of them, and I want to take you through two of them today. They really kind of represent the understructure of what it is we’re talking about.
Unwritten visual problem solving rule number one says this: “Whoever best describes the problem is the person most likely to solve the problem.” So the idea is this. If one of us were to go running into the boss’s office and say, “Oh my god, the sky is falling ‑ give me money to fix it,” they’ll probably throw us out. But if we went into the room and we said, “Look. I’ve created this map and it identifies who’s involved in this particular problem, how many of them are there, where are these things involved or these things involved, how do they overlap, when do they intersect and how do they intersect.” All of a sudden, the solution to the problem is probably going to be already very clear. So the mercenary subtext to this rule is, and this is absolutely true, “Whoever draws the best picture gets the funding.”
Dan Roam: I’m going to give you a couple of scary examples of this being true. Before I do, I want to do a quick little usability test, because for later on this will be important. Is there anybody in particular in the back of the room who cannot read the slide? This is the smallest text we’re going to have on any slide and there will be some later on that we’ll need to read. So if anybody’s having trouble reading this slide, please move up to the front if you can. Even bring a chair. Because we will need you to be able to read at least that size, so a little quick usability test. Now I want to give you an example of this rule. “Whoever draws the best picture gets the funding,” and it goes far deeper than that.
I want to start by just taking a little trip from where we are here in Phoenix. I live out here in San Francisco, so I flew here yesterday. We’re going to fly out to Washington, DC, but before we do that, does anyone want to know, guess, can anyone imagine why I’m using a Southwest Airlines napkin as my route map? If you know, don’t tell us. Because the greatest back of the napkin business success story of all time took place in 1967 back in San Antonio, Texas. There’s some people here from Texas, yes? There’s a few people from Texas, yeah.
All right, well back in 1967, two guys are sitting in a bar. The St. Antony’s Club in San Antonio. And they’re talking about a business idea. And one of the guys ‑ and I swear this is true. His name is Roland. Roland takes his ‑ we don’t know what they were drinking, but we knew what he drew because they saved the napkin. He said, “Look, here’s Texas. We have Houston down here. We have Dallas up here. And we have San Antonio over here. Why don’t we make an airline that just connects those cities?” And then he drew the triangle of fate.
That’s the kind of picture I’m talking about. That back of a napkin sketch became the basis for Southwest Airlines. Southwest Airlines was started on the back of that napkin. Southwest has gone on to be the most profitable and financially successful airline in history. To this day, it is the most financially successful airline in history. And in fact, dozens of other airlines, from jetBlue to Easy Jet over in Europe to Ryanair have all copied the Southwest model, all of which began with this very simply picture on the back of the napkin. So that’s why I like to use this napkin.
Anyway, back to DC, I was asked as I mentioned before to come out to the US Senate and give a talk. So it was the new policy committee. And before going to give the talk, as I hope all of us would do, I went in and tried to do some research so I could say I have lots of examples from business and information architecture about the use of simple pictures helping solve problems. But I need to find something from politics. But I couldn’t really find anything. I was doing my research, but I found something else that was really interesting and I want to share it with you.
This is a map of Mt. Vernon. This map was drawn, the date’s right up there, in 1776. Mt. Vernon, of course, was George Washington’s estate. Does anyone want to guess who might have drawn this map? It was George Washington’s estate. George Washington drew the map. I didn’t know this. George Washington was trained as a map maker, a surveyor, and a cartographer. And in his notebooks, they’re full of his sketches. I thought: “That’s pretty interesting, let’s continue this line of thinking.”
So here’s another one. This is White House stationery, this is actually Oval Office stationery. Someone is drawing a picture of a boat. It looks like a chessboard with an eraser, a flag that says “NATO” on it, blockade Cuba in a circle. Does anyone want to guess who might have been drawing these pictures? This was JFK. That’s right, John F. Kennedy was drawing these pictures during the Cuban Missile Crisis. These are the doodles that were taken from this notebook as he was talking on the phone, trying to avoid nuclear Armageddon.
Here’s an interesting one. Anyone want to guess what President might have drawn this? And what could that fellow have been thinking? Nixon, absolutely. Very good.
This was Richard Nixon. There have been studies done, sort of forensic IQ tests going back in time, trying to decipher what would have been to unearth, what would have been the IQ of various Presidents. It turns out that Nixon is probably one of the smartest people from an IQ perspective who’s ever been in the White House. But clearly that guy had a lot of issues.
Dan Roam:  You’ve really got to wonder what does that picture represent? Well now here’s a nice easy one.
Dan Roam: Who might have been drawing these pictures? That’s right. This was President Regan and I swear that was taken when he was in one of his cabinet meetings. Those were the pictures that he was drawing at that particular cabinet meeting.
Dan Roam:  So I thought that was very interesting. Those were nice pictures, they’re kind of funny, they’re kind of interesting. After the talk, a guy named Doug Steiger, who’s the head of new policy for the Democratic side of the senate‑‑came up to me and said, “Dan, great talk. Thank you. But I’m going to tell you the best political back of the napkin story ever.” He told me the story and I have checked it out. It’s absolutely true. It involves a guy who is an economist back in the ’70s named Arthur Laffer, who was with USC. But he was a consultant in Washingto,n DC in the ’70s. So Laffer is sitting in a bar again, in DC, with two other guys from the administration, that time President Ford administration. Again, we don’t know what they were drinking but we do know what they were drawing. They got talking about taxation. Laffer on his napkin drew the following picture. It’s a simple X‑Y plot, same thing many of us have drawn thousands of times, I’m sure. On the horizontal axis he plotted out the percent tax rate that the US Government is going to charge us on our income from 0% up to 100%. On the vertical axis, he plotted out the amount of money that the government actually collects in taxation from lots and lots of money down to no money.
He said, “OK. So guys,” and it was all men at that time, they’re all sitting at the bar, the boy’s club. He says, “Think about this. If the government charges 0% income tax, how much money is the government going to make? 0%.” He said, “But think about this, if the government charges us 100% income tax, how much money is the government going to make? Also zero, because no one will work.” If we have to pay 100% of our income back as tax, what’s the point? I’m not going to work at all. So then he drew something which became known as the Laffer curve. He drew a curve and said, “In fact there is some curve that connects these and isn’t it interesting that at some point, reducing the rate of taxation actually increases the amount of money that the government collects.”
Now the guys who were with him at the table found this fascinating. “Take us through that again.” Reducing taxes increases government collections. Wow! They really liked that. “Can we take that napkin?” He said, “Absolutely.” These two guys took it back with them to their boss. They were both chiefs of staff of President Ford. They gave him that napkin. They said look at this idea. That napkin made its way into the hands of the Republican National committee and into the hands of the Regan economic team. That napkin became the basis of Reaganomics, of supply side economics. The idea particular being, reducing the rate of taxation in particular for the most wealthy increases activity in the market and increases the amount of money that the government actually collects.
That napkin sketch became the basis of Reaganomics. Regan, as much as I may make fun of him with his doodles, when someone would come to him and say, “Wait a minute. Tell me this again. You’re reducing taxes in order to increase revenue for the government? How does that work?” He would draw that picture, pretty convincing picture.
Now, the interesting thing is that these two guys who were sitting at the table with Arthur Laffer that night are these two guys.
Who says a simple sketch on the back of a napkin does not have extraordinary influence? It absolutely does. Whoever draws the picture gets the funding. Whoever draws the best explanation, of the idea is the one that people will believe. Why? Because it’s simple. I can understand it.
Now the Laffer curve, ever since has been debated endlessly. Where is the curve? Is the fundamental assumption correct? Doesn’t matter. He drew the picture. That’s the picture that wins.
Now, moving along, we are obviously in a new era. Who might have drawn this picture? Exactly right. President Obama drew this picture. Turns out, our President can draw extraordinarily well. It turns out also that our President is left handed. Now that by and of itself doesn’t mean anything. But we do know that there appears to be some correlation between people who are left‑handed and may be more spatial in their thinking.
Get this, I just did this math the other day. Five of the last seven US Presidents have been left handed. That is a really crazy number. Five of the last seven. Regan was a forced righty. He was naturally left handed. But through education, at that time was forced to become right handed. So Obama, Clinton, Papa Bush, Regan and Ford were all left handed Presidents. Pretty remarkable when you think about it.
So the question I have… Regardless of your feelings about our present administration might be, I think everybody can acknowledge, and I have said this, all over the country, everybody agrees that President Obama is one of the greatest public speakers that anybody’s ever seen. There’s no question that verbally, he’s one of the most articulate and passionate conveyers of information and thoughts we’ve ever had.
But the question I have is given the fact that he can draw, and draw extraordinarily well, why is it that he’s not drawing pictures to help explain some of the extraordinarily difficult problems that we’re facing? Whether it’s the economy, whether it’s global climate change, whether it’s Afghanistan. All of these challenges, and in particular I want to focus for a few moments on healthcare. This is not going to become a political conversation, I promise you.
Regardless of where you may fall on the political spectrum around your feelings of this healthcare so called debate that has taken place over the last year‑‑the intent from all people involved could not have been the anger and anxiety that we have seen exhibited in the last few months. By no stretch of the imagination could this have been the intent.
This horrible anger, that’s splitting the country around healthcare just doesn’t make any sense. I think the real problem, and I know this is true, the real problem is not so much accepting a lunatic fringe all over the place, accepting that the problem isn’t that people disagree with what’s being said in Washington. The problem is that people don’t understand what is being said in Washington.
We all know healthcare passed. How many people in here are confident that they understand what the new healthcare bill actually says? We’ve had this battle that’s become in some people’s mind, the virtual new civil war regarding healthcare. But nobody understands what the actual legislation is about. That is the fault of our elected officials. Why is President Obama not drawing a picture? We’ll talk this through in great detail.
Instead, I want to know what it is that DC is actually conveying to us in terms of their information. Talk about information architecture. This is the actual house bill passed back in October. The house healthcare bill. You can download all of these things online. I downloaded it. I said this represents this enormous shift in the way American government is handled that will impact all of us. This is an important piece of government documentation. Someone must have the vision. I use that word intentionally. The vision of what this healthcare reform is about. What does it actually look like? Why are we changing what we have now? Good or bad as it may be for something else. There must be a picture.
Well, I thought, “This is an important government document. So of course, nobody is going to put a picture, a sight map, a mind map on the fist page.” So I continued looking and no, there are no charts or diagrams or maps or vision documents, images anywhere in the first eight pages. Not in the first 64 pages.
Dan Roam:  Not in the first 200 pages. Nowhere in the 1,447 pages, there’s not a chart, there’s not a graph, there’s not a sight map. There’s not a single diagram that says this visually is what it means to shift from this particular model to this particular model. This is an unreadable document. Nobody can understand it.
Is it any wonder that [laughs] some people would claim we’re on the verge of civil war about this. Because nobody actually understands what’s in it. I thought, putting my money where my mouth is, what would happen if someone tried to draw some pictures of what this healthcare debate is actually about? Now, I thought, I’ll do it. Why not? I have worked with healthcare companies in the past as a consultant. I know just enough to be really dangerous but the good new is I have met healthcare consultants who know a lot.
So I called one of the best, one of the smartest consultants I’ve ever worked with, a guy named Tony Jones, that Jennifer, you would know, who is a health care consultant, he’s an MD and an MBA, pretty interesting fellow, pretty interesting mix. Tony’s office is down in LA. I said, “Tony, I’m flying down there. I’m bringing along copies of the legislation,” this is about seven months ago, now. “And we are going to lock ourselves in your office with the white boards and we are not leaving until we’ve created a set of simple pictures that explain what is the business of health care in America today, what is the actual legislation that’s being debated, not about killing grandma and death panels, but the actual legislation that’s being debated, and how does that map into how the model might change.”
And so we did that. And I’m not going to take you through the whole thing, but I want to show you a couple of pictures that I excerpted from that document.
One of them, was this picture, which kind of lays the base out and says the number one thing we all need to understand baseline is that health care in America, unlike any other developed economy on earth, remains a business. It is all a profit driven business, that is in our DNA and that is what people at the end are really arguing about is whether health care should be a business or should it not. It boils down to that. But the real issues is it’s not just one business, it’s two businesses that are completely distinct.
One of those businesses is the business of the providers. These are the doctors and the hospitals and the pharma companies. Businesses that make money by making people healthy. At the other end is the business of the payer. These are the insurance companies. These are the organizations who make their profit by handling the payment of all of the money through this system. These two businesses hate each other because as a tax paying employed citizen, I am the only source of money going into this system. There is no other money miraculously being created. As an employed, tax paying person, I am the only one putting money into the system.
The doctors, the providers want more of my money to be able to do good things with that money and to earn a profit, fair enough, that’s what we do, and the health care companies want more of my money to be able to do good things and earn a profit because that’s what we do. It is getting so bad that I am running out of money, my business is failing because there’s not enough money to provide both sides in this equation, so government decides it’s time to step in and help. And to do that through regulation. And as you look at all of the legislation that was being debated, ninety five percent of it did not focus, anywhere on the doctor, on the provider side of the equation. All of the reform was focused on the insurance side of the equation. We’re going to reform the insurance companies.
In hindsight, I believe had the White House called this not “Health reform,” but “Insurance reform,” it would have passed without anyone batting an eye in a few months because everybody hates their insurance company.
In the end you could take all of the legislation that was being debated and map it onto a very simple spectrum from completely private, not restrictive, unregulated, unlegislated, private insurance, which is what the conservative side really wanted, all the way through a purely government owned, national health service kind of a model, which is never what the White House wanted. The White House initially wanted to have a private insurance supported by a public option as well. And you could map all of the legislation across this spectrum.
Now what we’ve ended up with, what has just passed is something like this. Health insurance companies are no longer able to throw you out because of pre‑existing conditions, or because you hit a limit, so what’s happened is that some of the regulations have been taken, have been put on them but there is no government option. So in the end that’s what we’ve ended up with. It’s debatable whether that’s a good thing or a bad thing but at least something.
So the long story short, I don’t want to go any further than that, is to say, I created that presentation about health care, I posted it on SlideShare, you all know SlideShare, it’s gotten a quarter of a million downloads. Now, that’s nothing compared to Lady Gaga’s new video or something, but let’s face it, this, now get this, this is a PowerPoint document about health care. What could be more boring on earth?
Yet we’ve got a quarter of a million downloads of people. And the comments are saying, regardless of where they come in on the political spectrum and believe me the comments come in from all sides of the political spectrum, some of them are scary. They all say at least, “Thank you for having clarified through these simple pictures what the essentials of the health care debate are actually about. Now that we understand what we’re debating, now we can eviscerate each other. At least now we know why we’re trying to kill each other. Thank you for clarifying that, now we know why.”
So it got picked up by the Huffington Post and then the Washington Post and then I get a call from Fox News. Now I had been on Fox before and yes, it’s Fox, now, I live in San Francisco‑ I know there’s someone here, from looking through the list of attendees, there are at least two people here who are from the Fox network so I need to be careful what I say.
I think in San Francisco that we have this sort of electromagnetic pulse signal that we send out that blocks the Fox signal from coming into the city. I don’t think it works all the time, but I’ve got to admit, I love going on Fox because they’re the people who like the drawings. So Fox asked me to come on in a prime time, this is remarkable, they gave me seven minutes on prime time, 5:00 PM. Eastern Standard Time, on Fox Business Channel to explain with my pictures to the Fox audience the essentials of American health care. And I thought, “This is magnificent. How wonderful is that?” We had a good time and people understood it, I think.
So then I get a call. Does anyone know where this is? Yeah. So then I get a call from the White House Office of Communication saying, “Dan, we have to talk.” You know, so I went to the meeting and it didn’t actually take place in the White House, it took place in the coffee shop across the street. It’s all very, you know, sort of, “All the President’s Men,” cloak and dagger sort of stuff. Because it turns out that the White House cannot hire consultants.
Has anybody every worked with the White House? Anybody here who’s had experience working with the White House? It’s very difficult for the White House to hire consultants because of issues around transparency, we want to make sure that every contract is vetted appropriately and it’s very challenging.
So what’s been happening is, we have started some discussions on how it might be possible to use these kind of simple pictures to clarify policy, not so much with the White House but through, interestingly enough, some of the government departments. And the two departments that I was told and have been helped a little bit to get into that are the most open to this kind of innovative thinking are the Department of Defense and the Department of State.
So I’ve had a little bit of an opportunity to work with the Department of Defense and I’ve got to tell you, it’s fascinating. It’s really interesting. Because these are the people, when I talked about bigger problems, I mean, the problems that need to be addressed are in some ways a little bit beyond the scope of what I remember as a consultant typically would be the scope of a problem that a client would bring to me. And it’s pretty fascinating to be involved in that a little bit.
So the question becomes, you know, why is it, if we have all these Presidents who are not drawing who could, why might that be? And I’m not going to buy the answer that it’s because we’re not visual. We’ve already proven that President Obama is visual, we know that. We know everybody’s visual. So why is it that the communications that come out of Washington are so difficult to understand? I mean there’s probably a lot of reasons, but here’s one thing that I’ve come up with.
We’re going to do a test here, in a moment. I have found in something like, 450 meetings or something, that in doing the test that we’re about to do, it does turn out that pretty much everybody falls across a very simple spectrum in terms of how we approach problems from a visual perspective. What I’ve found, I’m going to give you the result first and then we’ll do the test and see how the test matches to the result. What I tell you now will have zero impact on how you actually take the test.
What I have found is that in any meeting, it doesn’t matter what the industry is, what the level of people’s position within the company is within the meeting, whether they’re executives or newbies, you find that in any meeting, typically about 25 percent of the people, you get this really nice bell curve distribution, about twenty five percent are what we’re going to call a “Black pen person.”
Now just to give you a little, very quick overview. A black pen person, we black pen people are the ones who can’t wait after the meeting’s started, we can’t wait to run up to the white board and start drawing out ideas. And say, “Wait a minute, is this what we’re talking about?” And we know who we are.
About 50 percent of the people are what I’m going to call a “Yellow pen person,” a highlighter. These are, we are the people, we yellow pen people, who are sitting there watching this other person drawing and we’re kind of inspired by what we’re seeing. Our mind starts moving thinking, “Oh, there’s something there,” and we, every single time, invariably, it’s always the same, we stand up and say, “I can’t draw, but,” and then we say, “Do you mind if I add something?” What happens is, that’s why I call these like the highlighters, the yellow pen people, are really good sussing out in someone else’s drawing the area that’s really interesting to explore, and then we’ll maybe create our own little picture over here of that area and say, “This, I think, is worth pursuing.”
Now we’ve got a great combination between the two, between the black and the yellow pen people of creating this picture that is both big picture and starting to get into some of the details.
Well, for those of you with a statistics background, you’ll notice [laughter] that we’re missing about 25 percent of our people, we red pen people. And we are the ones who are watching these other idiots up there at the white board thinking, “You know, frankly this is all a bunch of crap because they’re so grossly oversimplifying the problem, they’re probably making it worse.”
I don’t mean to point a finger because we all wear these different hats at different times, but we red pen people are the ones who really do have the greatest grasp of the details and the facts. It really bothers us. It’s just horrible to see these simple pictures being created that are missing so many of the nuances and the important critical details.
But what did we remember from before? The person who draws the picture wins. We’ve got to get these red pen people. We have to participate. So here’s what we’re going to do now. We’re going to do a test. On your napkins what I’d like you to do is follow along with me for a few minutes as we’re going to do a test.
What’s going to happen is I’m going to ask you a series of questions. Here’s why in that usability test I wanted to make sure that everybody was able to read the projector. If there’s anybody who can’t read that size of type, I’m going to need you to move up to the front. We’re going to go through a series of questions.
I will pose a scenario like this one that says, “I’m in a brainstorming session in a conference room that has a white board.” Then I’ll present you a series of possible answers. What I’d like you to do is read through the five possible answers, pick the one that’s closest to what you would do, and write down that number on your napkin.
My wife used to be an art director at “Cosmopolitan Magazine,” and I always like to think of this for anybody who’s ever read “Cosmopolitan Magazine.” This is like the “Cosmo Sex Quiz.”
Dan Roam: Here’s the scenario. Here’s what we’re going to do. So think about it like that only without the sex part.
Dan Roam: So again, Question #1. I’m in a brainstorming session. There’s a white board. Here’s what I do. And I’ll give you a minute or so for each one. Does anybody need any more time? Are we all good? We go on to the next one? Someone hands me a pen and asks me to sketch out a particular idea.
Dan Roam: I saw a gentleman yesterday who had this beautiful bandolier. Who’s the gentleman with like 46 markers? That thing’s amazing. I saw it from the elevator from the 10th floor coming down. It was beautiful. Is that bandolier here in the room? Can we show it? Whoever has that, would you mind showing us what you have?
Hmm? Not here?
Dan Roam: All right. Well, does anyone know who’s the guy who has it?
Jennifer: Jess McGraw.
Dan Roam: Jess?
Jennifer: Jess McGraw.
Dan Roam: Find Jess. Jess, are you here? Oh. You don’t have the bandolier with you?
Jennifer:  The pens.
Dan Roam: Your pens? Everybody, I want you to accost this man later on and take a look at this.
Dan Roam: All right. Are we getting through here? Anybody need more time? A couple more seconds? The next scenario. Someone hands me a complicated spreadsheet and asks me to look it over. OK. I’m going to press on. Just a couple more. Traveling home from a conference, perhaps even this conference, I’m in the airport. I run into someone who I saw at the conference and they say, “Oh, I forget. Your name? Yes, Mary. And what do you do again?” I…? Explaining what I do. Explaining what my job is now.
Dan Roam: And you know what? As of this morning we’re going to change question number three to now say, “I pull out my iPad.”
Dan Roam: All right. We have one more to go. Everybody good? This is an easy one because we’ve all been there. I’m an astronaut floating in space. The first thing I do is…? Keep your comments to yourself. For now. You know who I’m talking to. As you go through that, we’re going to total these up. We’re now done with our test. But before we do, and there’s someone in the room who already said it, so I want you to be very quiet. By a show of hands, how many people noticed something odd in the sequencing of those questions? Raise your hand if you did. OK, so a quarter of the room.
The person who shouted it out, what’s the problem with the sequencing of the questions? There is no Question E. There was no Question E. We went from D to F. Now the reason why that is is because I was asked to come and give a two day conference at Pfizer out in New York. This was a couple years ago now.
Day One I was going to talk to the business strategy people, and on Day Two I was going to talk to the project management people. And on the flight out, I was going through the presentation one more time. I was doing exactly what you’re not supposed to do: sit on the plane going through my PowerPoint.
I thought, “This is just too long.” So I just started pulling pages out. And one of the pages was one of these questions. But I didn’t think to renumber the sequence. So I pulled E, threw it away, and didn’t think to renumber it.
The first day again was with the business strategy people. We did the test. We blew through it. Nobody said anything. The next day was with the project management people. As we were going through it, when I jumped from D to F, everybody in the room said, “Wait! Where’s E?”
Dan Roam: And I thought, “This is more important than the test itself!” The business strategy people, none of them either noticed or cared that there was no E.
Dan Roam: And the project management people, we could not move on until we’d resolved [laughter] the issue of the missing E. I think that’s more telling. So here’s the deal. What I’d like you to do is add up your numbers, and we’re just going to do a quick show of hands. Add them all up. How many people identified themselves as a “Black pen person?” OK, we’ve got, oh, I’d say it’s about a fifth or a sixth of the room. Let’s call it five percent. How many people identified themselves as a “Yellow pen person?” OK, it’s a lot. Let’s go to the other end. How many people identified themselves as a “Red pen person?” OK, it’s roughly…wait! Those hands didn’t stay up very long. It’s OK to be a red pen person.
Dan Roam: I’m a red pen person half the time. You should see what the editor’s like. So I’d say that it’s less. It’s maybe 10 percent. But we still get a distribution, so we get something like this. It’s a little bit less. It’s a little bit bigger and then a little bit less like that. Now here’s the scary thing. Why is it that nobody in Washington, DC draws pictures? It goes back to our educational system. I gave this test. I have given this test, as I mentioned, hundreds of times now and the answer is always some kind of distribution as we’ve seen. Most people are in the middle, and then you get some spread out over the sides, with one exception.
It blew my mind, and I swear to God that this is true. I gave a talk to the NEA, the National Education Association. Teachers and academic administrators, 150 people in the room. Every single person, the same test you just did, identified themselves as a “Red pen person.”
Our educational system! And who knows what is the cause and effect here? Where is the finger to be pointed? We don’t know. But what we can derive from this is in this limited test, highly invalid, highly suspect, but nevertheless compelling.
Dan Roam: If our teachers and our academic administrators 100 percent believe that a picture is not a particularly valid way to convey an idea, and that is wildly off from the distribution of how we actually believe we should solve problems, no wonder we’re afraid to draw. No wonder from the age of six no one is encouraged to continue to use visual problem solving as a viable way to test intelligence. The SAT test includes critical reading, writing, and math. Your determination of whether you’ll get into your university has absolutely nothing to do with your ability to visually solve a problem, absolutely nothing to do. No wonder that by the time people ascend to the level of leadership, this is not going to happen.
And that is an enormous mistake because what it means is every time when we finally do get pictures in a business meeting, they all look like this. Why is it that given this broad range of visual talents and abilities that we have, when it comes time to communicating in a business setting this is what we always get?
In all fairness for those of us in the room, when it comes time to visual communicating, this is often what we generate. How many people have ever made a picture that maybe looks something like that? My beautiful site map that I labored over for weeks, and then I showed it to the client and they ran out of the room?
Dan Roam: Now I debated heavily whether I show you the next picture or not. I’m going to show it to you. This is a picture I don’t like. This happens to be the poster that all of us were given. I didn’t know that until last night. Who am I sitting with at dinner but Dave Gray, of X‑Plane who’s company made the poster. I’m thinking, “Oh God! I love this picture. I love to look at it. It’s beautiful and wonderful. It does absolutely nothing to me to explain how a website gets made which is what the picture’s about.” I think the type of pictures that I’m talking about, this is not what we want to be doing. Now, I ran, I told Dave I was going to show it anyway. He said that was OK. Is it still OK that I show it? OK.
Dan Roam: It’s beautiful, make no mistake. I have made pictures like this and I love doing them. But I’ve got to realize that it’s same with my beautiful sight map that I worked on for so many days. The intention, at the level I’m talking about, this is not every level but at the level that I’m talking about now, is to communicate ‑‑ to get what’s in my head into your head in the fastest, most efficient and most believable way possible. If I wanted to explain how a website got made, I would not do something like this. What I hope I would be able to do one day is to make something that’s very simple. So again, point being this simple picture on the back of a napkin. Now, for the rest of this session, we have about half an hour to go. I want you on your napkin to follow along with me as we figure out a way to draw a simple picture, a napkin picture of any problem that we can conceive of.
Here’s how we’re going to start. This is the way we start. Every back of the napkin problem solving picture, every single one, we don’t think about what our in‑goal’s going to be. No. Nor do we think about how would I even start. We remove that from the equation. We start always the same way. Just draw a circle. In the upper‑left hand corner draw a circle. This is the way I recommend starting every picture no matter what it’s about. Draw a circle and then give it a name.
In this particular case, call it “Me.” For a little extra credit, go ahead and make it look like me or you. Then we draw a second circle and make this one bigger and kind of fluffy down here in the middle. We’re going to label this one “My problem.” Now what’s happening is by virtue of our making these simple drawings on that napkin, and our being able to see them, a whole bunch of channels, neurobiologically speaking, in our brain are now opening up that would not have opened up if we just talked about it.
If I told you, “Imagine a relatively small circle at the upper left hand side of your page with a label ‘Me’ under it, and now imagine a bigger circle in the middle that’s called ‘My problem’,” an entirely different set of neurons are firing than are when we draw the picture. When we can actually see it and talk about it. We’ve now got all cylinders firing. What our brain really loves to do, and this is why PowerPoints so often do fail, our brain more than anything else gets excited when it understands something. The same kind of endorphins and dopamine that fire off when we get really excited are the same ones that fire off when we understand something.
When we suddenly have that “Oh my God, I get it!” moment, it’s like a shiver goes down the back of our spine. Our brain wants to understand. When someone gets up in front of us and starts to present something, we really want to get it. So our job as the communicator is to eliminate every thing from what we’re showing that’s going to stop the person from getting it. Because we want them to understand it.
So our brain is now ready by drawing these simple little pictures. We want to know what the connection between them is. Why is one bigger than the other one? What’s the next circle going to be? Our brain is already primed and ready to go and with our brain primed I want to stop and tell you a quick story. This is a story… The summary will be a picture that shows how all of this stuff works.
This is a story about two more business people. This is a guy named Ron Walton who is the son of Sam Walton and is the chairman of the world’s largest corporation, otherwise known as Wal Mart. This is Peter Seligman, who is the head of Conservation International, the world’s largest conservation organization. Now, by rights, these two guys should have nothing to do with each other. They should probably, according to our business beliefs, probably hate each other. Because one wants to consume and sell and the other wants to conserve. Well, the two of them are very good friends. The reason for that is because they both like to track outdoors with their family. They like to spend a lot of time traveling outdoors.
One time when they were on a trip, not planned, they happened to meet. This was up at the Northwest Passage. Both of them were on different expeditions that were looking at the ice pack and they met. They started traveling together because they hit it off. Peter Seligman started taking Rob Walton’s family to places where you could see, you could literally see, the impact that humanity has had on the planet in terms of climate change.
So one of the places again, that they continued to visit is Northwest Passage where for the first time in recorded history, the ice has broken up. You can now sail through it without having to stop, which you couldn’t do before, or though go down to the Amazon rain forest, which we talk about, we talk about deforestation. But they would go and look at it and they will see it. Of course, the intent here from Peter Seligman’s side was to motivate the Walton family to understand that there is a connection between human consumption and our impact on the planet, and to see it. Well, it worked.
Because after some of these trips, Mr. Walton said to Mr. Seligman, “OK. I get it. I want the Walton foundation… We’re going to give you 40 million dollars as a start to do whatever you want with at Conservation International.” Mr. Seligman said, “I don’t want your $40 million. I mean that would be lovely. But I’d like something else from you. I would like you a commitment from you that you will at least think about instilling within your organization to the degree that you can some sort of understanding of what environmental sustainability might mean.”
“You’ve got the world’s biggest company. You’ve got the world’s most complex supply chain.”
“You’ve got the world’s best ability to be efficient in delivering products to market. What would happen if you started to make every little step, somewhat more sustainable?”
Mr. Walton said, “OK. I’ll try it. But the guy we have to convince is Lee Scot.” He’s the CEO of Wal Mart or at least he was up until six months ago and he left on a good note. Because he was generally considered to be a pretty successful CEO. Let’s face it, in the end of the day, Lee Scot doesn’t answer to the environment, he answers to his share holders. If whatever he decides to do isn’t profitable for them, it’s not going to fly. So he’s the guy that we really need to convince.
So they did a test project, a pilot project, ‑where they said, “OK. What’s a product we can make that’s environmentally sustainable, that we can test to see if it’s profitable or not?” So they made organic cotton Yoga wear, believe it or not. That’s what Wal Mart decided to do. So they created a new line. They went out and they bought almost the entire organic cotton crop of Turkey which is the world’s largest provider of organic cotton. They made Yoga wear and sold out in three months at enormous profit. So they’re certain he’s convinced, from a business perspective‑‑being more environmentally aware actually could work.
Then in 2005, Katrina took place, wiping out New Orleans. As we all know, FEMA was not particularly agile on its feet in responding to that catastrophe, but Wal Mart was. Wal Mart was down there instantly, with hundreds and hundreds of truck loads of food and water that they were sending down and the way that, at least, Scot described it. I met him and he gave a talk. This is a couple of years ago. Now it was pretty inspiring. He said, “These are our people. If you look at the citizens of New Orleans, these are the life blood of an organization like Wal Mart. This is where we’re from, this is who we are. We are not going to let our own people go down.” So on their own, purely philanthropically, I believe that, they just sent materials, truck loads, food, water, building materials, you name it. They were the first responders to help people out in New Orleans until the federal government kind of got its act together.
What Lee Scot was saying at this talk is, “Why can’t we be the company that we were during Katrina? Why can’t we do that every day? And I don’t mean giving stuff away, but I mean being that thoughtful about what we do.” So he decided to go ahead and make Wal Mart become a flagship company for environmental sustainability. Depending on whether you like Wal Mart, you believe it. Or whether you don’t like Wal Mart, you think it’s all a bunch of crap. It’s a bunch of green washing.
Now let’s face it, there are two kinds of people on earth. There are people who love Wal Mart and there are people who hate Wal Mart. And they will never mix, and they will never change their mind. So what Wal Mart said is, “Look we’ve got to come up with a simple message to explain what environmental sustainability does actually mean from our perspective.”
They put out a tender to a bunch of PR companies and design organizations and what not. I had a friend at Wal Mart who handed me a copy of the RFP. The problem is all Wal Mart has is a tremendous amount of data. It’s not emotionally sexy, it doesn’t make any difference to you, we can’t understand it. So I said, “Why don’t we…,” in my response, “just create a set of pictures that make the data visual, so that we can understand it at a root level.” And among other things, “Why don’t we make a simple little model of the Wal Mart supply chain that actually shows how it all works and it’s too big to understand on a global basis. So let’s make an essentially like a little scale model. Like I would have made as a kid that just is a sliver of the entire supply chain. Then we can look at it in more detail and understand what would work.”
This was my proposal to them and I won the contract based on these little drawings. So in the end that’s what we did. We built this beautiful little 3D model that this is all of the aspects that make Wal Mart operate, from stores to transport to production to disposal and all of that. Then that model, you could break up into these different layers. Each layer represents a different aspect of environmental sustainability from carbon output to electricity consumption.
Then from those models you would be able to build visuals that people could understand. So instead of a big table of data that no one understands, this is how much CO2 is put up by Wal Mart around the world on a comparative basis. So you can suddenly see how much CO2 is put up in the United States versus how much is put out in Japan versus the UK.
But the point I wanted to make is that those were not the pictures. I love those. Those were a little bit like the X‑Plane pictures. In fact I was always inspired by X‑Plane, these beautiful little 3D models. But I realized in hindsight, those were not the pictures that mattered. The pictures that mattered were the pictures that we were drawing in the executive meetings. These extraordinarily simple little sketches, that said, “Wait a minute, if we break it up into these layers and each layer we can come up with some way of visually showing if this is how much we consume today, this is what we’ll consume tomorrow.”
This is the picture that actually made the difference because this is the picture that the decision maker’s really got. Which brings us to our second unwritten rule of visual problem solving. The more human your picture, the more human your response. Which really means we like to look at things that match the way our mind sees. I want to do a little test of this for a moment. Then I’ll run quickly through helping you figure out the rest of our napkin picture. Hopefully leaving enough time for a little bit of Q&A.
There are going to be four pictures that I’m going to show you. They’re all very simple. They’ll look very much like this one. I’d like you to look at this picture for a couple of moments and just see what you see. Now I’m going to move to the next picture. This is A. I now want to show you picture B. This is B. I’m going to move back and forth. This is not a test. I want people to really see what’s going on here. I want you to notice that some things have changed. I’m going back to A now. This is picture A and this is picture B.
Does everybody see that somethings have changed? How much time do you think might have passed from picture A to picture B? Is it a day? A few seconds? A few minutes, OK. Now I’m going to show you picture C.
Dan Roam: Something obviously has happened. Now I’m going to show you picture D. Something else has happened. How much time do you think might have passed between picture C and picture D? A few hours, OK. That’s all that we’re going to do for that little test. But as were through what just happened? A pretty amazing thing just happened. Look at how simple these pictures are. They’re nothing but some stick figures drawn in black. There’s no color. There’s no shading. There’s no drop shadows. There’s no 3D effects. There’s no color at all.
And yet, as extraordinarily simple as these pictures are, we just saw every major fundamental aspect of our visual processing system kick into action. It takes such an extraordinarily limited amount of information for us to activate every major processing center that we have. What I mean by that is as we learn more about the neurobiology of vision, we’re beginning to understand that vision is an extraordinarily complicated process, “Duh!” The more we understand about it, the more complicated it becomes. But there are a couple of things that we do know. Vision works as both a serial and a parallel process. That is to say the amount of information that is out there for us to process every second is overwhelming to our brain. We don’t have the capacity to process everything that’s out there.
So what our system has evolved is to be able to split up the entire work among a number of different work streams. So one of those work streams is called the “What pathway,” and this is the real name. It’s called the “What pathway.” Another work stream is called the “Where pathway.” Another one is called the “How pathway.”
One could make the argument, and I’ve had this argument with a handful of neurobiologists. They’ve all agreed that it’s fundamentally correct. One could make the argument that essentially the way we see the world is we break it up into six different work streams.
We process most of them independently and simultaneously. Then we stitch them back together in order to create the entire picture of the world that we see in front of us. There’s a reason why I’m telling you all of this. It’s something that I like to call the “6 by 6” rule. It tells us this, “To create a picture of any problem that we can conceive of we do not need to know how to draw hundreds of different kinds of pictures. We need to draw how to draw six.”
The rationale is if our vision system already breaks the world up into six different streams for processing, all we need to do to convey an idea is make one picture that taps into each one of those streams, shows that stream the information it needs to create the whole picture. Now we’re going to go through this on a napkin for the rest of our time.
I’m going to do this fast. We’ve got about 10 minutes more to go. Then I’ll leave time for some Q&A. So we’re out of here by 10:15. But back to our napkin now. What I’d like you to do is slice our problem up into six slices. Just like a pizza. We don’t have to think of our problem as one big scary thing that we can’t understand. We’re going to slice it up into six different slices exactly the way our visual system does and break it down into pieces.
The first one is the who and the what. This is what’s triggered by the what pathway. What the what pathway does is that it recognizes the things in front of us. It says, “Oh, that’s Dave. Oh, that’s a light. That’s a door.” Think about this as the nouns of our world. That’s what the what pathway does. It identifies the things that we see in front of us. The picture that we would draw to represent that slice is just a simple portrait. What I mean by that is, “Here’s a man. Here’s a woman. Here’s a car. Here’s a box.” Just a little simple portrait. The most basic thing that we need. Just enough information for us to understand what is the thing that we’re describing. That’s pathway number one.
Here’s an example. This is something called the “Wong Baker faces scale.” This is used in emergency rooms. This is developed by two doctors for use in places where there may be a language problem or people may not be verbal at all. The doctors trying to diagnose what’s wrong with this patient and they can’t communicate verbally. Well the doctor points to a part of the patient’s body and the patient points to… This very simple portrait conveys a tremendous amount of information. The simpler it is the more information it actually conveys. The more essential it is the faster we tap in to what is the difference between this and this.
Here’s a very simple little portrait. This represents a visual description of the second most important financial decision that most Americans will make. You’re wondering what I’m talking about? Well, am I going to buy an automatic or a manual? Very simple little picture. How’s that for manual?
Slice number two, the how much pathway. Doesn’t know what things are, it’s triggered by what things are because then it starts to count them, loves to count. We’re really good at counting. We’re really good at counting up to five. For most all of us, if we were to take and make little piles of things, pennies or marbles on the floor, and then look at them, the maximum number that most of us could look at and know how many it was without counting would be five. Our mind is really happy with that. Because once we get to six we have to stop and count. Our mind, remember it’s trying to process everything as fast as it can. It doesn’t like to stop and have to count it wants to just see it.
So what the how much pathway is doing is it’s trying to make guesstimates of quantity. That’s what it is doing. So the picture that we would draw if we want to reflect the how much statement, or how much is a chart, a visual representation of quantity. That’s what charts are for. So here’s a chart. This happens to show the price of tea in China.
Here is a little pie chart and yes I think pie charts are just fine. Dr. Tough Tea will have to argue that later. This is a little chart that shows the typical break down in a meeting, in a typical meeting, of how people go about solving problems visually.
The where pathway now. Slice number three. This one’s really cool. It has no idea of what anything is. But it knows where everything is. Completely distinct pathway separated by 30 million years of evolution from the part of our brain that identifies what an object is, the part that knows where it is. Has no idea what stuff is but it knows what its proximity to me is and it’s proximity to something else.
The image of what you can imagine this would look like, has anybody ever seen like a sonar scan picture or a radar scan? You don’t know what the objects are but you see shades of gray that indicate how far away they are? That’s kind of what our where pathway sees. Doesn’t know, doesn’t have a clue that that might be a person, that that might be Dave, but it knows there’s something 3‑ feet away from me, slightly below me, not moving towards me, “I probably don’t need to run.” That’s what our where pathway does.
The picture that we would draw to reflect a where problem is a simple little map. So say here’s my home, here’s the river and here’s where the treasure is buried. A simple little map that shows where things are located. A map can show where all the pieces fit, a map, of course, can show, we know this, talking, preaching to the choir here, a map can show where all the pieces fit within an organization or within a site map, we make maps all the time.
But think about what a map is doing, it’s doing one thing, it’s showing us where things are. A Venn Diagram? It’s just a map. It’s not showing a geographical region but it’s showing a conceptual region. Where do these ideas overlap?
When? Things now get really interesting, slice number four. We’ve come all the way around here. Now, the when pathway is pretty complicated because it’s keying off everything that’s come before. Dave, I’m going to single you out one more time. Would you do me a favor? Would you just stand up? This will be real easy for you. I’d like everybody to look at Dave Gray from “Explain,” thank you Dave, you’re doing a great job so far.
Dan Roam: Now, I’ve just identified what you are. Forgive me for calling you a “what” but that’s what my brain says. That’s “Dave,” we know that. Now our where pathway for all of us is also kicking in saying, “OK, Dave is that proximity away from me.” Now Dave, would you walk over here and then walk back and then sit down, that’s all you need to do. Now watch Dave as he does this. You may go back. Thank you. And sit down. Dave, thank you.
Dan Roam: Now what we just saw is a demonstration of our when pathway. Here’s what it does, it turns out that the number one way we recognize the passage of time is by what we see. Our what pathway said, “Dave,” our where pathway said, “There,” then a minute later, or at some other point it said, “Wait a minute, the what has moved, the where is different.” Now, is that a different what? Is that a different Dave? I don’t think so. The only thing my brain can deduce that happened is time must have passed, welcome to the fourth dimension. We see the fourth dimension all the time. We see time all the time. In fact, it turns out, if we go into a sensory deprivation tank, you know, we’d lose sense of everything right away, but if we just close our eyes and we’re in a quiet room, all of our senses impact us, but vision is the number one, one of the first things to go is our sense of time. If we don’t see what’s changing their where, we lose our sense of when. Does that make sense?
Pretty cool. So the picture that we would draw when we face a when problem is one we’re all familiar with, we just draw a timeline. When is one thing happening in relation to one other thing happening. Which one comes first? And which one comes after? I’m kind of a space geek, so here’s a nice picture of a where picture, you know, JFK says in 1961, “We’re going to the moon.” We’re going there. Pretty good vision statement. I mean everybody can see it every night, we know exactly where we’re going. If only health care were that simple. You know, we’re going there.
That’s wonderful as a where picture unless you happen to work for NASA, in which case the question becomes, “We’re going to do it before the end of the decade. We’re going to do it by 1969.” You say, “When? When are we going to do that?” Well, all of a sudden the project managers better start kicking in with their Microsoft project and their timelines and their Gantt charts. Because now we’re going to say, “When does everything need to happen in order for us to reach that particular deadline?”
So now we need a when picture. We need a timeline. And yes, there are lots of different kinds of timelines but they all show the same thing, when do things happen? We can look at that one for awhile. If you want to confuse people, put your timeline in a circle, I’ve all seen us do it, I’ve done it enough times myself. A sure way to get a client to just fade off is to create a circular timeline and then we review it, and then we iterate and then we review it and then we iterate.
We’re getting near to the end, because we’re just going to sneak in under our timeline, so this is perfect. Slice number four. Our brain is really working at this point. It’s combining everything we’ve seen up to this point to try to deduce cause and effect based on what we have seen. The whats in their various how muchs, in their wheres, are moving over when to allow us to start to deduce cause and effect.
What I mean by that is, what we’ll notice is, if dog sees birds, dog will run to birds. And if dog intersects with baby carriage, parents will panic. And that’s the picture that we saw in C. I don’t know how many of you noticed it, but I noticed this because I’ve done this before. Every single time I go for picture A to B to C, which is the one where the dog hits the baby carriage, everybody goes, “Oh.” We’re doing that from stick figures. We have an emotional response from stick figures. I mean holy smoke, what did we just trigger? Well, we triggered our cause and effect. We triggered the how.
Now a how picture, if the problem we want to describe is how does something work? We could summarize it by saying, “What will happen if I push this button?” What will I trigger if I push this button? What series of events? How will this flow? How will the system respond if I push this button?
The picture that we would draw, of course, is a little flow chart. This happens, which triggers this, which triggers this or if not, then that. A visual representation of how something works. And you’ll notice, with each one of these we’re getting increasingly complex in what we’re deducing and what we’re saying about the world.
The first three, all took place simultaneously. What it is, how much of it and where it is, are all happening at the same time. Then they get put together with when, over time and then we start to build this bigger picture.
Here’s a very simple little flow chart of how the human brain works. Sensory information comes in, goes into our reptilian brain, it gets processed and we act. We take certain behaviors based on what our senses have taken in. We humans, with our very sophisticated, fancy neocortex up on top are able to do a whole lot more fancy and sophisticated analysis and be able to execute a whole lot more sophisticated behaviors. Unless we’re talking about health care, in which case it goes like that.
The last slice, the why, combines all of the previous and this is when our intellect really starts to kick in and say from everything I’ve just seen in those little pictures, what rule can I derive? Why is the world the way it is? The picture that I would draw, is a very simple one, well in this case, what I’ve been able to deduce from those, that A, B, C, D picture of those stick figures is, I guess dogs really love birds but birds don’t love dogs.
I’ve been able to come up with a very simple visual equation that describes why the way the world is the way it is. So the picture that we would draw, there’s actually two, I’ll give you the simple one because we’re out of time. Had we had more time, I’m going to break that thing. The simple one is we draw an equation. What I mean by that is it’s the same thing as drawing our little picture of dog loves bird, bird does not love dog. The equation I like to draw is this. Very simple picture, square plus triangle equals circle. A very simple little visual equation that summarizes everything that we’ve seen up to this point.
Now I’m sure some of you are saying, “Dan, by what possible stretch of the imagination does square plus triangle equal circle?” Well, all of us know that triangle means delta which means change, so a square plus change gives me a circle, so there, you’ve got it.
We are now done. We’ve gone all the way around our little problem pie. If I don’t destroy my iPhone I’ll see that we’ve got about seven more minutes to go. There are two routes we could go now. We’re going to do this by a show of hands. I have a five minute story I could share with you to summarize all of this, or we could call it a day right now and use the rest of our time for Q&A.
How many people want to hear one more story and I just keep going on? And how many people want to do some Q&A instead? I’m going to share one more picture and I will go through this fast because let’s test the model, all six, with one big question, “Why does visual thinking matter?” Some of you have seen this before, so you’re going to have to go along for the ride one more time.
We’re going to go through all six slices, starting at the beginning, “what” all the way through “why” to try to understand why visual problem solving is so good for us.
What is visual thinking? Well, it’s a biological neuro-chemical vision science process by which we make sense of the world around us through our visual system. That’s what visual thinking is. Fine, enough of that.
How much visual thinking do we have? You already know the answer, because I told you.
If this is our total capacity to process incoming sensory information, let’s fill it in for vision, and you know when to tell me when to stop.
We’re seeing a lot of stuff. Boy, are we seeing a lot. We see a lot, more than we hear, and that’s where we’ll stop. Our entire processing capacity, that’s how much goes to vision.
How much do we use in a meeting?
Dan Roam: Blah, blah, blah, blah. “No, you’re wrong, sir!” Where did visual thinking begin? Now we’re on slice number three.
How many people know where visual thinking began? It began in France. I do not lie. Visual thinking began in France, at least as far as we can prove.
“What is he talking about?” Everybody’s heard of the caves of Lascaux. How many people have heard of the caves of Chauvet? Not so many people.
The caves of Chauvet are really cool. So what we’re going to do is zoom in to south central France, not far from the caves of Lascaux. If we continued zooming in, we would eventually come to this beautiful river that has cut out in the central mountainous region of southern France. If we continue down this river, we would come to this incredibly beautiful natural bridge which has been there for tens of thousands of years.
For hundred and hundreds of years, people have used this as a place for recreation. In fact, you can see there’s little kayaks down here. I don’t know if they’re scale, but this is pretty good. This is a kayak down here about that big. There’s another one here. People swim here at the beach. For thousands of years, people have been going to this place, and they think it’s beautiful. And it is.
But in 1993, very recently for the first time, a spelunker named Monsieur Chauvet discovered the entrance to a cave hidden on the back side of that arch. And he started with his team to explore that cave, and they were eventually able to map out in an incredibly complex series of caves that as they did their carbon dating, were far older than the Lascaux.
What they were able to find in there are a series of unbelievably beautiful pictures that people had been drawing.
And this is just one wall, this is one part of the wall. These four horses here, I don’t even want to draw on top of them, they’re so beautiful. These four horses were drawn over a period of about 800 years. People kept coming back to this cave for 800 years, drawing similar pictures… Those four horses…
To this day I defy anyone to draw a better picture of a burro than that.
Here’s something interesting. We’ve got a rhino. Who in southern France would have seen a rhino? Pretty wild. I mean, these bulls are incredible.
So when did this begin? How old are these pictures? I’ll give you a meaningless number, since we’re good at five.
32,000 years ago is when those pictures were drawn. And as I say, it appears from carbon dating that people continued to go back beginning 32,000 years ago for the next 800 years to the same cave and draw throughout all of those walls these unbelievably beautiful pictures.
Now, we know that human life actually began in Africa, and we have been able to find shards of things, but in terms of actual human markings that are clearly an intentional human marking, the oldest that we’ve been able to find are these at the cave of Chauvet 32,000 years ago. These are are the oldest representations we have of humans making markings.
Now I thought, 32,000 years… Again, it’s a meaningless number. I really wanted to understand how long ago that is, so I thought, let’s look at time in a different way. Let’s not think about it in terms of years. Let’s think about it in something that we humans understand. Let’s think about it in terms of generations.
So I made a little chart, which is my all time absolute favorite picture that I’ve ever drawn.
Let’s say each little character represents one generation. And let’s just say that on average, a generation from one mother to one child throughout all of human time has been roughly 25 years.
I mean, we can debate it may have been 15, 20, whatever. But let’s say roughly 25 years. So instead of talking about years, let’s talk about generations, because that’s something we can all imagine.
For myself back to my mother to grandmother to my great grandmother, and I wanted to map out how many generations have there been in 32,000 years. Very few. This takes us back to Columbus. 1492. That many generations. That many grandmas and grandpas. That’s it. All the way back to the time of Columbus.
I could draw it in one line. I could count it on two hands. Holy smoke. That’s not a long time ago at all.
Let’s go back a little bit further. Let’s go back 2,000 years to the beginning of the numbering system of years that we have now, to the time of Jesus Christ. That’s how many grandmas and grandpas there have been.
This was breathtaking to me. I always thought history was long. I thought 2,000 years was a long time. It ain’t nothing. That’s how many generations have been since the time of Christ.
Now let’s continue all the way back 5,000 years to the beginning of recorded history.
So here we’ve got Caesar here, five generations before Jesus Christ. Socrates here. We’ve got Muhammad up here. We’ve got the Buddha, the original Gautama Buddha right here. We’ve got Nefertiti, representing the height of the Egyptian empires here. We’ve got Abraham back here. We’ve got the beginning of recorded history, 3,000 B.C. That is it.
You know, it’s amazing to me. You’ll watch “Star Wars,” and they’ll talk about the Jedi Knights have ruled the universe for thousands of generations. No. There are no thousands of generations. That’s how many generations have existed since the beginning of recorded history. It’s 200. I can count that. That’s it.
I don’t know how you feel. I start crying when I look at this. History is so short. The only reason I was able to get that is because I drew it in a picture. I swear I’m going to start crying.
So then I thought, OK, but I want to go back 32,000 years. That takes us 5,000 years back. Let’s go back 32,000 years. How many grandmas and grandpas have their been and their babies since 32,000 years, since the first time that anybody we’ve found made a mark on a wall, drew a picture.
By the way, this is the beginning of, of course, as best as we can find, spoken written language. Takes us back.
So if I compressed everything in this picture into one line, that’s how long it would take us back.
Now let’s keep going. This takes us back 16,500 years back to the caves of Lascaux. Here’s when Lascaux was, and if we complete the whole picture, that’s how far we go back to Chauvet.
Every one of those little dots represents one grandma to one mother to one daughter, all the way through. I was just blown away. That’s it. We really got to get our health care figured out. We’ve got to take care of this planet. It’s not very long that we’ve had it.
Anyway, how did it begin? We’ve only got two more questions to go.
Well, evolutionarily speaking, we first had a reptilian brain stem that was able to process a little bit of fundamental visual information. That’s where much of our “where” pathway is. Crocodiles are really good at knowing where stuff is. They have no clue what anything is, but they know where it is.
Then we’ve got this limbic brain on top of that that allows us to have emotional responses to what it is that we were seeing, and make maybe more emotional decisions about how we would react.
And then we got this fancy old neocortex on top that allows us to do really sophisticated visual processing.
Our last question now. Why did visual thinking begin? So we wouldn’t get eaten.
And my last question for all of you is, why does visual thinking still exist?
Dan Roam: So we won’t be eaten.
Dan Roam:  Oh boy, we are right at the end of time. Thank you so much. I really appreciate you staying through the whole thing.

Sketchy Wireframes


When it comes to user interface documentation, wireframes have long been the tool of choice. However, using traditional diagramming tools like Visio, OmniGraffle, and InDesign, most wireframes today look the same as their ancestors did from a decade ago – assembled with rigid, computer-drawn boxes, lines and text. While these artifacts have served us well, they can also be slow to produce, burdened with unnecessary detail and give a false impression of “completion.”

To compensate for the drawbacks of traditional wireframes, many practitioners put aside the computer in favor of simple pencil sketches or whiteboard drawings. This speeds up the ideation process, but doesn’t always produce presentable or maintainable documentation.

There is a growing popularity toward something in the middle: Computer-based sketchy wireframes. These allow computer wireframes to look more like quick, hand-drawn sketches while retaining the reusability and polish that we expect from digital artifacts.

The same wireframe in sketchy and traditional representation.
The same wireframe in sketchy and traditional representation.

The Traditional Wireframe Problem

Throughout a project lifecycle, wireframes can be used for different purposes depending on the stage. In the early stage, wireframes act as a tool for exploration and concept development, when sweeping changes are expected and encouraged. As the project continues, parts of the wireframe begin to be “locked down” as functionality is reviewed and “signed-off.” During this process, wireframes can become a confusing hybrid of conceptual ideas and finalized functionality. By the end of the process, wireframes can turn into a highly detailed functional specifications document.

The problem here is that traditional computer wireframing tools, like Visio, OmniGraffle or InDesign, lay out drawings as hard-lined boxes, lines and fonts. As a result, wireframes look the same regardless of which stage of completion the wireframe is representing. Early-stage, conceptual wireframes look identical to late-stage, functional specifications. This differentiation becomes especially murky in the middle of the project, where conceptual and final elements are comingled on the same page.

Sketching to the rescue?

To compensate for the drawbacks of traditional wireframing, some designers ditch the computer in favor of hand sketching. An informal poll by Konigi.com (as of 8/24/09) showed 22% of respondents identifying sketching as their primary tool for wireframing. Hand-sketching of wireframes, proponents argue, allows for faster expression of ideas and freedom from artificial confines of diagramming software. Sketches don’t require the same level of detail, and can be produced faster than traditional computer-based wireframes, allowing for a more iterative design process.

Why not sketch…

If hand-sketching has so many advantages over computer-based tools, why don’t we all ditch our mouse pads for sketch pads? There are four major reasons:

  • Drawing ability – Wireframes are essentially presentation tools, and not everyone may feel that their drawing skills are “presentable.” In team environments, there can be a wide range of drawing skill levels… from the “can’t draw a straight line” people to the “can’t put down their sketchbook” people. This leaves a disparity in the quality of sketched deliverables produced by the team. Many organizations feel it’s best to standardize their deliverables by forcing everyone to use the same tool.

  • Perception – When people become accustomed to seeing fully fleshed-out wireframes, introducing sketchy may be a challenge. Some may see the architect as suddenly becoming “sloppy” or “lazy.” In these cases, it is critical to sell the benefits of sketchy wireframes to stakeholders and opinion leaders.

  • In situations where wireframes are intended to live past the initial concept stage and turn into functional specifications documents or user guides, hand-sketching is not the most appropriate method. Hand-drawn sketches give the wrong impression of flexibility at later stages of development when the interface has already been “locked down.”

  • Reusability – Hand drawing is great for getting ideas down quickly. However, when wireframe documentation is lengthier than a couple of pages or when the documents must be re-worked over a long period, sketching loses its speed advantage and becomes a burden. In an electronic medium, changes can be made across pages and documents very quickly.

  • Prototype flexibility –Many practitioners prefer to go directly from hand-drawn wireframes to interactive prototypes, bypassing the more traditional wireframe process. However, in many situations, wireframes are used to generate interactive prototypes for proof of concept and/or usability testing. Hand-sketched wireframes are excellent for paper prototyping, but the amount of work involved increases quickly if they need to be scanned into the computer and converted into interactive prototypes. For on-screen prototypes, it is much easier to start with wireframes that are already in an electronic format.

Enter the computer-generated sketch

To compensate for the problems of both traditional and hand-sketched wireframes, certain programs allow you to create the look of hand-sketching with no drawing ability required, while retaining all of the benefits of a digital tool:

  • The style gives the impression of a work-in-progress, yet still retains a “polished” feeling that aids in acceptance by the workplace
  • Components are easily reusable for longer documents
  • Wireframes can be re-purposed for interactive prototyping

Sketchy Wireframes in Action

I discovered the need for computer-based sketchy wireframes while working on the website redesign of a well-known print media brand. I found myself presenting wireframes to executives, who would critique them in the same manner that they would a print-spread: with a heavy focus on fonts, text placements and graphic treatments. Despite frequent disclaimers that the wireframes were for high-level discussion purposes only, each presentation would drift into fixations of irrelevant details. To accommodate them, I found myself spending countless hours polishing the wireframes to look beautiful, when I should have been spending time on concept development and user testing.

To make matters worse, as we removed features from the wireframes that were determined to be “out of scope,” we continued to receive requests to bring them back, right up until the end of the project. Clearly, the wireframes were not helping to convey the right message.

On the next project, I generated the conceptual-stage wireframes using sketchy Visio stencils created by Niklas Wolkert. I began all of my wireframe presentations with an explanation of why the wireframes looked like sketches: they were intended to be malleable, rough outlines. I also prepared the executives for the next phase by telling them that the sketchy look of the wireframes would be removed as decisions became “finalized.”

Comparison of the sketchy wireframe stencils by Niklas Wolkert (right) and traditional ones by Henrik Olsen (left) at guuui.com. Image credit: Henrik Olsen.
Caption: Comparison of the sketchy wireframe stencils by Niklas Wolkert (right) and traditional ones by Henrik Olsen (left) at guuui.com. Image credit: Henrik Olsen.

The improvement in the executives’ perception of the process was immediate. The boxes and lines of the wireframe no longer had to look perfect, and the hand-drawn fonts couldn’t possibly have been mistaken for an intentional design. The executives, feeling less compelled to fix the visuals, were free to talk at a high-level about architecture and strategy. As the project transitioned from concept to execution, I removed out-of-scope features and converted the style from sketchy to traditional. This virtually eliminated later-stage requests for functionality that had previously been removed.

The reaction to computer-based sketches

Having used computer-based sketchy wireframes on a number of projects, I’ve found many ways that they can decrease confusion with teams and stakeholders:

  • Clients and Executives – People in this group typically want to push projects forward as quickly as possible. Consequently, the more “finished” the wireframes look, the faster they will expect to see the finished product. You can do yourself a disservice by making your wireframes look more complete than they are. To quote Kathy Sierra, “How ‘done’ something looks should match how ‘done’ something is.”

  • Programmers – Programmers who see traditional wireframes too early in the process may misinterpret their functionality as “signed-off.” Don’t be shocked if you hear frantic questions like “Did we agree to this?” Programming requires meticulous attention to detail, so programmers read wireframes with an eagle eye. Consequently, they may expect a level of specification from wireframes that isn’t appropriate in the early stages.

  • Designers – Designers make their living with their visual creativity, and they resist anything that could constrain it. Consequently, in situations where designers must work with wireframes created by someone else, designers can perceive them as a creative straightjacket, or worse, as a threat. A sketchy representation can help reduce friction by removing unnecessary details and adding a certain amount of “fuzziness” to the wireframes, thereby giving designers more leeway in interpreting the look and feel of the interface.

  • Users – In my research, I’ve found that users who are asked to comment on traditional wireframes can be intimidated by an overly finished look and feel. This is mirrored by a general consensus in the usability industry that the “less done” a demo looks, the more comfortable users feel with giving feedback. Where traditional wireframes can elicit comments like “I don’t like the font on those words,” sketchy wireframes are more likely to elicit comments like “I don’t know what those words mean.”

Computer-Based Sketchy Tools

There are now a number of programs that are capable of generating computer-based sketchy wireframes. However, in working with them, I’ve found that many of them are missing what I have identified as four essential capabilities necessary to be considered a “complete” sketchy wireframing tool:

  1. Ability to Draw New Sketchy Shapes –
    These days, many components of user interfaces are standardized into stencils that can be dropped onto wireframes to build them out quickly. While this can be a real time saver, not all UI problems can be solved with prepackaged stencils. In fact, one could argue that the best use of wireframes is to illustrate new concepts that have not become standardized. Many tools use pre-built, sketchy-looking stencils to allow designers to create sketchy wireframes. However, at some point you will need to create new shapes that aren’t available in your set, and a true sketchy tool must enable you to create new ones in the same sketchy style.

  2. A sketchy tool should allow you to draw.  These were created in Visio using custom line styles. This tutorial tells you how.
    A sketchy tool should allow you to draw. These were created in Visio using custom line styles. This tutorial tells you how.

  3. Easy Conversion from Sketchy to Traditional Style –
    Sketchy wireframes are a great tool for encouraging creativity, exploration, and collaboration. However, at some point, your blue-sky, creative ideas fall away and you are left with what you are actually going to build. In environments where wireframes morph into spec documents and user guides, those rough lines and hand drawn fonts must be converted to a more finished, traditional style to avoid the impression that your technical documentation is still changeable.

    Does this mean you have to go back and re-draw all of your sketchy wireframes with straight lines? Not if you can avoid it. Fortunately, certain programs allow you to convert your existing sketchy lines and fonts to traditional style without having to recreate them.

    Some software automatically converts from sketchy to traditional lines.
    Some software automatically converts from sketchy to traditional lines.

  4. Realistic Lines –
    It’s always been difficult to approximate the look and feel of true hand-drawings using software tools, but some do it better than others. The quality of drawings generated by a computer-based sketchy tool could have an impact on whether the wireframes are perceived as “conceptual” or just plain “sloppy.” These are the 3 major components needed to completely represent hand-drawn styles in wireframes:

    • Wavy Lines – No human can match the rigidity of a computer’s lines. Adding waviness and movement to lines humanizes them.
    • Varying Line Weights – When drawing conceptual wireframes, there are often areas of the screen that have yet to be explored. One way to represent this is to fade out lines as they enter these areas.
    • Smudging and smearing – These effects help to reduce focus on unimportant areas of the wireframe.

    These lines, created in Fireworks with a graphite line texture, could hardly be mistaken for true hand-sketches.
    These lines, created in Fireworks with a graphite line texture, could hardly be mistaken for true hand-sketches.

    These lines, created in Illustrator, are much closer approximations of true sketching.
    These lines, created in Illustrator, are much closer approximations of true sketching.

  5. Prototype Flexibility – For those who prototype their products, speed and efficiency of workflow is a critical issue. In this case, the benefits of creating a sketchy look and feel will become irrelevant if doing so increases the time needed to create prototypes. Fortunately, some tools allow themselves to slip naturally into the process by generating interactive prototypes that maintain the sketchy look and feel.

  6. In interactive sketchy prototype created in Visio and imported into Axure.
    In interactive sketchy prototype created in Visio and imported into Axure.

Comparison of Computer Based Sketchy Tools

Software developers are starting to recognize the importance of computer-based sketchy wireframes, and there is a growing assortment of tools to create them. This is a quick breakdown of how each of the major tools matches our criteria for a complete computer-based sketchy tool:


Draw Shapes

Easy Conversion

Realistic Lines

Prototype Flexibility











Expression Blend 3





Fore UI



































None = No Support
Partial = Partial Support
Full = Full Support
  1. Assumes prototype flexibility using a 3rd party program called Napkee
  2. Assumes use of custom line styles, as demonstrated in this tutorial


As the industry evolves, there is a growing trend toward hand-drawn styles, as evidenced by an expanding amount of literature and workshops on the subject. This is a positive step in the evolution of our field. Sketchy wireframes allow practitioners to guide creativity and problem solving in the early stages of projects, rather than getting lost in a sea of documentation. Hopefully, this trend will continue as software manufacturers focus on enhancing their tools for creating computer-based sketchy wireframes.

The Content Conundrum

rocks, intro image

As web designers and information architects, we often dismiss deep consideration of content when we design interactive experiences. By content I’m not only referring to the various forms of text (e.g., headers, body copy, error messages) but also imagery, graphics, and videos or audio that make up the full interactive experience.

Sure, we have a sense of what content is available, and we’ve likely considered it to some extent when creating flows, wireframes, and prototypes. But the design artifacts that we create represent only part of the overall user experience that we’re designing. The content that sits inside of our design framework is often the final arbiter of success, yet we sometimes diminish its importance and separate ourselves from it. The more we separate our design activities from content development, the greater the risk of design failure.

Recognizing The Problem — The Content Gap

There’s often an unsettling discrepancy between the stakeholder approved wireframes and visual comps and the actual product in production. What you see in those environments is sometimes a far cry from those polished wireframes and those shiny, pixel-perfect visualizations that were filled with placeholder content (such as lorem ipsum text, dummy copy, and image blocks). What you’re seeing in production environments now holds the real content. The imagery doesn’t support the interactions, is meaningless, useless, or worse, contradictory to the design intent. The copy, headers, and labels are unclear, too long, too short, or simply irrelevant.

What happened?

More than likely, that content was discussed, created, and iterated outside or separate from the core design review process and ultimately plugged into a content management system (or pasted into the code by a developer) much later in the development process.

The example illustrated in Figure 1 shows two examples of web content. The image on the left represents a screen shot of the approved design that was delivered to the production team. The image on the right is a screen shot of that same page taken from a functional test environment after the real content was included. As you can see, the experience breaks down considerably with the amount, type, and format of the real content. The information is more difficult to scan, the primary call-to-action has been pushed well below the fold, and the choices that users need to make are less clear.

These two screens show what the content gap looks like. On the left is the mockup next to what it looked like in production.

While this example highlights only a small portion of the overall web site, the problem manifested itself throughout the bulk of pages that made up this interactive experience. So what might be perceived as a small problem becomes a much bigger problem when considered across the entire interactive experience.

Exploring the Causes

These content gaps are both procedural and cultural within organizations. By procedural, I’m referring to the tangible processes used by an organization to design and develop a web product. Often times, these ‘processes’ are influenced by the organization’s values and overall culture. There are four common reasons why content gaps occur.

Flawed Processes

There’s undoubtedly a wide spectrum of web design and development ‘processes’ in use today. Most often, however, organizations use one that aligns more closely with either a traditional waterfall process or alternatively, an Agile one. In theory, both models have mechanisms built-in to eliminate and minimize surprises (including content gaps) but in reality, both tend to exacerbate the problem but do so in different ways. Rigid waterfall processes fail because they tend to segment activities and related roles. Designers often design totally separate from content ideation and development. Agile processes fail because they’re typically developer centric and move at speeds and iterations more akin to code production than to experience design and content development. The site is often being coded before the design or content are ever completed.

Content The Design(er) is King

We’re at a point now where usability is table stakes, and persuasion and message is necessary to differentiate products.

The value of most design projects is typically placed in the upfront design and strategy work. It’s here that the ‘big ideas’ are generated and explored. During this initial phase, are the right people involved in the design process alongside of us, exploring solutions? I’d argue that we rarely involve our content partners, even though we’re essentially creating a framework for communication and messaging. It’s here that content specialists thrive. We’d benefit from including those who specialize in communication, writing, persuasion, and instruction more directly. We might argue that as designers that we have those skills, but then we shouldn’t rely so heavily on placeholder content in our designs.

There’s a lot we can learn from traditional advertising here. In advertising, copywriters often drive the creative process. Their skills with communications and persuasive messaging are often unparalleled within an agency. We’re at a point now where usability is table stakes, and persuasion and message is necessary to differentiate products. In fact, some leading companies are beginning to recognize this change and develop tools and/or POVs on this topic (See Eric Schaffer’s article, “Beyond Usability; designing web sites for persuasion, emotion, and trust” and Forrester Research’s report, “Use Persuasive Content to Improve the Customer Experience”).

Design artifacts rarely include “real” content

I understand the need for lorem ipsum text and placeholder imagery. I am an information architect, after all. When working on an overarching framework for a web experience and creating a flexible design system, it makes sense to start with concepts and relationships, and to establish the right models and structures first. However, the more we start illustrating these concepts at the page level, the more we must concern ourselves with content and the overall message we want to create. By relying too heavily on placeholder text and graphics, we begin to communicate a level of reality that is problematic. It’s at this point in the process that the actual content should be considered and where our design deliverables should communicate these details.

Obviously, exploration of visual styles, hierarchy, and the overall visual language is crucial at this stage. That said, effective content to support those elements is absolutely essential for design success. The content works in conjunction with our visual language and style to help people move through and understand the information space they’re in. The more the design and content can be explored, iterated, and finalized together during this phase, the fewer problems we’ll encounter when the site goes live. Dr. Browyn Jones said it best in her 2007 article, titled Better Writing Through Design:

“Ideally, you should work with a writer from day one to design the voice of the copy in conjunction with the visual language of the site. And getting a writer involved early can help you solve lots of other problems—from content strategy issues to information architecture snags. Remember that writers are creatives too, and they are, in many cases, the keepers of the content your design ultimately serves.”

Lack of value assigned to content

When taken as a whole, the general perception is that content teams are production teams and therefore non-creative. Taken as a whole, content teams are typically highly focused on production and publishing issues. An unfortunate side effect is that these individuals are brought in much too late in the process, immediately playing catch-up, and trying to understand the bigger design decisions that were made. In many cases, the only information that they have to go on is a lot of ‘lorem ipsum’ or other placeholder content.

What Designers Can Do to Address these Problems

As a design community the first thing we can do is recognize the problem and want to fix it. I’d suggest that we look at it selfishly at first, realizing that if content fails, our designs fail. Period. There are a number of tactical things we can do with every project to mitigate the risks.

1. Rethink the need for specific content

Do you really need that introductory text? What about those thumbnail images? What will those elements really accomplish for your design? Are they necessary? Many of the content components we include don’t contribute to design goals or the users ability to perform a task. Simply remove those from the design entirely. The more concrete we are about what is and isn’t open for interpretation (or worse, misrepresentation) the fewer surprises we’ll see in those functional environments.

2. Explore Information Graphics & Visualizations

Take a step back from your designs and see what information can be communicated more effectively using visualizations and/or informational graphics. Let the user’s ‘scanning’ behavior work to your advantage. What can be communicated better with simple imagery than with text? How can that general concept be applied to your overall design paradigm? This critical extra step will improve and streamline the user experience. If you’re not the best person to create these assets, bounce your ideas off of the visual designers and production artists. Reviewing your own work this way will dramatically improve your design. As a bonus, the more perspectives you hear during this process, the better informed you’ll be to solve the design problem.

3. Write (some) content

If you can’t get a copywriter or content expert working with you from day one, spend some time writing draft content or sketching actual imagery and place it into your design artifacts. The goal isn’t to be perfect, but to communicate to stakeholders and partners the intent behind a particular content element or component. Bring the design to life and create actual content, headlines, text, instructions, headers, and imagery. Force feedback on those elements at the same time This will force you to think through the necessity of the content, the importance of the message, and force the same thought from your stakeholders. This means using lorem ipsum sparingly, particularly when designing critical web pages or elements that significantly impact the experience. Don’t rely on someone else to do it without first thinking hard about it yourself.

4. (Really) Collaborate with your content partners

The collaboration that we demand from developers should parallel those we have with our content partners: copywriters, strategists, production artists.

The collaboration that we demand from developers should parallel those we have with our content partners: copywriters, strategists, production artists. Often times, the content teams or copywriters are working with brand, marketing, or product teams on the creation of ‘final’ content. They understand what those teams need to accomplish and what they’re trying to communicate. Rather than have that process happen without your oversight, get involved early and often with these people and describe your vision, solicit their input, and ask for help clarifying your message and assumptions. This back and forth (like the one we expect to have with development) needs to happen with our content partners as well. Become friends with them. Remember, their skills at persuasion, messaging, branding, and simply overall writing prowess can only improve our designs.

5. Package real content with the visual mock-ups

Whether it’s visual comps, or a prototype, it’s important that whomever is responsible for creating and approving the content is actually involved with the visual designer and prototyper as they ‘package’ that deliverable. It’s impossible to fully evaluate the effectiveness of a web experience without having the content represented and under the same microscope as the design. Brand, product, and even training teams all have their own perspectives about what the content must communicate and are contributing to its development and we don’t want our design to fall apart once this ’collaborative’ writing process starts. Assign accountability to content upfront and place content professionals under the same creative deadlines you’re marching to.

There are a variety of tools and software emerging that can help you work with content. For example, Adobe InCopy hooks into Adobe InDesign. It’s just a matter of time before we start seeing integration points with Photoshop and other standard web design software and tools. But even without formal tools, the important step is that ‘real’ content is represented and tells a more complete story about the design you want to put out there.


It’s up to you to assess whether these content gaps are a problem in your design environment or not. Admittedly, this problem is more applicable to larger web sites and online businesses given variety of stakeholders (read: opinions). That doesn’t mean that these concepts don’t apply to the social web, or smaller marketing or micro web-sites. They do. It’s just that how critical this issue is depends on the size and scope of the website or application you’re designing.

This problem is common in many organizations (small and large). As a design community, we hold the power to 1.) change how we think about content, 2.) bring other roles into our processes, and 3.) change how we communicate with stakeholders and partners. Collaboration is what we strived for when developers shut us out, now it’s our turn to open up and let our content partners in and build even better experiences for our customers.

Photos for interaction

When developing user interfaces, designers increasingly use custom graphical elements. As the web browser becomes basic technology for software interfaces, more and more elements derived from graphic and web design replace the traditional desktop approaches to the concrete design of human-computer interfaces.

In the near future, this development will become even more relevant. The barrier between web pages and desktop software is beginning to disappear, and modern rich client user interface technologies such as Silverlight/WPF, Air, or Java FX enables designers to take the control over the whole user experience of a software product. Style guides for operating systems like MacOS or Windows become less important because software products are available on multiple platforms, incorporating the same custom design independently from OS-specific style guides. Software companies and other parties involved begin to use the power of a distinct visual design to express both their brand identity and custom interactive design solutions to the users.

While this implies a new freedom for designers working in the field of interactive software products, it strengthens the importance of visual design for the design of user interfaces. Designers working on concrete graphic solutions for a specific interface are breaking away from established standards defined by a software vendor. It is now the responsibility of those user interface designers to choose graphical elements wisely to make a product’s interaction principles visible and usable.

Elements of interactive visual design

Following the roots of visual design in print and online communication, the design of a visually compelling and functional application must take into account different requirements, even though it takes the same methods to realize its goals: A dynamic visualization of the interactive product in form of text, images, and colors. In contrast to pure one-way communication design striving to create identity and media, the main goal of such a design process for interactive products is much closer to product or industrial design — namely the creation of a product that serves the user in a optimal way. It requires a strong collaboration with the disciplines of interaction design, software development, and product management.

The role of photography in software user interfaces

Photography has both challenges and opportunities as graphical element in user interface design. I chose photography as an example for a classic communication design instrument,  but the ideas are also applicable to typography, illustration, motion design, graphics, and the like. One important aspect of these thoughts is the required collaboration between the different design disciplines involved in the creation of a user experience, and how to optimize team performance for most valuable ideas and outcomes.

Case 1: Photography as content

In software applications, photography in most cases is used as content element, since photos express situations of human life very well and thus are well suited to capture and represent a certain message. The images have a semantic meaning, communicating information to the viewer and user of the respective web or software application.

Examples for this type of application can be found not only in private photo collection software such as iPhoto but also in enterprise content management solutions for web sites and product catalogues, or the web shop itself. To the user, the photo is not an element of decoration or design, but it is the actual content or a part of it.

On the visual design side, the challenge is to present this content in a way that makes it visible and reveals context and meaning. Photographic content tends to come to the fore due to its strong graphical impact, so other elements should be designed to support that effect and not to compete with it for the viewer’s attention.

The challenge of well-representing imagery content elements in a user interface is often to provide adequate metadata-driven tools to allow enhancing images with meaning; take tagging people at Facebook as an example, which turns photos into something findable. Finding a a meaningful visual representation of photographic content and this data is a common challenge to visual design and information architecture.

Case 2: Photography as design element

While the use of photography as a design element in user interfaces is rather new, there is a long tradition of using photography as a design element in advertising-related online media. This treatment as design element follows the rules of brand communication and takes photography as integral element of the web site design.

But contrary to its usage as a content element, the image is used in web design as a medium to communicate a message to the user in order to create a certain context for the real content. Some sites, such as financial institutions or software suppliers, are working with stock-like photography showing photos of people or buildings, while other businesses can combine site content and corporate communication in one image, like on fashion sites.

Benetton Web Site

Benetton uses the photo on their home page to convey both a product and a brand message to the visitors. The photo is in the focus, but is receipt more like a visual expression of emotion than as actual site content. The web design uses the photo like an advertisement would do: It is part of the site’s visual design and has been chosen by the designers. The product, derived from the site’s content, is turned into the medium to make an impression to the visitor.

Photography in interactive media is often a trigger for engagement and interaction. Interaction designers working on the product’s interaction flows can thus provide visual designers with key information to select and apply visual elements, in order to start the conversation, and keep it alive.

3. Photography in software UI Design

Unlike other digital products, the visible part of software usually makes no significant use of photography by means of communication design. Today’s desktop software interfaces consist of text, rectangle areas, and icons, along with with a lot of transparency or 3D effects. If not a necessary content element, photos are only used in splash screens of desktop applications.

In web interfaces, static images in header bars are quite common, resulting from the "hybrid" characteristics of those applications between a software product and a web site. In most cases, the photo serves as decorative element with no semantic meaning and is thus reduced to a very small amount of space of the screen; it is not important for the product’s original purpose. This is done in order to provide as much space as possible for the informational content that is useful to the user.

SAP Enterprise Portal

The image above shows SAP’s enterprise portal product in a standard visual design. The small photo showing a bridge in the header bar is part of the UI design, while the images at the bottom are content elements related to the text messages.

Like in web design, the image is used here as an element of design but loses all its visual power due to its jammed position in a design that puts all emphasis on the representation of information. The "mise en scène" of the interface suffers from the poor integration of the photographic element, totally separated from all information. Its meaning in the application context is reduced to a vague bridge metaphor referring to the function of a portal.

The best of both worlds: towards a new quality

With every release, software providers make a step towards a custom graphical representation and improve the visual design quality of their products. To take a real advantage of photography as a medium, there is a need to treat it differently than it is done today in the software industry.

At same time, a lot of effort is being made to make applications more "shiny and glossy", to better imitate real world structures on the screen. Sometimes, like in current reporting tools for business intelligence, this additional glitter reduces the visual perception of information instead of enhancing it.

The following examples and recommendations are not always easy to follow, because a meaningful integration of this medium in a UI design that centers around representation of information and providing a tool for efficient usage is a difficult task. Nonetheless, visual elements such as photography have the power to reveal a message instantly and powerfully to the user to complete and to establish a visual identity. Designers should use these possibilities to trigger the user’s attention to support a holistic interaction design and not to distract her by decorative elements and visual clutter.

Examples for photography in interactive applications

Designklicks Designklicks

This example screenshot shows Designklicks (now seen.by), a German website that collects and tags user-generated imagery. Just like Flickr and other photo-centric web sites, the images are in the focus of the design and are visually strictly separated from other design elements like icons, logos, buttons, and links. For a visual representation of the complex information architecture, it allows the user to sort and present the content in different ways, from a simple grid to a navigable 3D space.

Space by the Barbarian Group for Getty
Space by the Barbarian Group for Getty Space by the Barbarian Group for Getty

These screens are taken from an art project for gettyImages, done by the barbarian group. It uses widescreen photos to build a three-dimensional flow of cascaded rooms, connected to each other by graphical signage elements appearing in the images.

Société Générale Customer Portal

The bank Société Générale used a photo as main art on their web site, emphasizing the fact that they address everyone with their services. The main navigation appearing on the start page is embedded into the photo, but at the same time arranged in a clearly separated layer above the image.

VDW Fine Art Website

Photography is the main design element of Van De Weghe Fine Art, an art gallery in New York. All graphic design elements remain very reduced while the full screen photo is used to create a virtual room for information and interaction.

Take the blinkers off, and think about experiences as a whole

People in the roles of information architects or interaction designers tend to concentrate on their part of the job and leave subsequent visual decisions to the graphic or visual designers, which is of course always a good way to start. Nevertheless, all designers (including the two disciplines mentioned before) should be able to actively think about and contribute to the concrete, sensual appearance of the final product, since this is what design is all about.

So why posting this on a site dedicated to the "design behind the design"? Because interaction designers and information architects have become strong conceptual thinkers, driving an experience in terms of concept as well as it’s soul.  Visual design should enhance and implement this vision, which is in fact in most cases the contratry of "making things pretty."

Recommendations for photography in next-generation interfaces

  • Integrate the images into the interaction design. This can be achieved by making areas responsive to user behaviour, enhancing its function from a visual element to an instrument of interaction. Due to its realistic and nonverbal nature, photography can be equally or more powerful than icons, buttons or other classic interface elements.
  • Work with screen space. Place images in a way that they have a real impact on the overall appearance instead of putting them into small banner-like screen areas.
  • Photography invokes an emotional reaction and has the capability to create a certain ambiance more easily than other media. Use pictures that make the user feel comfortable and adequate to the application context.
  • Clarity, structure, movement, separation, union – photos can convey messages instantly to the viewer, by means of blur, motion, composition, and of course motive. Work with these as design elements.
  • If used as content element, think about alternatives to simply placing photography on a grid. There are a lot of possibilities to make images "tangible" to the user. Think of multiple layers, movable objects, or 3D approaches.
  • Keep the subject of the application and the nature of the content in mind while designing. Choose photos that convey a real meaning and make sense in the application context. Avoid standard (stock) images or those with only decorative function. Prefer custom-made images tailored to your intentions and requirements.
  • Combine and integrate all elements to create a holistic interface design where all visual elements work together and make the interface.

See also:

Interactive Identity: Bridging Corporate Identity and Enterprise IT
Visible Narratives: Understanding Visual Organization by Luke Wroblewski
10 ways by gettyImages

Coming soon:
Part II – Typography in User Interface Design

Cues, The Golden Retriever

In every waking moment, our brains are processing the stimuli in our environment and responding, consciously and unconsciously, to what is going on around us. This may mean something simple like stopping automatically at a crosswalk based on the color of the traffic signal. Or it may mean something more deliberate, like deciding to turn left after orienting yourself by reading a street sign.

Both consciously and unconsciously, we also make decisions while interacting in an onscreen environment. We move automatically during routine tasks and through familiar interfaces. But what do we do when the interaction onscreen requires a very deliberate and thoughtful interaction—how do we determine the correct response to the stimulus? We need cues to help us draw from our experience and carry out an acceptable response. Cues are like little cognitive helper elves who prompt us toward a suitable interaction, reminding us of what goes where, when, and how. Cues can be singular reminders, like a string tied around your finger, or they can be contextual reminders, like remembering that you also need carrots when you are shopping for potatoes and onions in a supermarket.

When we’re arranging content and designing interactions for the onscreen environment, providing cues for users helps them interact more effectively and productively. Increased customer satisfaction, job performance, e-commerce, safety, and cognitive efficacy rely on deliberate interaction with the technology and thus easily benefit from the smart use of cues.

I’d like to frame a discussion of cues by touching on a mixture of topics including memory, a few theories from cognitive psychology, and multimedia research. It may get a little dry, but stick with me. The integration of these three areas not only affects how information is encoded and retrieved, it influences how and when cues might best be used.

Remembering Memory

Let’s refresh your memory on the topic of memory—stuff you probably already know. This is the foundation of how and why cuing is effective.

First, there’s the idea of encoding and retrieval (or recall). Encoding is converting information into a form usable in memory. And we tend to encode only as much information as we need to know. This is a safety valve for over-stimulation of the senses as well as a way of filtering out what we don’t need for later retrieval. Retrieval is bringing to mind for specific use a piece of information already encoded and stored in memory.

Memory is generally labeled long-term memory and short-term memory (or working memory, in cognitive psychology parlance). Our working memory holds a small amount of information for about 20 seconds for the purpose of manipulation—deciding what to do with sensory input from one’s environment or with an item of information recently retrieved from long-term memory. The familiar rule is that humans have the capacity to hold seven items (plus or minus two) in working memory. In contrast, long-term memory is considered limitless and information is stored there indefinitely. Information from working memory has the potential to become stored in long-term memory.

The Integration of Multimedia and Memory

Ingredient 1
By its nature, interaction in an onscreen environment can be considered multimedia. At the very least, visual elements (images, application windows, the cursor, etc.) are combined with verbal elements (semiotics, language, aural narration, etc). These are called modalities and they are processed differently in the human mind using different neurological channels: this process is called dual coding and it’s when images and words create separate representations for themselves in the brain[3]. This is important because cues unique to a given modality can be used to better retrieve information originally processed with that modality. For example, color coding the shapes of the states on a map as red or blue helps us store for later recall the political leanings of a given state—the shape of the state triggers our remembering the color.

In a “real world” environment, stimuli from the visual and verbal modalities (among others) guide the way we interact with that environment—influencing our working memory and long-term memory. These stimuli can get to be a lot of work for the little grey cells and it helps when the two modalities share the load—the cognitive load—of processing information. The same is true for the onscreen environment as well.

Ingredient 2
Cognitive load[1] describes the tasks imposed on working memory by information or stimuli from the environment, in our case the onscreen environment. How much information can be retained in working memory—how much can we encode before our working memory is full and new information has no place to go? And if it escapes working memory, chances are slim that the information will make it into long-term memory.

So what happens when a modality is limited by cognitive load? In short, the working memory gets full fast. Encoding, cuing, and retrieval are affected. The interaction onscreen impacts the encoding necessary for later recall, particularly when different modalities are vying for attention. A limited working memory makes it difficult to absorb multiple modes of information simultaneously[2].

But if the modalities compliment one another, more information can be processed when they work in tandem than would be possible using a single modality. A large body of research exploring the use of multimedia and computers yields a couple of useful general guidelines:

  1. When presenting information onscreen, text and visuals are not as effective as seeing visuals and hearing narration.
  2. If text is the chosen way to convey verbal information, it should be in close proximity to the visual element it is related to (like labels on a map).

A big no-no is narration which is redundant to the text visible onscreen. This is a bad practice because the brain works too hard mediating continuity between the two cognitive channels; the reader is distracted from the content because of the mechanics of constant comparison of text and voice. It actually detracts from successful encoding. Naturally, if the encoding is faulty any use of cues used for later recall of that information is compromised.


Okay, now let’s look at cuing a bit more closely. The idea of cues and cuing is a theory more formally known as encoding specificity by its pioneer Endel Tulving. Memories are retrieved from long-term memory by means of retrieval cues; a large number of memories are stored in the brain and are not currently active, but a key word or visual element might instantly call up a specific memory from storage. In addition, the most effective retrieval cues are those stimuli stored concurrently with the memory of the experience itself[5]. (This implies that most cues are external to the individual and we’ll accept this characteristic for the sake of this discussion.) Citing a popular example, the words “amusement park” might not serve to retrieve your memory of a trip to Disneyland because during your visit you didn’t specifically think of it as an amusement park. You simply thought of it as “Disneyland.” So the word “Disneyland” is the cue that retrieves the appropriate gleeful memory from all the other memories warehoused in your brain.

It’s important to note two chief categories of cues—discrete or contextual. In other words, it may be that a user is being asked to respond directly to an onscreen prompt, or she may be interacting with the technology in a certain way because of the elements present in her onscreen setting. Most of us are probably familiar with the Visio interface and can recognize it instantly. When we’re working in it, we automatically use its features without thinking about the act of using its features. When concentrating on a project, we grab an item from a stencil, move it onto the workspace, size it, label it, etc. We don’t use Visio to try to re-sample a photograph’s resolution or check a hospital patient’s vital signs—we “remember” that Visio is capable of certain functionality because of the cues surrounding us in the Visio environment. This is an example of contextual cuing.

Reminiscing about Disneyland is one thing, but some tasks and interactions require more cognitive load to complete and the cues should be employed appropriately. For example, onscreen controls for a large piece of machinery, one which is dangerous when used incorrectly, require an operator’s focused attention. Cues provided in such an onscreen environment need to be deliberate and explicit. For example, a large red stop sign icon appears onscreen to warn the operator that he has forgotten a safety procedure.

External cues such as work environment, physical position, or teaming around a table may also affect interaction onscreen. If we anticipate the physical environment in our designs, we can control the cues onscreen to accommodate the users in that environment. In our large machinery example, perhaps onscreen cues are related to observing its movement or the sounds it makes. Or if crucial interaction needs to take place in a busy or noisy environment, like punching your numbers into an ATM, discrete and/or contextual cues which accommodate that external environment appear onscreen.

Cues also need to be salient and germane—they need to have meaning and relevance appropriate for the situation, task, or environment. They need to fit into the schema[4] of the interaction. Schema can also be regarded as a semantic network[6], where information is held in networks of interlinking concepts; it’s linked to related things that have meaning to us. Tim Berners-Lee says “a piece of information is really defined only by what it’s related to, and how it’s related.” So naturally the cue that recalls such a piece of information will need to be related to it, too.

The use of meaningful cues is tied to how memory functions. Memory is bolstered when its meaning is more firmly established by linking it to related things. This is because it’s less work for the short-term memory to plug new information into an existing schema: if the new information is encoded relative to its context, the cue that retrieves the information should also be related to that context. A rather glib example might be memorizing several new varieties of wine using colored grape icons to represent different flavors. When recalling those wines, cues in the form of smiling farm animals would do no good in helping you select a wine that goes well with spaghetti.

Humans are fallible, though, and sometimes even the best thought-out cues may not be effective. For example, if the context or subject matter is unfamiliar, cues which rely on it will not be helpful. In fact, sometimes the context is so unfamiliar that cues are not recognized for what they are; if information is not recognized as relevant or meaningful, it will be disregarded. People are better at recalling information that fits into their own existing schemas. There’s a semantic network unique to each of us. Fortunately, Tulving (1974) assures us, “even when retrieval of a target event in the presence of the most ‘obvious’ cue fails, there may be other cues that will provide access to the stored information” (p. 75). One preventative measure against designing ineffective cues is a thorough usability study. Or we may provide cues that address more than one modality. Each situation is as unique as its context, so it’s not possible to make recommendations here; the issue of ineffective cues can arise and it is important for us to acknowledge the risk (and any potential fallout!).

One general prescription for the symptom of ineffective cues is to provide the cue immediately before the desired recall, either immediately preceding interaction or positioned near the recall artifact (e.g., password field or bank account number field). In other words, cues need to prime the information they are designed to help retrieve. Another strategic method of cuing is pattern completion—the ability to recall complete memory from partial cues. The simple act of grouping items may be a sufficient retrieval cue. It may even help establish a context or schema for the user, thus increasing the subsequent effectiveness of your cuing system.

Related form and function in the onscreen environment can also act as cues. Context dependent menus are a perfect example of this, like the grouping of drawing tools in Word. The four-sided icon represents the function for drawing boxes. The same icon indicates very different functions in other Word tool palettes (or in other applications)—the user doesn’t have to remember exactly what each of the four-sided icons does: their context is the cue for reminding the user of their function. An easy text-based example might be placing an arts festival event with an ambiguous title in the same column onscreen that lists similar events.

Jason Withrow’s B&A article Cognitive Psychology & IA: From Theory to Practice explores this idea in greater detail.

Another cuing strategy is one mentioned above in passing, the use of mixed-modality cues. This strategy draws on the advantages of splitting the cognitive load between two encoding systems.2 , 3 Cues for one modality can be presented in another modality if the original encoding matches that set-up (i.e., an image-text mix is the cue for recall of the same image-text mix). A perfect example is discussed in Ross Howard’s article on what he terms ambient signifiers. Audio is piped over the PA of a large transportation network. Each train station in a large city has a unique audio melody associated with it. As Howard points out, not only is the destination station’s audio a cue to get off the train, the commuters memorize the melody for the station prior to their destination, priming them for their actual destination. This is an interesting example because it also takes into account the environment in which the stimulus-response cue is introduced. With preoccupied or bored daily commuters crowding onto a train stopping at homogenous-looking stations, what cues might help them successfully get home? The computer game Myst used a similar technique by using sound cues to help the intrepid player solve puzzles.

But what happens when elements of the onscreen environment are really similar (or ubiquitous)? Our brains err toward efficiency: events and elements that are similar are generally encoded in terms of their common features rather than their distinctive characteristics. This is great for helping us fold new information into existing schemas and contexts. But it interferes when the IAs and designers need the user to distinguish between the similar events or elements. This situation is described in the interference theory, which states that the greater the similarity between two events, the greater the risk of interference. So it becomes a balancing act: maintain continuity across the interactive environment while at the same time establish a distinction between elements you want the user to retain. Something as simple as color-coding might be a means of distinguishing information onscreen. Position may be another. Think of a process being taught or conveyed on a training website, a process whose stages have big bold numbers respectively highlighted across the top or side of an interface. Not only does this help with chunking (breaking the information into digestible bits to avoid an unreasonable cognitive load), but when enacting the process later, like on a factory floor, it’s easier to visualize the numbers and remember the correct procedure.

Two notable phenomenon are related to using position onscreen as a cuing strategy. Primacy effect is the increased tendency to recall information committed to memory first and recency effect allows that items memorized last are also easier to recall. This may influence how the information is organized on a web page and how the cues might be used. (By the way, recency items fade sooner than do primacy items). One example might be a corporate intranet website with crucial information buried in a feature article. If you place that information in a single sentence synopsis at the top of the home page, you may plant the important points more permanently than forcing the readers to sift through the longer article. Any cues related to that information will likely be more effective.

Philosophy from 10,000 Feet Up

There’s a Chinese proverb that says “the palest ink is better than the sharpest memory.” I include this proverb because the palest ink serves as metaphor for how even the most understated of cues employed in an onscreen environment can be an effective recall or feedback strategy. And this strategy nurtures the perception that the computing technology is in concord with what is natural for the human user.

It’s been encouraging to watch the evolution of computing technology move away from forcing the human user to adapt to its form, function, architecture, and singularity. The continued momentum toward a more human-centered, ubiquitous interaction environment is encouraging. Humans are very dependent on the dynamics of stimulus-response cues in their natural environment; it’s important to establish a similar dynamic as we take part in designing interaction within their technological environment. The conscientious use of cues is not a panacea, of course. Because the use of cues onscreen mirrors the common stimulus-response paradigm which humans are used to in the natural world, however, it’s one of the more effective tools we can use when we design interactions.


fn1. Sweller, J., & Chandler, P. (1994). “Why some material is difficult to learn.” Cognition and Instruction 12(3): 185-233.

fn2. Mayer, R. E., & Moreno, R. (2003). “Nine Ways to Reduce Cognitive Load in Multimedia Learning.” Educational Psychologist 38(1): 43-52.

fn3. Paivio, A. (1986). “Dual coding theory.” Mental representations; a dual coding approach. New York, Oxford University Press: 53-83.

fn4. Schank, R. C., & Abelson, R. P. (1977). Scripts, plans, goals and understanding; An inquiry into human knowledge structures. Hillsdale, NJ, Lawrence Erlbaum Associates.

fn5. Tulving, E. (1974). “Cue-dependent forgetting.” American Scientist 62(1): 74-82.

fn6. Collins, A. M., & Quillian, M. R. (2004). “The structure of semantic memory.” In Douglass Mook (ed.) Classic experiments in psychology. Westport, Conn.: Greenwood Press: 209-216.