1) Forest for the Trees – In the case of large development teams working on a single project, they’re often divided into multiple individual scrum teams, each with their own focus. As they move forward, their marching orders are to do what’s right to complete their user stories in the most timely fashion, but there’s no default mechanism for the teams to communicate with each other, so when it comes time to review the products at the end of the sprint, you have a product with lots of individual features that meet the letter of their requirements, but isn’t necessarily (or even likely) a coherent design across the product. Great products aren’t defined by their individual features as much as by their user experience that span multiple features.

2) Shrink to Fit Quality – This is similar to mine #4, but I think it’s an important variant. I agree with your insight that Agile should be a methodology that allows the team to flex scope rather time or quality, but in practice I’ve found more often than not that the time constraint often forces the team to flex quality because the ‘right’ way to address a requirement is to build something that’s ultimately too big to fit into a sprint. If there’s another cheaper way to achieve a similar result and does fit in a sprint, it necessarily becomes the recommended plan of record. The quality of the individual solution might turn out to be just fine, but the bigger problem is that it wasn’t the right long-term solution in the first place. This also fits back into the first item, because I’ve found that teams would rather build a lot of one-off specialized solutions to their issues that become a sustaining nightmare, rather than cooperate to build a single ‘right’ solution that may be bigger than any of the one-off solutions, but ultimately would be for the benefit of all the teams.

Lastly, I wanted to point to one of the other challenges I’ve seen. I immediately came to the same conclusion that so many others have that the UX team needs to be working a sprint ahead of the rest of the team so we can ‘feed the machine’. The problem though is that we can’t work in a vacuum. We often need input from the product owners or technical staff, but given that they’re usually knee-deep in their own sprint, it’s difficult to get level of participation and focus necessary to design the right thing for the next sprint. The best solution I’ve seen for this to date is to have the product owner only loosely participate in the development for the sprint, but instead be working on user stories in the backlog and participate in planning for the next sprint.

Again, thanks for the great article. At times it felt like you were channeling items straight from my brain.

First some bits I agree on:

1) there’s often a focus on shipping new features at the expense of tidying existing ones
2) the push to ship unfinished features is short-sighted; frightening users away with a promise that things will improve is a precursor to ‘brand damage’
3) any development that doesn’t include time for user testing AND accessibility testing and allow time for findings to be worked on and integrated is broken
4) untrammeled product / project managers making ill considered decisions are a massive problem
5) dogmatic scrum ‘masters’ do exist

On the first 3 I can only say that pressure for visible movement will always be with us. Being seen to get one step ahead of competitors (external and often internal) is one price we pay for competition but competition is innate to capitalism. The only hope is an enlightened management acknowledging complexity and giving people enough time to work through problems…

Number 4 I’ll get back to.

And number 5 is just an extension of a general truism; all development methodologies quickly sink into dogma. Unfortunately organisations are suckers for process, workflow and box ticking. (And as many people have asked, “have you ever met anyone who FAILED scrum master training?”) Anyway all process is a creativity sink. This is equally true of UCD which, applied dogmatically, is one of the least user centric approaches I can think of. Putting real code in front of real users (with assorted accessibility requirements) one of the best. So long as management give you time to act on the findings.

Afraid the rest of this comment has more of the usual ‘someone on the internet is wrong’ type smell about it. Apologies but…

…your opening minefield comment of “one has to ask whether [agile] was devised to treat a symptom of the larger cause: the business doesn’t know what it wants” is true but understandably true. Things change. Business priorities change, businesses merge and divest, the competition changes, *user expectations change*. Failing to acknowledge this and ploughing on with ‘big design up front’ doesn’t address that problem and doesn’t work. It also doesn’t address the fact that in a complex system it’s often impossible to judge the feel of a website until real data hits real code; design that works on paper too often breaks in reality.

(I’ve worked in the past on a music website that followed the full UCD methodology: research, personas, experience models, sitemaps, wireframes, photoshop comps… In the time it took to work through all that both last.fm and itunes shipped. Our site was 5 years out of date before it even went live.)

For fear of typing a longer comment than the original post I’m only gonna deal with your first mine (an unclear role for design) but I suspect most of the rest drops out of that plus the usual concerns about management allowing time.

You say “some [...] developers may have design skills. But that’s not a particularly common scenario”. This is either a very restrictive definition of design or just wrong. The question is: where does design happen?

Every time a developer creates a database or creates a model or creates a URL scheme or marks up a document that’s design. Writing code is design. Many (the majority?) of creative people I’ve worked with have been from the developer side of the wire.

The Google example you use is interesting in this context. Google was definitely ‘designed’ but no user experience professional or even the majority of software engineers I know could have designed it. The Google breakthrough was not the realisation that “people just want to find what they’re looking for, not learn how to drive a search engine first”. The breakthrough was making that possible. And making that possible was about software engineers and mathematicians taking academic models of citation, making the analogy to link density and *designing* a solution that used eigenvalues and n-dimensional spaces and serious maths to crack the nut. You could have spent months ‘concepting’ and taken the conclusion that “people just want to find what they’re looking for” to Yahoo! but the concept would be useless without the design and the design was tricky to say the least.

Back in the day flat websites were ‘designed’ by designers who handed over their wireframes and specs and photoshop comps to developers with the request ‘make it so’. Designers were creatives; developers just trades people. Having come across the UCD +/vs Agile arguments a few times there seems to be a temptation to wrestle back control and return to these glory days. But with the complexity of modern web applications it’s just not possible. No one person knows enough to be given overall control of design direction. This includes product/project managers, which is my easy answer to point 4 above.

You go on to say that ‘in good software development, a conceptual interaction model that has been thought through beforehand, outlines how the user navigates the system, performs tasks and uses tools in generic terms’. For a website of any complexity interaction design (together with quality copy and visual design and quality a/v) is the small part of the visible 20% of the application. It’s not an unimportant part but the other 80% of the application also needs to be “designed”.

In a later comment you say that “I personally don’t care what happens in code (though perhaps that’s just ignorant of me and I should) what matters to me is that the interface is consistent from one section of an application to the next”. For an interaction designer this is fine but again I think it’s short sighted to assume that design stops at what the user can see / feel. Taking Twitter as an example: it’s a beautifully designed service but the design insight was all about low friction communication and open APIs. Many Twitter users interact through clients and never go near the Twitter website. The Twitter interface is consistent but more important is consistency in the application / client interface all of which required design.

So decisions made further down the stack (what gets modelled, data flow, data licencing (particularly if you’re dealing with user data), url design, content negotiation / device detection, caching, document design, feed design) are not just examples of design but fundamental to a wider definition of user experience (in terms of seo/findability, website as platform, pointability, perceived performance, accessibility etc). If you don’t spend time designing that stuff then obsessing on a “consistent interface” is just painting pigs with lipstick.

To quote Steve Jobs (and why not?) “Design isn’t about how something looks; it’s not something you put onto the outside of an already-built product. It’s how you build the product, from the inside out.” In web terms it’s about design decisions all the way up the stack and working well with the design decisions of the web (statelessness, HTTP, URIs, HTML, CSS). If you’re not interested in code and not interested in technical design decisions of the web (which have moral and ethical implications) then I’d suggest you’re not a web designer and might be happier working in a different medium…

In conclusion I’d say the role of design in agile is clear. Everything is design; everything is development. Things work best when different people with different skills work together to solve problems. Attempting to rebuild the walls between design and development (even by staggering sprints) is a mistake. As Craig Webster says “A [user] story is a token for a conversation. Unfortunately very few teams actually have the conversation.” It’s up to designers and developers to have those conversations and up to management types to give them the time to do so.

***

One final point in answer to @doug’s comment: “what we do NOT do is completely rearchitect the UX each iteration – UX is just too fragile and expensive to change”. Yes UX is fragile and expensive to change but it’s actually a *lot* cheaper to change than any other aspect of a web application. Web apps are a layer cake of database, models, controllers, views and css. The further down the stack you make changes the more impact on the layers above. Changing mark up + css is much, much cheaper than remodelling your database eg. Not that constantly changing your UX is a good thing but the pain point is usability/consistency not designer/developer overhead.

You mention, “Agile does a good job of flexing to the winds of change. But one has to ask whether it was devised to treat a symptom of the larger cause: the business doesn’t know what it wants.”

Your observation is the root problem that our company works to solve for development teams. Even waterfall, when combined with rich prototyping and easy collaboration, enables stakeholders to interact with a simulation early in the process. The result is an unambiguous understanding that allows the client and the development teams to iteratively flesh out the project prior to writing code. Waterfall teams that integrate prototyping become more agile because they elicit and define requirements through iterative visualizations rather than comprehensive documentation. This process would be a hybrid of the Agile philosophy and waterfall method you mention.

Prototypes also allow Agile teams to “plan the big picture” while accelerating their development process. Creating an interactive visualization prior to each sprint ensures that everyone is on the same page and sprinting in the right direction.

I’d really like to hear your thoughts on how you think interactive prototyping fits into Agile UCD.

Thank you,
Andrea
@ProtoShare

In a place such as the BBC, where you have ample talent, creativity, a core value of quality, far less drive to get things done quickly and an editorial heritage that doesn’t lend itself to software development, Agile is a perfect solution. Why? Because these qualities demand you get the product right and they’re more important than the imperatives which drive other commercial companies into cutting corners for perceived savings. This environment somewhat coccoons you from the experience so many others have. These people, particularly those who put design and build in the same sprint can fail to produce anything good and have a miserable time in the process. How do you save money paying a team of people to build something that doesn’t work for the user? Fact: Using paper and pencils is far cheaper than writing code. Developers can use pencils too! Fact: You can learn a great deal from user testing paper and pencil regardless of the richness in the interface. There’s a lot of value in a prototype, you agree. But you seem fixated on that prototype being high fidelity, working code. I don’t buy that is the only or most cost effective way to learn. There is a time for testing working code and that’s once you’re confident you have the broader brush strokes of overall proposition and information structure right for the user. All applications have this. Not just flat websites.

The other great topic for discussion is: What is design and who does it? Naturally, every team member’s work has an impact on the user’s experience. Only some work at the part the user touches. Some of those have a job title that suggests their primary function is to worry about that before all else: User Experience Designers. For the purposes of this article, that’s who i had in mind when I wrote “an unclear role for design”. It’s interesting quoting Steve Jobs to back the point that design starts on the inside. But that isn’t actually how Apple works. The first thing Jobs looks at is a photoshop comp or a sketch that defines the user’s experience. Not a working prototype to be “skinned” later (which is his point). When Jonathan Ives designs a new peice of hardware, he sketches it (and the outside first not the inside). Sure, good designers appreciate how things have to work. But the point is, it has to work for the end user and there are some disciplines which are more tuned into that world than others. No database architects I’ve known are focused on or skilled at designing a GUI that is easy or fun to use. I’m sure they exist, but I haven’t run into this special breed of unicorn. But that’s not what you’re getting at with the Twitter example.

This opens up a different point. Who are users? Are the patrons of an open API users? Yes, they are and perhaps our database architect is a better designer for this “persona”. Could database architects benefit from an API user persona? I’d like to think that makers of  third party products have needs and goals too. Interestingly, these API  consumers sometimes need to make GUIs at the end of the day because raw data is kind of opaque to their customers: “Joe Public”. So I don’t think caring about how that gets done puts me out to pasture just yet. But thanks for the inference.

Firstly my “five years out of date” comment was misleading. I didn’t mean the project took 5 years. From (pained) memory it took about 12-15 months. The point I was trying to make was that during those months iTunes took off and Last.fm happened. The market changed and user expectations changed. It was 5 years out of date because it felt like the market moved on 4 years in those months. Which is often why businesses “don’t know what they want”; or rather why businesses change their minds and why agile is better suited to cope with this.

The other major problem with that project was despite all the personas and wireframes and sitemaps and photoshop comps by the time we’d written the mark up and added the css the data we had couldn’t be queried in a way that allowed us to build the product we’d planned. Much of the project went in the bin and what we ended up shipping was a very small subset of what had been intended. Starting by designing the data model and working up just lessens the pain.

I’m not blaming UCD for this and not saying that some design shouldn’t happen up front. I am saying that designing from the visual layer down is much less efficient and scaleable than designing from the domain model up.

I should also say that this experience isn’t just confined to my current employer. I’ve seen plenty of projects fail in plenty of different organisations because they’ve spent too much time designing the skin and not enough time designing the skeleton.

We (designers, developers, IAs) spend a lot of time with paper and pencil (or more often whiteboard, pen and post-its). But when it comes to user testing web design there’s no substitute for working code on top of real data. You can theorise interface details til the cows come home but in my experience trying to second guess how the application will ‘feel’ when the data hits the code is just misleading. Things that have worked in my head and on paper just felt wrong when seen in reality.

I wouldn’t say I’m fixated on on the prototype being high fidelity; just that writing prototype code AND real code is inefficient so why not make the prototype the real thing? As (possibly more) important is the use of real data. Interfaces need to evolve as the data volume increases. Taking my current project as an example, views that used to work when the system had a low volume of data now need to be redesigned as more and more data hits the back end. Attempting to get the design right without being able to experience the experience is too often futile.

Even in the simplest terms different browsers have different capabilities and different configurations and different people set them up differently. Again, the sooner you can see real users with real setups interacting with your application the sooner you can see what works and what doesn’t and make changes. Does the fancy javascript/ajax interface degrade gracefully? Is it accessible to those with javascript turned off? Are the pages accessible to users and search engines?

So onto design and who does it. I’m slightly confused by your statements: “Naturally, every team member’s work has an impact on the user’s experience. Only some work at the part the user touches.” I suspect it’s more accurate to say only some work at the part the user consciously touches. Taking my earlier example of HTTP caching. It’s a small detail but set up incorrectly the impact on UX (in terms of perceived response times) can be immense. It’s even worse on mobile where lack of caching means more return trips to the server and bigger data bills come the end of the month – which has to be a user experience issue?

Possibly the fact that “some [people] have a job title that suggests their primary function is to worry about [ux] before all else” is the problem here? Maybe we just need to go back to visual designer, interaction designer and information architect and embrace the fact that it takes more than those to make a user experience?

Taking a wider example from the project I’m currently working on. We have a single data model but with different business logic and ux on top we have 2, 3, 4 different ‘products’. The important point is that the domain model which underpins all of this is built taking user’s mental models into account. And my point is user experience starts with what you choose to model. I can’t think of a better link than Tom Coates’ Age of Point at Things [1] which talks about data modelling as the starting point for user experience (comparing ‘episodes’ on the BBC ( a concept that exists to users but which is not in any BBC business systems) to the lack of book works on Amazon books and performances of songs on iTunes). Taking a user centric approach to domain modelling and building upwards and out from the data model enables freedom of movement in the layers above which allows the site to evolve over time without changing the underlying model.

I’ve seen lots of projects built in the opposite direction; taking the ‘ux’ layer and working down the stack. That works fine for a first iteration but because the underlying model is designed to build a specific interface it gets trickier and trickier to warp the data model as the product evolves. Eventually the cost of changing the data model to cope with interface updates gets prohibitive and the whole stack needs to be rebuilt.

You go on to say “some disciplines [..] are more tuned into that world than others. No database architects I’ve known are focused on or skilled at designing a GUI that is easy or fun to use.” The point I’m making is not that dba’s are gonna build beautiful guis. That breed of unicorn may or may not exist; like you I’ve never really met any. The point is they do design too and that design has a ***fundamental*** impact on user experience. If they model the wrong things no amount of shiny css is gonna make that better. And correcting any mistakes they make is far more expensive than fixing the gui. Because, again, the further down the stack (css > markup > controller > model > database) you make changes, the greater the impact on the layers above.

I can’t really comment on the Apple example since I honestly have no experience of how they work. If it is from photoshop comps I fear that just depresses me…

On the final point about API design, I honestly believe that no-one benefits from personas. They’re an abstraction of real people; the world has quite enough real people without resorting to characature. And no persona is gonna give you the insights you need about modelling works not products or episodes not versions. Again a good api is reliant on a good data model but again the api (as user) experience relies on good http design, good document design etc. Much like experience for humans it’s a good mix of skills that makes an API work; no one person knows enough to make everything tick.

[1] http://www.plasticbag.org/archives/2005/04/the_age_of_pointatthings/

Starting at the UX layer, in my opinion, is not always starting at the interface layer. It’s about understanding what the right user experience is, regardless of what channel they touch and the medium they use. I think we agree on that too.

We will have to agree to disagree about personas and paper prototypes. I believe everyone benefits from personas because they stop us talking about ‘The User’ as some amorphous conglomeration of our own perspectives and forces us to adopt different lenses to look at the same problem. I also agree that our assumption of what will work can change slightly when we put an idea into working code (usually not fundamentally). But I can’t get past the fact that you learn a great deal from testing on paper and thus can save yourself a great deal of heartache, even if you add to that learning with working code.

Thanks for the comments, It’s great to have some good Socratic dialogue around these thorny issues!

But whether this is the best way to address the business not knowing what it wants is debatable. Yes it is a very good tool, but there are other ways to guide a business toward a great user experience that involve really understanding the problem they’re trying to solve (a.k.a. “the opportunity”). This involves a number of different types of research to uncover various opportunities i.e. design research with methods such as Contextual Inquiry to understand user behaviour; or comparative research that looks at analogous experiences in different industries and markets; or technological research that identifies trends in technology that may lead to some interesting ideas. This step narrows down the strategic options to the right playing field and provides some great fertile soil in which to plant idea; in so doing it maximises any investment in a prototype.

I am now working for my final project for my master degree in Strategic Product Design in TU Delft in the Netherlands. I thought I am being overly ambitious to bringing UCD into agile development process (as many people told me).

In my opition, currently the UCD approaches are not appealing for business investment as its really being implementing in early stage of development (like R&D), and no one gonna be sure if in the next step the designer or marketers will really take advantages of the result. Despite all challenges as you describe in your article, I believe UCD and agile development somehow can bring a lot of benefit for companies and help bring new innovative product to the market faster. And at the end UCD should become one of strategical platform in business environment, not only in design environment.

Please keep write inspiring articles! it helps more people than you ever expected.

Thank you
@lilisukirman

(also @ liliskirman)) Have you heard of Menlo Innovations (http://beta.menloinnovations.com/)?

They are an Agile (XP) and UX (“High-tech Anthropology (R)”) custom software shop in Ann Arbor, MI that has been successfully wrangling with all of these issues since *2001* (do you know of anybody else that was trying to do this that long ago?). I worked with them for a couple of years before bouncing across the pond (Germany) and found it a deeply satisfying way to make software. I’m very hopeful that this model of development will catch on much more widely. Articles such as this one which attempts to clearly map the terrain are a big step in that direction. Thanks!

In my experience, it does tend to be developers who are particularly drawn to Agile — and it has all to do with the battle lines between them and business sponsors, but slowly, as the practice matures, it’s being recognized that simply taking a willing user hostage for the duration of a project may not be the best way to cover the UX role .

@ Michael: I’ve got to say that while I not-infrequently wanted to yell at my computer screen while reading your posts, I do very much appreciate your taking the time to spell out your position so thoroughly. “Design” is an over-burdened word (here in Germany, for example, it nearly exclusively means “Graphic Design” and given how much half-English/half-German one tends to speak in the context of software dev, you can imagine the confusion when a word that is not entirely clear in English, means yet something else in the native tongue). I think we can actually all agree that the intellectual and creative process we native English speakers tend to mean when we say “design” takes place at every level of a software development team (especially if they’re doing OO). To the extent that there’s anything innovative going on (and every new project demands some innovation), somebody has to make sure it fits with surrounding patterns and will function across all the imagined contexts for which it is intended – that is, somebody has to design it. That happens at many different levels — from the size, type, placement and labels on the UI controls to the deepest level data repository. If anywhere along this chain, the models used don’t map to those in the mind of the targeted end-user, the thing just doesn’t work, period. Yes, the deeper layers have a grandeur all their own that is appreciated by far too few (though competence on this level is certainly well-compensated — so it’s some other sort of “appreciation” we’re talking about…), but how exactly do you propose that the designer of that deepest layer should map his design to the model in the head of its eventual end-users? I’m sure all of us have had experiences with databases that were wicked fast and theoretically sound, but that spit out nonsense and/or refused to take our input as we normally conceived it. And, unless we’re talking about a team of one, it’s not only important that the db designer know that end-user mental model, but that her understanding of it is shared with everyone else on the team, including the sponsor. To that end, somebody is going to have to go out and meet the putative end-users “in the field” and get a deep enough sense of their goals and desires to represent that back to the team. I don’t know what tools you would propose for this process, but in my experience that is best accomplished by describing the persona for whom the system is to be designed and following that up with the design of the initial contact points between the user and the system (the UI). From there, those whose design and other skills are more in the area of modeling abstract, logical spaces and articulating them in some version of machine code can pick up the torch and carry it forward. If your plan is to make software tailored to the end-user, then you’ve got to have a model of said user in mind. Absent an articulated persona, that role will be filled by any of a variety of agents, and likely not the same one for any two members of the project team (to say nothing of the sponsor – who knows what exactly they have in mind!). Such situations often do not end well. It doesn’t have to be that way. (Of course, if you are a dev team of one-three people and you are all similar and designing for people just like you and you don’t develop any idiosyncrasies during the project based on your “inside” perspective that won’t be shared by your intended users, then, sure, it would probably work out – without any more than the usual software dev project pain.)

My sense – yes, we are all designers. I don’t’ mean that trivially. It’s important to remember. Further, for a project of any size, one brain (actually I prefer two, and paired) should be modeling and facilitating the selection of a targeted end-user based on first-hand field observation and interactions, while another brain (or, again, a set of brains, working in pairs) is modeling the mechanisms that can empower that end-user to achieve the goals that are motivating her interaction with the machine, in the first place. And, of course, this all has to happen in a project context given its contours by clearly articulated business goals (teasing out clear statements of goals and posting them where everybody can see them is something UX folk are quite good at ).

Hmm, do I have to put an “End Rant” tag, here, now?