Integrating UX into the Product Backlog

Written by: Jon Innes

Integrating UX into the Product BacklogTeams moving to agile often struggle to integrate agile with best practices in user-centered design (UCD) and user experience (UX) in general. Fortunately, using a UX Integration Matrix helps integrate UX and agile by including UX information and requirements right in the product backlog.

While both agile and UX methods share some best practices—like iteration and defining requirements based on stories about users—agile and UX methods evolved for different purposes, supporting different values. Agile methods were developed without consideration for UX best practices. Early agile pioneers were working on in-house IT projects (custom software) or enterprise software [1, 2].

The economics are different in selling consumer products than when developing software for enterprises—UX matters more for consumer products. Jeff Bezos cares if users know how to click the button that puts money in his pocket more than Larry Ellison cares about any button in Oracle software. Larry makes money even if people can’t use his software. Oracle sells support contracts and professional services to fix things customers don’t like. Amazon and other online businesses can’t operate like that. They have to get the UX right, or they go out of business fast. User experience factors rarely get the same level of consideration when the end-user is not the same person as the paying customer [3].

Agile teams and UX problems

I’ve encountered two problems common among agile teams when helping them improve the user experience of their products or services:

  1. UX work is frequently overlooked during the release and sprint planning efforts.
  2. Teams often fail to measure the UX impact of their iterative efforts.

These two problems become more serious when combined.

When UX work goes undone and the impact is not measured, the team doing the work has no idea what is going on. The feedback loop is broken. Both agile and UX methods emphasize iteration, but productive iteration requires good feedback loops.

You can conduct development iterations (the focus of agile) or design iterations (the focus of UX), but if you fail to measure the impact of the iteration, you won’t see the real benefits of an iterative process. You will have no real idea if your offering is any closer to meeting the needs of the end user. The User Experience Integration Matrix (UXI Matrix) addresses these problems by tying UX to the project backlog.

Integrating UX into the product backlog

You can conduct development iterations (the focus of agile) or design iterations (the focus of UX), but if you fail to measure the impact of the iteration, you won’t see the real benefits of an iterative process. You will have no real idea if your offering is any closer to meeting the needs of the end user.

Scrum (one of the most popular variants of agile) advocates you create a Product Backlog, a collection of stories that describe user needs. The team iteratively refines the requirements, from rough (and often ill defined) to more specific. This is achieved using stories from a user’s perspective, which have conditions of satisfaction associated with them. This concept is adapted from UCD’s scenario based design methods. In my view, this is far better than other traditional approaches to documenting requirements that are often detached from user’s goals.

Various agile gurus [4, 5] discuss how to break down requirements from high-level stories to user needs targeted at specific users. Even if your team follows Jeff Patton’s story mapping method [6], (which I highly recommend) to create structured hierarchical representations, you’ll often find you want to analyze the stories by different factors as you groom your backlog.

I’ve worked with teams who want to analyze stories the following ways:

  1. Order of dependency or workflow (self-service password reset depends on user registration)
  2. Criticality (which of these stories must be done so customers pay us next month)
  3. How much work will they take to complete (show me all the epics)
  4. What related stories do I have (find requirements with related UI patterns)
  5. By role or persona (show me all the stories that impact persona X)
  6. What stories have a high impact on the UX, so we can focus on those?

If the project is small (both in number of team members and number of stories) you might be able to get away with rearranging story cards on the wall. However, in my experience, things inevitably get more complex. You often want to consider multiple factors when reviewing the backlog. A UXI Matrix helps you track and view these multiple factors.

Creating a UX Integration Matrix

The UXI Matrix is a simple, flexible, tool that extends the concept of the product backlog to include UX factors normally not tracked by agile teams. To create a UX Integration Matrix, you add several UX-related data points to your user stories:

Column Name Possible Values Description
Persona Persona’s name Identifies the persona a user story applies to
UX complexity 1 to 5 (or Fibonacci numbers if you’re into that sort of thing) Estimates design effort separate from implementation effort
Story verified Y/N Is this story fiction or fact? Is it based on user research or customer input?
Design complete Y/N Is the design coherent enough for development to be able to code it (or estimate how long it would take to code)?
Task completion rates 0 to 100% The percentage of users who have been observed to complete the story without any assistance.
Staffing Resource’s name Who’s owns the design, at whatever level of fidelity is agreed to.

With these columns added, your product backlog begins to look something like the spreadsheet in figure 1.



Figure 1: UX Integration Matrix, a simplified example

Figure 1: UX Integration Matrix, a simplified example

Advantages to using the UXI Matrix

The UXI Matrix helps teams integrate UX best practices and user-centered design by inserting UX at every level of the agile process.

Groom the backlog

During release and sprint planning you can sort, group, and filter user stories in Excel:

  • Group by themes or epics, or anything you want to add via new columns
  • A primary persona, a set of personas, or number of personas associated (more = higher)
  • Stories you’ve verified via user research or ones you need to verify
  • Stories you’ve completed but need to refine based on metrics
  • Export stories into a format that can be used in information architecture work
  • Explore related UX deliverables like personas, mockups, and research findings via hyperlinks to them

Note the summary rows near the bottom of the example in Figure 1. Those values can help you prioritize new and existing stories based on various UX factors.

Reduce design overhead

Perhaps my experience is unusual, but even when I’ve worked on teams as small as seven people, we still had trouble identifying redundant user stories or personas. My heuristic is that if a story shares several personas with another story in a multi-user system, then that story may be a duplicate. Grouping by themes can also help here.

Another useful heuristic is that if a persona shares a large list of user stories with another persona, it’s likely the personas should be merged. Most of the time personas that do the exact same things with a product can use the same design, unless of course they have very different skills or something, which becomes evident when reviewing the personas or conducting user research (which all good personas are based on in my view).

Facilitate collaboration

The UX Integration Matrix helps teams integrate UX best practices and user-centered design by inserting UX at every level of the agile process.

Another major benefit of the UXI Matrix format is you can share it with remote team members.

Listing assigned staff provides visibility into who’s doing what (see the columns under the heading Staffing). Team members can figure out who’s working on related stories and check on what’s complete, especially if you create a hyperlink to the design or research materials right there in the matrix.

For example, if there’s a Y in the Design Complete column, you can click the hyperlink on Y to review the design mockup. I’ve worked with teams who like to track review states here: work in progress (WIP), draft (D), reviewed ( R ), etc. (instead of just Y or N).

Track user involvement and other UX metrics

The UX Integration Matrix also helps track and share key UX metrics. One key metric to track is team contact with real end users. For example, if you’ve talked to real users to validate a persona, how many did you speak with? Another good metric is, how many users have been involved in testing the design (via usability tests, A/B tests, or otherwise)?

You can also capture objective, quantitative UX metrics like task completion rates, click/conversion rates, and satisfaction rates by persona. It makes it easier to convince the team to revisit previous designs when metrics show users cannot use a proposed design, or are unsatisfied with the current product or service. It can also be useful to track satisfaction by user story (or story specific stats from multivariate testing) in a column right next to the story.

Review UX metrics during sprint retrospectives

Scrum-style reviews and retrospectives are critical to improving both the design and team performance. However, few teams consider UX metrics as part of the retrospective. During these meetings, it’s helpful to have UX metrics next to the stories you are reviewing with the team.

I ask the team to answer these UX-related questions during the retrospective:

  1. Did we meet our UX goals for the sprint?
    1. Does our user research show that what we built is useful and usable?
    2. Are users satisfied with the new functionality (new stories)?
    3. Are users likely to promote our product or service (overall) to others?
    4. Do we have product usage metrics (via site analytics) that meet our expectations?
  2. Is there existing functionality (stories) that need to be refined before we add new stuff?
  3. What user research should we do to prioritize stories going forward?
  4. Do we have enough staff with UX skills to meet our objectives?
  5. Going forward should we:
    1. Work a sprint ahead to ensure we validate UX assumptions?
    2. Do a spike to refine a critical part of the UX?
    3. Refocus UX work on select user stories or personas?
    4. Improve our feedback mechanisms to capture factors we are missing today?


Annotated example

Figure 2: The UX Integration Matrix inserts key user experience activities and context adjacent to user stories into the product backlog.

Improving your team’s design decisions

Once you start tracking in the UX Integration Matrix it becomes easier to have informed discussions during reviews and retrospectives. I use the UXI Matrix to set UX goals with the team, help prioritize stories in the backlog, track UX work in progress, and to help answer the classic agile problem “what does done mean”; not just for the entire product or service, but for individual stories.

I’d be curious to hear from others who would like to share their experiences with variations of the above or similar methods. On the other hand, if you’re an agile guy that thinks this all is very non-agile, I’ll ask “can you really prove your method creates a better UX without this stuff?”

Start using the UXI Matrix in your next sprint. Download and customize this Excel template to get started:

[1] Larman, Craig. Agile and Iterative Development: A Manager’s Guide. Pearson Education, 2004.

[2] Sutherland, Jeff. “Agile Development: Lessons learned from the first scrum”. Cutter Agile Project Management Advisory Service, Executive Update, Vol. 5, No. 20, 2004: http://www.scrumalliance.org/resources/35.

[3] Cagan, Martin. Inspired: How To Create Products Customers Love. SVPG Press, 2010.

[4] Cohn, Mike. User Stories Applied: For Agile Software Development. Addison-Wesley Professional, 2004.

[5] Cockburn, Alistair. Agile Software Development. Addison-Wesley Professional, 2001.

[6] Patton, Jeff; “It’s All In How You Slice It”, Better Software, January 2005: http://www.agileproductdesign.com/writing/how_you_slice_it.pdf.