Build and manage the product backlog
By Mitch Lacey. Owner, Mitch Lacey & Associates, Inc, a consulting firm specializing in agile and scrum adoptions and improvements.
January 2012
In this article, Mitch Lacey explains the importance of a product backlog, describes what makes a good backlog, and provides some best practices for creating and maintaining your backlog.
Applies To
Application Lifecycle Management; Team Foundation Server (TFS)
In this article:
Introduction
The Product Backlog
User Stories
Estimating
Prioritization
A Living Product Backlog
- Grooming
A product backlog is a prioritized list of all the features and functionality needed to complete a project. In TFS, you manage your product backlog using work items. Your choice of work item types will differ depending on the process template used to create your team project. To learn more about process templates and the work item types they support, see Work with team project artifacts, choose a process template.
A good product backlog is at the heart of any well-functioning agile team and has the following characteristics:
It is prioritized to ensure that the team builds the most important features first.
It is estimated by the team to give the product owner clarity and enabling him to answer questions such as “When will these stories be done?”
It includes all the work necessary to complete the project.
It is a living document, constantly changing and evolving to reflect the current realities of the project.
A good product backlog does not automatically ensure good software. However, the lack of a good product backlog often results in incomplete software that does not meet the requirements of your customers and stakeholders.
Managing the product backlog is a full-time job. Simple techniques can help change what can be an overwhelming, time-consuming process to an interactive, iterative process that effectively engages team members, customers, and stakeholders. It’s essential, therefore, to learn solid techniques to help you build, prioritize, and maintain your product backlog.
The Product Backlog
The product backlog is a prioritized, living master list of all the features and functionality needed to complete the project. Product backlogs commonly include requirements user stories (e.g. requirements), bugs, research tasks (spikes) and engineering improvements. These items are estimated in abstract units that are often called story points.
Product backlogs include all work that the project will require and evolve over time. As such, they contain not only the new features for a product but also bug fixes and research—anything that will take the team’s time. All work that the team will do must come from the product backlog: it’s the front door to the project. If the product backlog includes all work, the product owner, the team, and management have a better picture of the work that remains and reduces last-minute surprises.
At the start of any project, it’s unlikely that you'll have a nice list of clean, well-defined product backlog items ready to be estimated and prioritized. Instead, you’re likely to have a stack of story note cards and maybe a functional specification or two. As the product owner, it’s your job to get this mess into some sort of order so that the team can start to estimate the backlog.
User Stories
The first step is to convert all of the ideas and notes into user stories, which express end user desired functionality (what the software should do) but not the implementation details (how to create software that meets those requirements). The title of each user story should follow the format, “As a <user>, I want <functionality> so that <reason>.”
.png "An Example of a Story Card")
Like the product backlog itself, each user story will evolve over time. Over the course of the project, your team will prioritize and estimate these stories, add detailed information to them, and break them down into smaller stories or delete them altogether. Those that are brought into sprints are implemented and delivered to your customers.
To read more about user stories, refer to Create your backlog and Storyboard your ideas using PowerPoint.
After you convert all the ideas and notes into user stories, it’s time to estimate and prioritize.
Estimating
By definition, estimating is imprecise. However, you can become much better at creating relatively accurate estimates with time, practice, and the participation of your entire team The first step is to gather the team to provide estimates on the product backlog items. When the team estimates each story, they give it a relative size estimate with an abstract unit of measurement, called a story point.
Estimates are essential for two reasons:
They help answer the question, “When will we be done?”
They help the product owner determine each item’s priority.
Estimates give the product owner and the team an idea of a particular story’s cost, which, in turn, helps the product owner prioritize the stories relative to each other. The bigger the item’s estimate, the more expensive it will be to implement in terms of time and resources. Therefore, an item that might have been high on a product owner’s wish list might decrease in priority if it comes at a high cost.
The team can use Planning Poker, the Big Wall, or other techniques to estimate the work in terms of story points. For more information about estimation techniques and a quick lesson in story points, see Estimating and Groom and estimate the backlog.
After the team has estimated all of the stories, it’s time to prioritize.
Prioritization
All product backlogs are prioritized in terms of business value and risk. The product owner compares each item on the backlog against the others to determine its relative priority. To do this, the product owner considers each item’s size, its value to the business, and any associated risk. Items are then sorted in descending order of priority. High-priority items appear at or near the top of the product backlog, and lower priority items reside at or near the bottom. Teams choose work for the upcoming sprint from among the highest priority items, so that the most important items are completed first.
Not every item in the backlog is the same size. Some can be completed in one sprint, but others are so big that the team isn’t exactly sure what’s involved or how long it will take to implement them. That’s OK. As items move closer to the top of the backlog, the team will make them smaller and more focused so that everyone better understands the work that they will address in the upcoming sprints.
As with estimation, prioritizing the initial product backlog can be daunting. How do you effectively balance competing stakeholder demands, while considering the end product, risks, and costs? Luckily, several techniques exist that make the task easier, including Innovation Games and Relative Weighting. See Prioritization and Groom and estimate the backlog for more on these and other techniques.
Whatever prioritization technique you choose, you must order the work to ensure that the team builds features that give the most value to the business, the stakeholders, and the customers. If you don't prioritize the work, you increase the risk of delivering lower priority user stories instead of higher ones and incomplete user stories when resources and schedules change.
(For more information about the nature of backlog items, see Create your backlog and Work with portfolio backlogs).
A Living Product Backlog
What I have described so far has focused on going from nothing to an estimated and prioritized product backlog. Unlike a requirements document, product backlogs are not created at the beginning of the project and then left on a shelf. Instead, the product backlog evolves, expanding and contracting with the realities of the project. Some product backlog items will become irrelevant and need to be removed. Others will bubble to the surface and need to be broken down into smaller stories. New user stories will be added, estimated, and prioritized as additional requirements emerge.
The team and stakeholders are involved in creating and managing the product backlog, but the product owner has ultimate responsibility for it. The product owner must ensure that the backlog is clean, prioritized, and up to date so that both customers and the team have a clear picture of the working plan for the project release. Even after the project is in full swing, product owners still have plenty of work to do to keep the product backlog in shape:
Adding and prioritizing new stories
Asking the team to estimate new stories and re-estimating old ones as they are better understood
Reviewing upcoming user stories with the team to break down items as necessary
Meeting with customers and stakeholders to review and add requirements
Anyone may add items to the product backlog at any time, but only the product owner may prioritize them. The product owner is also the only one who can assign a priority to a story. All other members of the team, and stakeholders, should leave the priority blank when adding a story, even if they are using an electronic tool that prompts them for that information.
When a story is added, the product owner will make a preliminary assessment of its priority based on his understanding of it. He will discuss the story with its creator to better understand it so that he can, in turn, answer questions from the team. At a predetermined time during each sprint, the product owner will meet with the team to discuss new stories and collaboratively estimate them so that he can more accurately prioritize them relative to other stories in the backlog. This process is called grooming the backlog.
Grooming
Product backlog grooming, as mentioned previously, must happen regularly.
In Scrum, the team should spend 5%-15% of its time each sprint in grooming activities. The team must understand what’s coming up and what’s changing so that they can break down any large stories that are rising in priority, estimate any stories that have been created, and do some emergent design and planning for upcoming stories. To ensure that this happens, the product owner and the team should, during each sprint planning meeting, set aside some time during that sprint to groom the backlog together. To read more about sprint planning, see Sprint Planning and Work in sprints.
During a two-week sprint, I like to hold this meeting during the second week. This gives the product owner enough time to have meaningful conversations with the customers and stakeholders, get an understanding of changes in the business, and to clarify user stories and stories that are new or rising in priority. It’s also a logical time during the sprint to start to anticipate the coming sprints. You can decide when to hold the meeting. The important thing is to allow enough time during the sprint to complete grooming activities.
During a typical grooming meeting, the product owner presents what’s new, what’s changed, and his plan for the next few sprints. In addition to estimating new stories and breaking down ones that must be completed soon, the team takes time during this meeting to review the current architecture of the system and to plan for or design upcoming features. During this process, story estimates often change, and new stories tend to appear as larger stories are broken down into smaller stories.
This process seems simple, but it can be a bit of a struggle. To help things run smoothly, the product owner must be prepared to answer questions. Conflict can ensue if, for example, the product owner is planning for a story to be done in the next sprint but cannot provide the clarity that the team needs to provide a good estimate. If this happens during your sprint planning meetings, the ScrumMaster should coach the product owner as to what information he should bring from the customers and stakeholders to the team.
At the end of each grooming meeting, the product owner should publish the changes to the stakeholders and customers so that everyone can see what’s new, what’s coming up, and the updated release plan.
A good product backlog helps ensure that the software you build has the most important features as identified in your conversations with customers and stakeholders, and defined in your user stories. In order to create and maintain a good product backlog, you need to stay actively engaged with both the stakeholder/customer group and the team on a regular basis—every sprint.
Building a good backlog does not ensure a good system, but the lack of a good backlog will almost ultimately ensure that you have a system that does not do what the customers asked for. In other words, not keeping the backlog up to date will almost always result in project failure.
The product owner is a full-time job, and the backlog is their responsibility. Take the role seriously. Keep the product backlog in a good state—your customers will thank you.