Wednesday, September 05, 2007

Sprint Burndown Charts for Feature Teams

The Sprint Burndown chart is a one of the most useful tools in Scrum. It helps make the progress of the work being done against the Sprint Backlog visible to the team and others.

The team can use the backlog to estimate if they are going to meet the goals based on the velocity (slope) of the Burndown and react as early as possible if there is a threat to meeting their commitment.

There are a number of issues that have come up with Burndown charts as they are being used for Feature Teams however. I've been thinking about the Burndown charts and would like to share some of the reasoning behind tweaking a core practice.

The teams are cross-disciplined
Early in its life, Scrum was mainly practiced by IT teams that were dominated by programmers which influenced the standard practices. For example, if a team is made up of C++ programmers and the Burndown is showing that they are falling behind, maybe due to problems with one story, then the entire team can pitch in and help each other reach the goal. Maybe they all come in on a Saturday and work together.

For game teams, especially feature teams, we have a much wider range of disciplines within each team. Cross-functional teams can lack this all-for-one and one-for-all benefit that comes more easily to teams of one discipline. Take the example of a Feature Team that has a single character artist that is falling behind in their work. The Burndown probably won't show this clearly well in advance. How does the team respond to being behind on character models? If the artist comes in on Saturday to catch up, does the entire team come in and do nothing or take the day off? Either solution does not foster the best team spirit.

Dependencies can also effect the progress of a story through the hand-off of work. If there is a chain of tasks that occur after the character modeler is completed with their work, then a delay in one task can shift the entire story and endanger the Sprint. Again, this is a problem that is not usually seen on teams of just character modelers or programmers.

Another problem is that artists and designers work better with time-boxed estimates rather than completion estimates. The difference is that time-boxed estimate are the limit that the designer or artists will take to work on something and completion estimates are more definitive estimates better applied to programming tasks. Artists can iterate, refine and polish a piece of art for as long as they like. It's often a subjective judgment to say it's good enough. Programmers have a bit easier time of determining completeness. Code either works or it does not. We usually don't have the urge to refine code forever until it is "beautiful enough" (there are exceptions though!). Should this difference be reflected in a Burndown differently?

Sprint Task Backlogs Change during the Sprint
The stories that the team committed to completing don't change, but teams are allowed to change their task backlogs during a Sprint. We've found that the Backlog task list can grow up to 50% depending on how well mechanic is understood. We tried to spend more time in Sprint planning to eliminate the uncertainty, but it produced worse results at the reviews by limiting flexibility within the team to explore and discover. We tried defining a great deal of detail within the "conditions of satisfaction" for the stories, but they didn't define the qualitative bar we were seeking; "Finding the fun" involves uncertainty even at the Sprint granularity. So teams adopted Product Owners as Pigs to shrink product decision loops down to necessary.

The obvious problem with so much flux in the backlog is that the Burndown chart projections aren't as transparent of the backlog progress as they should be. Fortunately there is still some predictability from Sprint to Sprint. A team that estimated 250 hours of work and successfully finished 500 within their Sprint can safely commit to 250 hours the next Sprint and expect the same expansion. Still, having a good day-to-day measure of progress is desirable.

The Value is in the Stories and not the Backlog
With so much change among a cross-disciplined team, the connection between a Burndown chart hitting zero hours and "doneness" in the stories becomes a lot fuzzier. In fact there is a real danger of teams becoming too fixated on the Burndown and not enough on "doneness". Traditionally you can look at a Burndown chart and read into the issues of the team. A consistent velocity on the Burndown (straight line from start to end) usually is a good indication that the team is too focused on keeping a constant velocity though the unconscious manipulation of their task estimate. Nature abhors a straight Burndown. This causes problems which are apparent at the review:
  • The team shows consistent lower velocity in value added to the feature/game.
  • The technical/design debt of the team can grow. Bugs, loose ends, etc are in full view.
As a customer I tell the teams "I don't care about you completing all the stories so much as seeing the team nail the higher priority ones". Great games are the ones that have a few great features rather than many mediocre ones. I would rather see a team blow us away with a few of the top stories and leave the bottom ones for the next Sprint than complete them all to the "letter of the law, but not the spirit". When you tell a team to keep going back to the same story (or minor variants of it) in an attempt to find the fun, then the team is not taking proper ownership of the feature. A great feature is going to take the time it needs to be fully developed. It's difficult to predict.

Creativity = Fun = Success
What makes working in the game industry so great is that value in the product is built day-to-day through the creativity of everyone on the team. Hmmm…what makes the product better is what makes working on the product so much fun? Give me more of that! Somehow writing "make it REALLY fun" as a condition of satisfaction isn't enough. It needs to happen at the team level.

An Ideal Feature Team Burndown - Does it exist?
So what can a team do to address these issues? As Ken Schwaber says "Scrum is about making things visible to that you can make common sense decisions".
So what are we trying to make visible?
  • Progress of each story towards being "done".
  • Dependencies and handoffs within the team.
  • When we are running short of time (as early as possible) to finish everything.
Suggestion - Visualizing Time-boxed Workflow
Perhaps a story is to finish a section of a level. Rather than detail each task out, the team will create time-box estimates of the stages that the production pipeline requires:
  • Initial concept - 1 days
  • Design modular pass - 3 days
  • Art pass - 3 days
  • Design tuning - 2 days
  • Art fine pass - 1 day
Time-boxing turns out to be a more useful tool for this kind of work (see above). When a pipeline of time-boxed work exists for a story, it would be great for the team to see where they are in the time-box versus where they should be based on remaining time.
How would we visualize this? The simplest thing would be to show:
  • What stage are we currently working in for a story?
  • How are we tracking for total time and stage time?
This could be shown on the task-board next to the stories. The boxes would be scaled for each stage (in calendar days) to match the two timelines on the bottom (one for calendar timeline, one for actual progress time). The focus of the daily Scrum would not be going around the room and asking the three questions, but starting at the highest story and asking about the completeness goal of each story. The answers to the three questions would naturally emerge from discussing progress at the story level.

Based on the Daily Scrum reporting, the progress token would be moved over. If the progress was slipping behind schedule it would become apparent each day and discussed. In the figure above, this is the case. Progress is behind schedule (current time).
The problem I see with this is that it is not as effective for stories that don't have such a pipeline of time-boxed work. Do we need to have some form of burndown for every story? If so, do we lose some value in a single Sprint? Is there some way we can still have a single chart to show the big picture progress of the Sprint?
The only way to find out is to try it out at the team level and evolve it. Retrospectives are great places to discuss these things and tweak them further. Ongoing practice refinement like this is critical to getting the most out of Scrum.


Mishkin Berteig said...

Hmm... You make an interesting comment by quoting Ken:

So what can a team do to address these issues? As Ken Schwaber says "Scrum is about making things visible to that you can make common sense decisions".

I have to question your identification of the issues. The issue that is being exposed is about how your cross functional team is composed of specialists, and not generalizing specialists. I wonder if this is because the team is not making a team-level commitment to the work? Are your team members committing at the start of each sprint to a common team goal? Is management supporting them unconditionally in meeting this goal? Is a c++ (or whatever) developer allowed and encouraged to assist the character designer and learn how to do that work too? Are you measuring people by their area of responsibility or their area of influence? Are your team members volunteering for tasks as they work or are they all allocated at the start of the sprint?

Changing your burndown chart won't fix any of these problems, and these are the things you should be worried about. I do believe, from your description, that you can improve your use of burndown charts, but it will actually just make these problems feel even worse!!!

Clinton Keith said...

Why would we want to "generalize specialists" when the specialties are as diverse as art, design and programming? How is the existence of such a highly diverse team indicative of a lack of commitment?

The issue is cross discipline hand offs which "Scrum by the book" does not address as well.

I am writing a follow-up to this post which describes how Kanban is a better application for this issue and that the Scrum task board is a simplified Kanban system.

This is more of a demonstration about how teams that take ownership and make commitments can alter the practices without harming the principles.