Thursday, December 14, 2017

The Challenges of Using Story Points for Games

Story Points - The Challenges
Many game teams have used story points to measure items in the their product backlog.  There has been a lot of confusion, misuse and overuse of them.

Let’s look at some of these challenges

Using points for stories that multiple disciplines work on
Story points are meant to drive cross-functional behavior and to focus the cross-functional team on delivering increasing levels of value (velocity).  This is done by encouraging discussions about improving collaboration.

For example, if QA is a bottleneck, then the discussion turns to questions such as:
  • “How can we create functionality that doesn’t require so much QA effort?”
  • “What can the rest of the team do to alleviate this bottleneck?”
  • Etc.
Cross-functional story points are meant for complex work that is a bit unpredictable, such as developing a new mechanic or character behavior.  Teams swarm on complex work, iterating and fixing things daily.  It’s should be a true cross-discipline effort

If the work is more predictable, such as content production, then we can use functional-oriented points, or directly using time estimates with Kanban tools.  With such work, there is a flow of handoffs and we can level that flow by measuring each step (however we’re still trying to optimize the delivery of value for the entire flow, not individual steps).

Thinking story points are precise
Story points are meant to be a fast and efficient way to continually estimate and reestimate the cost of features (stories).  This helps us prioritize them and to foster conversations that don’t take up enormous amounts of time.

There is a prevalent myth that the bigger our plan, the more accurate it is.  In reality, it’s the opposite.  The better plan is the one that is frequently revisited as we learn more.  A hundred one-hour planning sessions held over two years produces better results than 100 hours of planning done up front.  Story points allow such frequent planning by being brief and focused.  An hour or two of backlog refinement every sprint is usually enough.

Translating story points to time estimates
Story points are often directly converted to time estimates.  There are a few problems with doing this:
  • Time estimates depend on who is doing which part of the story.  Any two developers can have a magnitude of difference in the amount of time it takes to complete something based on skill, experience and other factors.  Cross-functional stories don’t provide this information.
  • As rough, relative measurements, story points are not precise enough to convert to time.  This is a reason that we plan using time for a limited horizon, such as a sprint’s duration.  A story that takes 5 minutes to assign points to might take an hour to task out in sprint planning.
So what good are story points?
Story points exchange the illusion of detailed up-front planning (big documents, detailed schedules) with more frequent and brief planning sessions based on the emergent knowledge we gain over the course of developing a game.  We learn the true cost of making something fun based on execution, not documentation.

By continually updating a forecast based not just on promises made at the start of the game, but based on what reality is telling us during development allows us to better manage the outcomes.  Agile planning is better at hitting fixed dates with quality than any other approach.

The big challenge is adopting the mindset to use them correctly.  Too often our stakeholders are not aware of the power of such tools and want specific answers up front.  That’s the biggest challenge.