Search

Tuesday, April 01, 2014

Agile & Levels of Planning

Recall the last time you took a long driving trip.  You probably came up with a plan for the trip.  You used Google Maps to explore various routes and distances.  If the trip took a couple of days, you reserved a decent motel along the way.  Before the trip began you filled the gas tank and printed a map.  You even might have planned to leave your home earlier or later to avoid rush hour traffic along the way.

When the day to travel arrived, you reset the odometer, set up the GPS and off you went.  However, the job of planning wasn’t complete.  There might be detours along the way.  Frequent glances at the odometer or GPS might inform you that you are ahead or behind schedule.  Also, the map and GPS aren’t enough.  You monitor your speedometer from time-to-time and constantly look out the windshield and at the mirrors in case another driver does something unexpected.

We use all these different measures and methods to maintain a plan and respond to change for something as predictable as driving.  We need similar tools for the far more complex and uncertain endeavor of making a game!

Figure 1 - Planning Onions for Driving and Games

The figure above shows “the planning onion”, which represents the different layers of planning frequency.    The inner layers of the onion are the more frequent inspection tools/cycles, while the outer layers encompass tools applied less frequently.

Layers of Planning
Let’s examine the layers of planning from the inside out using the driving example for comparison with how a Scrum-developed game would plan:

  • Daily - The team will meet every day in a daily Scrum to discuss the progress and issues which are affecting their Sprint goal.  This includes conversations about bugs, impediments, dependencies, or merely points about the quality of the game.  This is like looking out the windshield of the car during your trip.
  • Sprint - The team, product owner and domain experts forecast what is possible to accomplish in the next one to three weeks.  The duration of the sprint largely depends on how much certainty the team has with the work and how long the stakeholders can go without seeing something.  A team will forecast and track the work in any way they want for the sprint.  Some teams use hours, others days, while some teams will come up with their own units[1].  This compares to using the speedometer in your car to measure you velocity.
  • Release - The team, stakeholders, marketing and leads identify stretch goals for the game that they hope will be demonstrated in a “shippable build” at the end of the release.  Product owners might alter these goals during the release as quality, cost and true velocity emerges.  Forecasting and tracking during a release is usually done using metrics that size the features on the backlog (e.g. story points).  This level of planning and tracking is comparable to using an odometer during your drive.  The odometer gives you a more long-term view of velocity when miles are measured against larger units of time, such as hours and days.
  • Project - For a cross-country drive, you’ll likely have a map and a rough idea of your progress and stops along the way.  You may have a gas budget and looked online to see where major construction delays  might occur on your path.  If asked where you expect to be tomorrow night, you’d have an answer like “Denver”.  If asked where you would be at 2:15 PM tomorrow, you might not be much more precise.  Long-term project planning should be like this.  It focuses on the major cost, schedule, and feature goals and identifies significant areas of risk.  Similarly, long term planning will break out epics and define some constraints based on experience.  An example for “online multiplayer gameplay” might be:
    • Cost: 10 people for six months to implement and 20 people for six months to produce content.
    • Schedule: Starting in June, finishing in 12 months.
    • Risk: Online technology risks and core gameplay risks.

Just as we don’t forecast our speed down every mile of road while planning a cross-country drive, a long project shouldn’t forecast what teams will be working on every day for a year or more[2].  Instead, as the planning horizon extends out, the metrics for planning become larger grained and less precise.  There are a few reasons for this:
Reason 1: The further out our plan goes, the more uncertainty there is with the details.
This is best illustrated with the cone of uncertainty shown in Figure 2:

Figure 2 - The Cone of Uncertainty

This figure illustrates that the further we are from shipping the game, the more uncertain its cost, scope and schedule.  Everyone who has shipped a game should recognize that the great concept art and ideas and plans expressed in the initial design documents usually don’t resemble the shipped game very closely.  This is not a bad thing.  This is the nature of creating complex games that have a subjective and emotional dimension.  We simply can’t “plan away” this complexity and uncertainty.  This doesn’t mean we can’t plan.  It means we need to plan at a level appropriate to the level of certainty that exists.

Reason 2: Simply breaking down a long term plan into finer details won’t give us more certainty, it’ll give us less uncertainty and waste our time doing it.
 This is the hardest to convince people of.  The assumption is that a 300 page design document creates twice as much certainly as a 150 page design document.  When I worked in the defense industry, this assumption led me to create a 40-pound specification document for an Air Force contract.  The Air Force wanted so much demonstrated certainty from their contractors, that they received documents so big they couldn’t read them[3].

However, numerous studies have shown[4] that not only is uncertainty not reduced equally with more planning effort but, beyond a certain point, the attempt to plan away uncertainty with more effort (documents and spreadsheets) produces forecasts with less accuracy (see Figure 3).  The reason that a bigger plan creates less accuracy is due to human nature.  It’s in our nature to see a big document and turn off our critical thinking, just as the Air Force did when they saw our 40-pound document.  Had they read that document with a critical eye, they would have quickly found out that it was a rushed cut-and-paste job by a junior engineer.  Instead, they simply weighed it and awarded us the contract.


Figure 3 - The Accuracy of Planning Based on Effort

The tools that we choose, based on where we are in the planning onion, help us stay in the ideal range of precision, which gives us the best amount of accuracy without wasting effort.

[1] I’ve seen some teams estimate in “NUTS”, which stand for “nebulous units of time”.
[2] Although I’ve seen some producers try to do this!
[3] Ever wonder why new fighter aircraft are years late and billions over budget?
[4] One of many quote is http://tinyurl.com/leasydl

No comments: