Tuesday, August 19, 2008

Production Debt

Have you ever seen a schedule for a game project that picks a day for production to begin? I think I've seen that on every project. Some day is chosen, a year or so in advance, and you have to start creating "production assets (levels usually)" on that day. I've never liked it and it's never worked out very well.

The problems are:
  1. Teams never transition from pre-production to production as if a switch were thrown. Different asset types transition at different rates. E.g. we may have level production down, but our animation system is still lagging.
  2. Where does this date come from? How do we know how much time we need for pre-production, and more importantly how do we know how much time we need for production, especially when we have a fixed ship date? How do we know if the time slot we're giving ourselves is enough so that we don't paint ourselves into a corner? How many times have you entered a nine month production phase with 12 months of work to do?
  3. What are the criteria for determining we CAN be producing assets? The date comes and we have to be in production, so we usually tell ourselves and our publishers "sure...we're in production! All those assets you see there are really production assets that aren't done...yeah". You're still iterating to find the fun and you can't iterate as much in production!
Measuring Production Debt in Preproduction

Production work is a necessary debt that we create in preproduction. One of the goals of preproduction releases (or milestones) is to measure that debt. During the first few releases that debt will be uncertain. You might say "we believe that we have 1000 people months of production work, plus or minus 200". The project would continuously refine that number during subsequent releases. Towards the end of preproduction you want it to be a much more accurate number.

Why Measure Production Debt?

Imagine you are on a first person shooter project that has 12 levels planned and 12 months of production scheduled. During development your team implements some cool technology that allows every piece of non ground geometry to be destroyed or have holes blown in it. This is a great new addition but it creates a problem. It doubles the amount of effort required to build a level.

How do we react to this? The first problem is that we have to know the problem exists. In many cases we don't do a proper job of measuring the production impact of design and technology changes. We paint ourselves into the proverbial corner. This is why demonstrating the cost of production should be a part of every release review.

If we know that we will double the level production debt with a new feature developed, such as the destructible geometry feature, we can make solid decisions early. Some choices would be:
  • Drop the feature because we can't afford it
  • Drop half the levels
  • Start production earlier
  • Extend production and ship date
  • Scale up production resources
Some of these choices are harder than others, but they are all better choices than the one that is usually taken: trying to stuff 24 months of production effort into 12 months of schedule.

How to Measure Production Debt

The only way to measure production debt is to make a small set of production quality assets, measure the effort and scale the measurement. For our example, our release would demonstrate a small level with a couple of buildings that are destructible. If we estimated that the demo level was 10% of an entire level we could say that the level asset debt would be 10 times the effort spent on the demo times the 12 levels we are planning.

As mentioned this would be an inaccurate measurement in early releases. We can't always make truly shipping quality demo levels early in a project. Subsequent releases would get closer and the estimate more accurate.

The good thing is that we can expect that we'll get more efficient at creating levels during production and that will free up time to incrementally improve the levels during that time.

What Happens to the Production Date?

Publishers will rarely allow a team to tell them "we'll pick a production date when we know more". It's OK to pick a production date with the caveat that the date will be refined per asset type every release. Back the refinement up with higher quality asset demos every release. Publishers and developers should have ongoing discussions of the questions raised above when it's clear that new features are impacting production debt. Publishers rarely get good information about cost and benefit information about major features.

Production debt (like every other debt it seems) always grows. The problem with production debt is that we can't retire it in an ongoing was as other debts (design, technical, etc.) are. Don't make your team attempt to pay it off with crunch.

No comments: