Sunday, June 17, 2007

No Silver Bullet

A Disturbing Assumption

I occasionally come across comments like this one:

It'll be very interesting to see how High Moon's next game does, because I suspect that the hipness of agile methods in our industry will explode if it's a hit and collapse if it's a failure.

I don’t agree with this prediction either way and we have tried to emphasize this in the presentations:

Agile is not a silver bullet

Some of the key points we stated were:

  • There is no “right way” to make a game.
  • The talent of your team is the main factor in the quality of the game.
  • Agile has benefits:
    • Reducing wasted effort
    • Finding the fun first

I’m not saying:

  • Using agile makes you a hit factory
  • That High Moon has any greater insight than anyone else using agile
    • We’ve just been talking about it for a few years.
  • Waterfall will always fail.

As for High Moon, I’m enjoying myself. Its fun, challenging and we are improving how we make games every day. Will the next game be a hit? I don’t know. It has the strong potential but I’ve given up predicting success solely based on the development side alone. I’ve never predicted it based on the process used to create it!

Let me back that up with a look at a past project I worked on.

A Comparison

One game I worked on was Midnight Club at Angel Studios. It was not a game developed using agile practices which makes for good comparison.

Some facts:

  • Fixed price, fixed schedule development (it was a PS2 launch title).
    • We shipped on time and budget
  • The original scope included six cities
    • We shipped with two
  • The last four months of development was tough.
    • 72 hour weeks for most.
  • The game sold 2.5 million units
    • It was a success for the team and the customer

How did it succeed?

The keys to success were:

  • Talent
    • It was a great team, especially with the technical leadership of Dave Etherton and design leadership of Fred Marcus.
  • Experience
    • The team largely consisted of the Midtown Madness team. Vision wasn’t a problem.
  • Marketing of Rockstar
    • These guys were amazing. Who else would pay to close down Times Square for a night to get box cover shots?
  • Technology
    • Closely tied with talent and experience, but there is benefit to knowing your mature and adoptable technology base from the start. Adopting the PS2 was not easy though.

How would agile have helped Midnight Club?

Agile could have helped mitigate the crunch at the end. Many of us were truly burned out after MC1 and a few good people left. I don’t know if we could have turned out a third city, but dropping that city late in development had a major impact on the flow of the game. Knowing we were going to ship with two cities earlier would have reduced the effort wasted on the third and allowed the designers to adapt the gameplay for two cities.

There are two key points to summarize

  • We succeeded with waterfall.
  • Agile could have helped significantly with crunch and somewhat with gameplay.

Waterfall is not a blank bullet

Waterfall has been rejected largely due to the higher percentage of projects that using it fail (miss budget, quality or schedule goals or are canceled outright). That percentage is not 100% though. Process contributes to but does not determine success.

The point of agile means you shouldn’t accept or reject it blindly

Agile methods use an iterative approach. You are constantly in an “inspect and adapt” cycle. This applies to products, teams, whole companies and even the process itself!

Frameworks such as Scrum or XP suggest practices that support the “inspect and adapt” cycle. What you do with the results of your inspection is up to you. If you find yourself inspecting how teams work together, including everyone in how the operations of the company can be constantly improved and trying to honestly ask yourself if the projects are headed in the right direction on a regular basis, then your shouldn’t care about High Moon’s games!

Scrum is self modifying in nature. If you aren’t achieving the results, then change the process! Just try to preserve the principles though. Too many scrum adopters remove or ignore the principle of change to their detriment.

High Moon’s Next Game

Regardless of the success of failure of our next game, I believe that we are better off using agile to develop it.

If our next game is a hit, does that mean you can do the same by just adopting our practices? No! It means that we had the talent, technology and marketing to make a successful game. If the game fails, we will not blame our process, but ourselves.

The goal is to compete with each other on the basis of talent and not the illusion that we are competing on the basis of who has management that can make their people work longer hours. Toyota writes books about their system and gives competing auto makers tours of their companies and they are still the number one auto maker. Their culture creates success.


Building your studio culture to never be satisfied and to improve your games and how you make games is the ultimate goal. When I read articles like this, I get anxious to help build a culture like that. The irony is that you can't impose this kind of culture. It has to be grown though trust and ownership at every level. These are principles which were used to create agile. Agile can contribute to culture, but it's just a part.


Martin V said...

I agree in your main points, but I think in some ways you're beating a dead horse.

Clinton Keith said...


Thanks for the feedback.

I'd like to agree with you. I wish Waterfall was a dead horse, but it isn't. It's the single most employed methodology out there.

Perhaps the confusion is labeling all phased development as Waterfall. In the strictest sense, Waterfall is defined as a one way development with functional integration at the end of the project.

Most of those using a planned & phased development approach aren't delaying integration that long. They are integrating and running the game far more frequently. Milestones can be iterative in nature even if they are planned out in advance.

This is how I would characterize the effort on Midnight Club and in that respect it is a modern process (not defined by theory, but actual common use).


Martin V said...

Yeah, I guess no-one would ever call their process waterfall even though it is. I've also heard people call their process agile without seeming to know what it means. Still, we should not assume that any non-agile process is waterfall.

Clinton Keith said...

Agile is definitely overused.

I don't believe I've ever stated or implied that "any non-agile process is waterfall". I certainly haven't assumed it.

One way product development methodologies can be classified is by their iteration length and the weight of their ceremony (as defined by Craig Larman). Waterfall would lie at one end of this and Scrum on another.

Martin V said...

I was just trying to say that someone who reads your post may assume that anything that's not agile is waterfall. You and me obviously agree that it is not so. Sorry if that was unclear.

Clinton Keith said...


You're absolutely correct. Thanks for pointing it out.