No estimates and set-based design

I’ve been following the #noestimates discussion on twitter over the past few months.  I understand the reasoning on both sides of this divisive topic of whether teams should estimate or not.  My opinion on the topic is that the time spent on creating estimates and planning diminishes as a team gains maturity.  Mature teams create a more intuitive and reliable way of figuring out how to work, their velocity is superb and reliable and there is trust created in the organization that replaces the need to insist on such up-front commitments.  There are a number of benefits to this.  One benefit is that the time spent creating estimates is now time spend doing the work.  The second, and most significant benefit, is that the options and possibilities remain open longer when mature and trusted teams do not commit to a single solution early on.  This is the type of difference described in my book between point-based design and set-based design and a concept I wrote about in 2006.  Point-based design is where a single point is selected from a cloud of possibilities.  When we create a detailed estimate, we are selecting a narrow range of solutions from a larger set; we are deciding how to do something up front.  Everything outside that range (or point) is discarded.  The further out these estimates go, the larger the range of discarded possibilities become (referred to as the cone of uncertainty).

Set-based design is the idea that we keep considering the set of all possible solutions until we learn more through execution.  A classic example of this is the design of the Toyota Prius.  The vision for the Prius was for a “green car”, but the decision of whether to use a efficient gas, all electric or a hybrid power-train was deferred until all possibilities were explored to the point of knowing about the efficiency and production cost of each.  As a result of this approach, this revolutionary car was designed in half the time of more conventional designs.

Avoiding detailed estimates supports set-based design by avoiding the early, uninformed, commitment to a single solution that is required to assign a number (hours, days, points).  As mentioned, a no-estimates approach only works with a mature team.  The team has to execute in a way to produces the knowledge to refine the set of possibilities first.

For a game mechanic epic, this means that the team finds a way to demonstrate the core gameplay of the mechanic first and refines it every iteration.  Teams that build a bunch of parts of the mechanic that  later integrates into a fun experience are really doing point-based design; they are choosing the solution first.

From experience, I’ve learned something which sounds intuitively wrong at first:
Set-based design is one of the biggest cost savers you will find.

As demonstrated, it halved the design of the Prius for Toyota.  I’ve seen the same on game teams.  Set-based design forces teams to focus on the simplest things first and to maximize the work not done.  An example I’ve seen is with complex AI systems.  Many games start off by creating a list of everything the AI has to do in the game (point-based design).  Then an AI engine is created to do them.  Although integration is frequent, the AI isn’t really demonstrating emergent value to the player yet.  It’s a “work-in-progress that will payoff before alpha”.  Somewhere on this road to value, there is a major rewrite of the AI, a reduction in the AI requirements or an admission that “our game’s AI is not going to be very good”.

A set-based approach to this problem starts with asking “what does our AI need to do first?”.   The answer shouldn’t be “create path-finding NPCs”.  The player doesn’t care about the NPCs path-finding if they don’t add anything to their experience.  Perhaps having only NPCs that are close by and interact in a meaningful way is enough.  Maybe we’ll discover that we don’t really need them to path-find as much or in the way we'd planned.   Maybe there’s some “little trick” we can do to make the player think the NPCs are doing something complex when they are out of sight.

The additional benefit of a set-based approach is that the team is much more connected with the game.  They are asking themselves “how do we make the game better” daily.  This is better than the question being asked once every two weeks during sprint planning and far better than asking once every three months during release planning!  Teams that are more connected to the game have increased opportunities to add value and subtract work that doesn’t need to be done.

Again, I want to emphasize that abandoning estimation is not the goal, but a side effect of teams maturing.  The goal is to coach teams to maturity and continuous improvement.  When I train new teams, I don’t train them to abandon estimation.    That would be like training a snowboarding novice to do a 1080: it’s a fast path to crashing.  When a team has reached a level of maturity, they’ll find this on their own (with a bit of coaching along the way).

Productivity is a function of engagement, skill and accountability

Productivity is a function of engagement, skill and accountability.  Engagement, skill and accountability are cultivated through trust and respect.  Trust and respect are the foundations of a culture.  Studios with a foundation of respect and trust demonstrate it by investing in the growth of their people, not by treating them as capital resources that can be added or laid off to hit goals. They treat mistakes as opportunities to learn.