Tracking Individual Estimates

It's funny how you think of the best answer after the fact. One person came up and asked"How can you train people to make the correct estimates if you are not tracking their previous estimates and results".

This is one area that I personally fought over with Scrum. Fourteen months ago, I was in complete agreement with Joel Spolsky on how estimating should be done (which included tracking individual estimates and "actuals"). Now looking back on it, I see that 80% of what he describes is still the core of Scrum. However, the other 20% is still a command-and-control structure.

As I mentioned in the talk, ownership is one of the core benefits of Scrum. I don't think we have the "ownership" thing down yet at HighMoon. This takes time. We have seen the benefit of having tasks broken down and managed by the team. As I showed in the war room picture, the level of detail the team manages is far beyond what any one lead can. As they say, the devil is in the details and if you ignore the details, you'll get killed.

There is one core problem with tracking estimates at an individual level though. This is based on one of the most important lessons I've learned over the last year:

People base their behavior on how they believe they are being measured.

This is evident in how we've seen how small issues with our process has had big impacts on the behavior of people in the teams. This is one of the main reasons I recommend applying all rules of Scrum (at first) and hiring a coach after several months. Good Scrum coaches will look at your practices and can tell you about all those minor things you are doing wrong that have big implications. For example, we thought it was a good idea to record what everyone was saying in every scrum and publish it once a day to the team. The problem that resulted is that people thought they were being measured by the "good news" of what they reported and the scrums stopped being used to raise impediments and just report good news. We followed our coach's reccomendation to stop this saw an improvement (Scrums became more impediment driven). The reports were useful to me and the other directors, but this is one of those areas of Agile that former leads need to adjust to: not doing the command and control thing.

This is the core problem that I see with measuring estimate accuracy. People become concerned about underestimating that they begin to over-estimate to protect themselves. An over-estimated task will rarely be done early (this is called the schoolwork principle or something like that. Basically if you have an extra week than you think to accomplish a task, you will probably slack a bit for the first week).

Here's an example:

1. Task X is created. It should take 8 days for Programmer A to finish if nothing goes wrong.
2. Programmer A doesn't want to deliver late, so he estimates 12 days.
3. Programmer A takes it easy for 4 days. No sense in delivering early.
4. On day 12, an unforeseen risk is realized. It's going to take an extra 2 days to finish Task X.
5. That's OK, because programmer A's manager has a 25% reserve he's added into his estimates he passes up the chain to the master project.
6. Task X is completed in 14 days. Everyone is happy.

So originally, if Programmer A had focused on Task X from the start, then Task X would have been done in 10 days. So our scheduling process has driven up the cost 40%.

Scrum measures product value delivered in the game. This causes a different behavior than measuring estimates. Peer pressure on creating product value is far better than managerial pressure on correct estimation.

Just as with the daily status reports, my command and control side would like to see what everyone's estimation accuracy is, but I don't want to pay the price.