Monday, April 30, 2012

Valve's Culture, Self-Organization and Scrum

“If you don’t like change, you’re going to like irrelevance even less.” 
- General Eric Shinseki

In the spring of 2012 Valve's New Employee Handbook was leaked.  The handbook's release has led to a number of discussions about the merit of The Cabal (what Valve calls their process) and their work environment.   For me, it's hard to argue with success and everything I've read about Valve being a great place to work.

Since first reading about The Cabal in 1999 and attending a few GDC sessions since, I've been inspired.  That inspiration helped lead me to agile thinking.   I felt there was a connection with agile and the kind of place, like Valve, where I wanted to work.  A place where rigid process and hierarchies were considered a mismatch to creative development.

Valve's handbook states this belief near the start:

Hierarchy is great for maintaining predictability and repeatability. It simplifies planning and makes it easier to control a large group of people from the top down, which is why military organizations rely on it so heavily.

But when you’re an entertainment company that’s spent the last decade going out of its way to recruit the most intelligent, innovative, talented people on Earth, telling them to sit at a desk and do what they’re told obliterates 99 percent of their value. We want innovators, and that means maintaining an environment where they’ll flourish. 

The handbook goes on to describe the role of an employee in this environment.  The criticisms I've heard about The Cabal often say "you need the right kind of people for this to work".  I agree, but I think that the potential pool of such people is larger if you provide mentoring to help the transition into such an environment.  The handbook acknowledges this as a challenge:

There are a number of things we wish we were better at:
  • Helping new people find their way. We wrote this book to help, but as we said above, a book can only go so far. 
  • Mentoring people. Not just helping new people figure things out, but proactively helping people to grow in areas where they need help is something we’re organizationally not great at. Peer reviews help, but they can only go so far. 
This is a common challenge in any studio that is attempting to improve self-organization.  People have to unlearn a lifetime pattern first imposed in our public education systems, which train children how to work in a task-driven top-down hierarchical organization.  It's an even greater challenge for a studio that has been hierarchical, since it threatens the status quo (more on this later).

Self-organization and hierarchies aren't mutually exclusive.  Gabe Newell leads Valve; it's not a pure democracy, but Valve doesn't have many layers between him and an artist creating texture maps.  Nor is the artist being handed a list of texture maps he or she is assigned to create during the week.  The artist is expected to be a professional and is treated like an adult by being allowed to be personally accountable.  What self-organization does is to flatten hierarchies and reduce the number of lines of communication between people that need to communicate.

Scrum and The Cabal have the same goals
Scrum is a framework for iterative and incremental product development based around self-organizing teams.  Team size, sprint durations, Scrum roles, etc. are meant to foster self-organization.  Every 2-3 weeks, the team to inspects their work and practices and seeks ways to improve both.  The roles provide for clear interfaces and areas of ownership.  The benefit is that after awhile, every individual should find motivation in seeking these improvements.  This motivation builds on itself, accelerates and leads to a better working environment.  

Good working environments and profitability aren't exclusive.  While Valve enjoys the highest retention rates of almost any company, it also makes more profit per employee than even Google or Apple!

The goal of Scrum adoption is not to "do Scrum perfectly" but to establish a framework that will lead to such a culture.  It's been referred to as a starting script for self-organization.

Why is it so hard then?
So why do few companies ever achieve similar cultures?  Why is it so challenging for organizations that adopt the Scrum framework to become like Valve?  This is the big question.  There are many different explanations and reasons for this I've read from "people mostly suck" (a comment I saw today) to "Scrum sucks".  Both are simplistic and wrong.

I believe it mostly lies in cultural resistance.  As mentioned above, an organization that grows in a hierarchical pattern, resists the adoption of self-organization.  This applies to managers as well, who see their value tied to a command-and-control structure.  Even in the face of studio extinction, these forces resist change.  I once heard the comparison of a manager resisting change in a failing studio to that of the Titanic passenger with the finest cabin refusing to evacuate!

Resistance comes from developers as well, who focus on their tasks and discipline and leave accountability to their bosses.  This feels safe, especially in a culture that hands out blame like candy during Halloween.

Valve had the benefit of being founded with the principle above and maintaining it.  It doesn't hurt that Valve is self-funded and somewhat isolated from external customers.  It also doesn't hurt that they own their I.P.  But this doesn't mean the path to self-organization is impossible to move to from a hierarchical culture.  It's definitely hard and it takes time.

There is a revolution taking place in how we work that may take a generation to become common place.  We have more examples every year that show us how to get there and what the benefits are.

Update 1/19/14 - Valve has since posted the handbook on their website and the link above reflects that.


Anonymous said...

Great post Clinton!

We often struggle with how to go down this patch because we have set deadlines with our projects and clients. It's much more difficult to let teams self form when they HAVE to have X content and be delivered by Y date. We pretty much have to assign a set number of headcount to a project to hit our goals. With in those constraints do you see a way to adopt parts of that self forming mindset?

Clinton Keith said...

Hi Anon,


X scope, Y schedule and Z people, all fixed, is a classic formula for project failure. What your describing is fixing the three sides of the project management triangle and the fundamental rule is that you can fix one or two sides at best. Fixing all three might work for manufacturing, but not games.

Game dev lends itself to variable scope best to achieve schedule or cost targets because scope is the easiest and best to refine over the project.

Management will often insist on fixed scope not because it's necessary, but because they feel it's needed to put pressure on developers and create extrinsic motivators. In creative work, such motivators don't work as well as intrinsic motivators.

Self-organization leads to instinsic motivation, improved environments and subsequently higher productivity. This is proven (read Dan Pink's book "Drive"). Enlightened management is the challenge.

So I'd suggest the organization has to address this first before even thinking about self-organization.

Good luck,

Anonymous said...

Hi Clinton,

I totally agree with your sentiments. The underlying point is we all need to be held accountable, learn and improve, in other words behave in a emotionally grown up manner.

Unfortunately most people in business behave in an emotionally childish manner.

Changing people emotionally to embrace the responsibility and success / failure culture is often the greatest challenge to overcome.

Clinton Keith said...

Hi Nick,

Thanks for the comment. I completely agree!


Agile Master said...

I have seen cultural resistance to be less of a problem then the inertia of people. If someone has been doing a things for long long time she is likely to resist more then someone who did it for few months.

A side joke we used to say in our last company - anything more then 5 years in software is a furniture :)

Jokes apart, it does matter that people need to realize they are becoming furniture not a value add and it takes time.

Thanks for nice article.

Clinton Keith said...

Thanks for the insight.

Is such inertia an element of the organization's culture as well? Simply "doing the same thing" for years will create an inertia within the organization (defined as the group of people belonging to it) and resist change.

Cultures change simply through inertia. Individuals are much more malleable. Just see how quickly most people adapt when they move to a new studio!


Tobias Mayer said...

Just found this article, following a tweet by @alhui. Loved it. Favorite moment:

"Resistance comes from developers as well, who focus on their tasks and discipline and leave accountability to their bosses. This feels safe, especially in a culture that hands out blame like candy during Halloween."

Apart from the LOL factor in the last few words, this is a critical thing to be aware of. Too often we blame "management" for holding on to old ideas. As often—perhaps even more often—there is as much fear from the people doing the work. We cling to safety, to what we know, even when it hurts, simply because there is potentially greater pain doing something different. This is where risk and courage play in, and why change needs to happen incrementally, and not overwhelm people.

Sebastian Juarez said...

great post

Jeremy Murray-Wakefield said...

In large corporate environments, people like to "leave themselves at home" and stick to their prescribed "roles and responsibilities" as it minimises the risk of blame.

When adopting agile, this is one of the hardest hurdles to cross. In the larger corporate environments, when you explain what "cross-functional" really means, people are incredulous and find it hard to believe such cultures even exist.

One said: "I could never ask my developers to help with testing - they would be disgusted..."

xmodgames latest version said...

thanks for the post

Dom Kudli said...

Agile is definetely better than waterfall but it has some challanges in the terms of emotions in people. but I think you can overcome it but constant pressure. As with everything change is hard but by constant reminders they can learn.

Clinton Keith said...

Agreed. There is a lot of "bad agile" out there. It works to just talk about how we can improve with those values (and our own) in mind and worry less about the labels on it.