I came across the following great article:
This is an entire point that I forgot to talk about: Embedded QA.
As I mentioned at GDC, "thinking Agile" is really hard to adjust to. Our migration of QA is a perfect example of how we struggle with it.
Traditional QA is ramped up during post-Alpha in a traditional waterfall approach to vigorously test the game once all the features are implemented. We've all seen the problems in games where Alpha is the first real chance we have to play the game in some form near it's final state. This often raises the desire to change things even more. This is and example where discovering product value at the end of the project is bad.
Additionally, we underused QA. Good QA people are either frustrated at not having their insights into a game valued ("it's too late to change the game based on your suggestions") or their role is so undervalued in some companies that people are literally hired off the street to fill the role.
Additionally, we underused QA. Good QA people are either frustrated at not having their insights into a game valued ("it's too late to change the game based on your suggestions") or their role is so undervalued in some companies that people are literally hired off the street to fill the role.
We had been doing Scrum for ~ six months before it even occurred to us to reconsider QA. Scrum is about mixing things up. The word is taken from Rugby where the team moves the ball as a group, and doesn't divide up the team into roles as strictly as other sports teams. So why were we separating QA into a separate department that doesn't Scrum with us when our monthly goal was to produce a vertical and complete slice of the game? I can be pretty slow at times, but I eventually "get it".
So we started embedding individuals from QA into the Scrum team. They sat in the same bullpen as the rest of the team. They got their hands a lot more dirty. Some of them could do a bit of coding, others layout and others had some part time associate producer roles, but their core job was to excercise functionality as it came online. It was a huge success.
Our goal with Scrum is to eliminate Alpha/Beta/QA departments, etc (Although we may need a "stabilization Sprint" thrown in to have proper releases). Darkwatch had a rather extended alpha-beta period (having to go through a management buy-out of the studio and shop the game didn't help), so I can't claim we're there yet. Some of our largest remaining problems are:
- Interteam build stability : How to prevent separate teams from harming each other.
- Interteam dependencies : How teams can better support each other.
- Zero bug tolerance.
- Improved automated testing - We implemented this very late in the project.
- Improved handling of bad date in a data driven engine.
More topics to talk about...