Monday, November 02, 2015

Device Research, the Agile Way

New device development doesn't always start out with a single clean vision of the final product.  Often the product is a bit fuzzy because the knowledge of all the capabilities is uncertain.  There can be different visions fragmented by the different domains.  For example, marketing's vision isn't the same as the vision from software engineering, which is different from electrical/mechanical engineering's vision.  There are unknown areas of overlap as well as areas of non-overlap.

Still, there are questions about the overall vision that have to guide the product's R&D.  At first they can be expressed as questions that research seeks to answer:
  • Can we put the capabilities into a small enough package to be marketable?
  • Will the necessary processing power allow our cost, battery duration and heat dissipation to be below an acceptable level?
  • Etc...
There are many mutually dependent questions that need to be answered and some are critical.  How many products have you seen that have failed because, although they may have done most things right, they did a few crucial things wrong (like battery life).

Unfortunately for hardware-based products, we usually can't iterate rapidly on the entire product from the start, at least not well enough to discover these issues.  We still want a cross-discipline approach to our vision, even if  development doesn't support it.

Consider a simple scenario:  We have a set of new technology that we want to leverage into a
new product, for example the first generation iPod.  A key technical development that allowed the iPod was the famous "click wheel".  The click wheel allowed for a tactile intuitive user interface.  It was a big part of the iPod's success, but not the only part.  The design aesthetic, storage space and battery life were all part of the device's success.

(note: although Apple has been a client of mine, I did not work with the iPod team or know anything about the iPod development.  This example is speculative or based on published descriptions from employees).

Before the iPod, the market for mp3 players was saturated by hard-to-use, cheap players.  The vision for the iPod started by addressing what the current market lacked.  So the team explored design aesthetics, batteries, small, high-capacity storage devices and interfaces.  All of these areas of exploration overlapped with the vision of a small, easy-to-use player that could store many songs and which the user would be proud to own.

There was a certain amount of research that went into exploring each area of the iPod.  1.8 inch hard drives had been out for awhile, but newer 1-inch drives showed promise eventually.  Cost and capacity factors led to the 1.8-inch drives being chosen.  This had an impact on all the other areas.

So how do we work with separate groups researching separate areas of a new device, when it's too early to precisely define the device and impractical to iterate on a nearly-shippable version?

Can Agile/Lean Be Used?

Agile & lean practices are designed to explore emerging new products.  They aren't restricted to software products.  Its benefits can be applied for research as well.  However, implementations of agile for software development focus on a few areas that might not be available to most new device developers:
  • We can't have "potentially shippable" versions of the device every 1-3 weeks.
  • We often don't have a clear vision of the device we want to build until we do some research.
  • Stakeholders can be very nervous that research is open-ended and want detailed plans.
  • Researchers have trouble fitting their efforts into 1-3 week time-boxes that produce something that meets a "definition of done".
The concerns raised about applying agile can come from both the stakeholders and researchers as well:
  • Stakeholders: We don't want to have open-ended research with no end in sight.  We want to use more traditional project management techniques to put limits on the cost and time spent.  We need more control on a day-to-day basis.
  • Researchers: We can't estimate iterations.  They are too short to produce any "valuable" result that meets any definition of done.
To overcome these limits and concerns, I list some proven tips for using agile for R&D work:

Align your vision with research goals

Research has to align with the ultimate product's vision.  But sometimes a single product's vision depends on the results of research.   How do we reconcile these mutually dependent things?

The vision for a new device can start with a set of capabilities in a concept that we assume will change as we learn more.  It's critical that the people in R&D have a shared broad vision of the product they are researching.  This is where chartering techniques can help.  These techniques help create a shared vision far more effectively than passing around a large document:
  • Building "look-and-feel" mock-up devices
  • Creating a hypothetical demo video of the future device
  • Short customer-oriented presentations
Check out some of the pitch videos made for crowd-sourcing campaigns.  Many of these show what their devices might look like and how they would be used so they  generate enough excitement to draw millions of dollars of funding.  Isn't that level of excitement as valuable for those making your product?

Use Spikes

Spikes are user stories that limit the amount of time spent working on them.  They also are meant to produce knowledge that can't be estimated in scope up front.

An effective way of using spikes is called "Hypothesis-Driven Development".  One template for for HDD spikes is:
An example of this is:

We believe that implementing algorithm X will result in sufficient accuracy.  We will know we succeeded when we get 95% accuracy from our test data in the lab.  We will give up when we spend a full sprint without seeing this accuracy improve beyond 50%".

Set-Based Design

Set-based design is a design approach for considering a wide range of separate domain research that have overlapping and non-overlapping areas:

The approach is to explore an emerging product vision by exploring the range of separate domains and areas they overlap (orange area).   The idea is for research activities to refine the entire domain and converge on the best-shared solution.   This is fundamentally different from "point-based design" where the solution is chosen up front and the domains are forced to implement that point.  Knowledge of what works and what doesn't usually emerges as deviations from the point-based plan and considered as negative impacts to cost or schedule.

For example, suppose the iPod team had decided that the first iPod would have a touch-screen driven interface with solid state memory.  That's a potentially better product than the first generation iPod (in fact it's what the iPod eventually became), but in 2001, due to the existing technology, the memory may have been limited and the touch screen too battery draining.  Having gone down the long path of designing this device, Apple might have released a compromised or much-delayed product.

The Cost of Set-based Design

Set-based design can cost more in the short-term, but it can save your product in the long term.  To illustrate, if we had several contenders for a technical solution--all with various risks and costs associated--how would we work?  If each took a month to evaluate, we could be pushing out the ship date by many more months.

The answer is to research the solutions in parallel and to focus on failing each fast.  For the example of a touch-screen vs. click wheel on the first iPod, we'd focus on the areas of risk first.  How does each feel?  What is the cost of implementing each?  What is the power consumption?  We'd try to get these answers addressed  before making any further decisions (an iPod example is the creation of dozens of prototype cases, which Steve Jobs would choose from).

This tactic of avoid decisions made without sufficient knowledge is referred to as "deferring solutions until the last responsible moment".  We make better decisions when we know more, but we don't want to be in manufacturing when we decide to change the case.

These days, with on-demand rapid prototyping, 3D printing, emulation, etc. we can shorten the experimental cycle on hardware dramatically allowing us to do set-based design far more effectively.

Simplified stage gates
Other Useful Practices

Stage Gates
A project to create and ship a new device will change states and practices as it progresses.  Work will transition from researching foundation forms and technologies to prototyping the whole device to designing the production flow and moving into production.  These are stages can't be combined into each iteration as well as software-only products can, but the traditional problems that stage-gate development encounters can be mitigated with lean practices (this will be addressed in future articles).

Critical Path Management
The emerging vision of the device and the knowledge of what's possible will lead to the identification of a series of dependent activities or goals that need to occur before the device is ready for the prototype of production stage.  Identifying these paths and focusing efforts on improving flow through them starts early.

Many advances in hardware development practices (such as 3D printing, field-programmable gate arrays, etc.) have allowed teams developing hardware to benefit from the practices that software developers have been exploring for over a decade.  While iterating on an electrical or mechanical feature isn't as rapid as recompiling code, it's allowing more and more iteration and exploration into making better products.

Wednesday, September 30, 2015

The evil of tracking tools in the sprint

After 8 years of training and coaching teams I've noticed some very obvious patterns and anti-paterns.  One of them is the impact of bringing tracking tools into the sprint.

These tools are primarily used to:
  • Build the sprint backlog
  • Update the estimates during the sprint
  • Display sprint backlog details during the daily scrum
  • Spit out the sprint burn down
I understand that some teams, that cannot avoid being distributed, want to use such tools, but I rarely see them as positive for collocated teams.  It's not the fault of the tools, but how they are used.

One of the primary aims of Scrum is the boost in productivity seen with teams that have a sense of ownership in how they run their sprints.  Teams with ownership are accountable for their commitment.  They enjoy their work more and they explore ways to improve.

Sprint tracking tools can get in the way of that by:
  • Not allowing an "all hands" approach to managing the sprint backlog: one mouse, one keyboard, one operator.
  • The data in the tool is only brought out once a day because there are not enough licenses, developers don't want to learn the tool, or the extra effort isn't worth it.
  • The team's ability to customize the artifacts (task board, burn down, etc) are limited by what the tool can do.
  • The metrics of the sprint (sprint velocity, burn down), and even individual developer progress can be monitored by management.   Guess what happens when a developer or team is asked why their burn down is not diagonal enough?
  • Making the daily standup a status reporting meeting by focusing on individuals.
Even if the tool is being used in benevolent ways, the team can suspect otherwise.  In organizations that are trying to grow trust, this can be a killer.

Also, teams need their sprint backlog to be radiated at them.  Tools refrigerate the backlog by storing it in some cloud.

When I bring this up, I often hear from the Scrum Master that the tool makes it easier to do their job; tracking the sprint and producing the burn down or that there is no wall space.  It's often news to such Scrum Masters that their role is not to be efficient creating the artifacts, but helping the team build ownership and trust...or just tracking down solutions, like a portable task board.  I don't blame them necessarily.  Adopting a tracking tool for sprints are often an interpretation of how Scrum should work in an amber or orange organization.  Teams in these orgs who are new to Scrum might even want tools in the sprint so that the "Scrum fad" has minimal impact on them.

The sprint backlog is owned by the developers on a team to help them organize and manage their shared commitment and forecast for achieving the sprint goal.  It serves no purpose external to the team.  

Courage, commitment, focus, openness and respect aren't always easy to instill, but it can start with a bit of tape and some index cards.

Thursday, August 13, 2015

Tips for the Daily Scrum

When visiting a studio, I'll often ask to observe the daily stand-ups meetings.  It's the best way to sense where a team is and how to focus coaching time for the visit.  An effective daily stand-up meeting is an energetic swarm of committed team members that need to quickly synchronize their efforts and identify anything that is threatening their shared goal.

Some daily stand-ups (or daily scrums), aren't like this.  They are monotonous, manager-led and pointless to the attendees.  Teams here lack any sense of commitment or ownership and they eventually question the merit of the practice and perhaps even abandon it.  Unfortunately, abandoning the stand-up doesn't improve their chances of building commitment or ownership, which are principles of Scrum.  So I've compiled a list of pointers on ways to improve the daily stand-up meeting and improve commitment and ownership.

The stand-up is for the team.  This meeting is for the team to synchronize their efforts towards achieving their goal, not just as a status report.  If their are any "management artifacts" that need to be created (such as an update to a tracking tool), it should be done outside the meeting.

Anti-Tip: A sprint's commitment is on quality, not quantity.  If a studio wants a predictable amount of garbage, then taking over and running sprint meetings for the team is the best way to do it.

The rules aren't fixed.  The "answer the three questions" rule you read about in books is only the starting place.  Teams add questions, change the format, and do anything necessary to address their needs.

Anti-Tip: Make Scrum a process by fixing all the practices.  This will ensure that continuous improvements will cease.

It's about the commitment to the goal, not just the tasks.  Task tracking is critical, but work will arise on a daily basis that wasn't considered during sprint planning.  Teams need to react to this new work and balance it daily against their goal.  They need to inform each other about new developments and threats to their goal.  They should care about accomplishing the tasks, but they should care about the goal more.

Anti-Tip: Force teams to use task tracking software.  They'll measure success by completing tasks, not adding value.

The ScrumMaster must know their role.  ScrumMasters do not run the daily stand-up.  They facilitate them.  The difference is that the rules for the stand-up come from the team and the ScrumMaster ensures that those rules are followed.  If the team has decided that being late to the stand-up is not acceptable, they'll likely have a rule in place to chastise those who are late.  Paying $1 or singing a song is common.  ScrumMasters must not let them off the hook. 

Anti-Tip: When you solve all of today's problems for the team, they'll expect you to solve tomorrow's problems as well.

Respect the time-box and start time.  Some of the most effective stand-ups I've seen were five-minute "hurricanes of energy and activity".  The team shared what they needed to share and went back to work.  Stand-ups that take longer than 15 minutes are just wrong and need to be fixed.  Also, don't waste time by having a variable start time (based on who shows up late).

Anti-Tip: Ignore the time-box and see who shows up on time (& vice-versa)

Prefer a physical task board.  This is not always possible to have, but here is no electronic replacement that is equivalent.  A physical task board with cards allows everyone on the team to handle the tasks simultaneously at the same location.  It encourages team ownership.

... & Avoid projectors.  This might seem a bit redundant given the last point, but projectors used in a stand-up meeting should be renamed "conversation killers".  Projectors usually involve a software tool that provides information in a linear, one-at-a-time way with someone, unfortunately often the ScrumMaster, at the keyboard/mouse controls.  If defeats the non-linear, swarming-on-the-goal, value of the stand-up.  Software tools are often a critical part of project planning and tracking, but they have no place in a co-located daily stand-up.

Anti-Tip: Usually tools allow only one operator (mouse/keyboard).  When there is one operator in the meeting, guess what happens to the engagement levels for everyone else?

It's owned by the team.  The team picks the format and time of day of the daily stand-up.  Since they own it, it should run the same way, regardless of whether the ScrumMaster is there or not.  One simple test I recommend to ScrumMasters is to occasionally hide around the time of the daily scrum and see what happens.  If the team meets and runs the stand-up the same way as when you are their, you are doing a good job.  If they don't meet or stand around wondering what to do, they have some things to fix.

Anti-Tip: Standing in a circle answering three questions is a secret formula which leads to the magical process fairy flying overhead, sprinkling productivity dust on people.  These practices have nothing to do with team engagement through ownership.

(this post was edited from an earlier post)

Friday, May 22, 2015

Level of Detail for Product Backlogs

Replacing a 300 page design document with a product backlog filled with thousands of small user stories doesn't provide much benefit.  Responding to uncertainty, emergence and change requires a plan which is flexible, communicates the game's vision and doesn't try to document uncertainty away.

Games need a product backlog with a couple of hundred stories at most.  Upon hearing this, a question often asked is, "how do we encompass the scope of a large game in just a couple hundred stories?

The Level-of-Detail Metaphor

A useful metaphor to answer this question is the "Level-of-Detail" (LoD) technique often used in video games.  This technique reduces the workload of a renderer by selecting one of several versions of an object to draw based on how far it is is from the camera.  In the figure on the right, a distant boat will first use the low-polygon model on the bottom.  As the boat approaches the camera, it switches to using the higher polygon (or triangle) models above.  Ideally, as the boat approaches, you won't notice the models changing (old-popping).

The reason for  using LoDs is that graphics engines and hardware have limits on the total number of polygons that can be rendered at 30 to 60 times a second.  If you want to have hundreds of boats shown in your game, using LoDs is a common way to render them.

LoDs for User Stories

The same approach is used for the product backlog.  The figure below shows some example LoDs for user stories:

Just as LoDs allow games to show a full, complex world on the screen, the LoD approach to product backlogs allows you to encompass the entire vision for the game in one view.  Highly detailed documents or backlogs don't allow you to "see the forest through the trees" and, as with a forest, are easier to get lost in.

Level of detail is increased through the practice of splitting stories.  You'll find some examples there.

Thursday, May 21, 2015

Acceptance Criteria: or How I Learned to Stop Worrying and Love Conditions of Satisfaction

"Of course, the whole point of a 
Doomsday Machine is lost, if you 
*keep* it a *secret*!"  
- Missing acceptance criteria?
A definition of done applies to all product backlog items, but sometimes you need to define a set of important requirements that are unique for individual stories before they are worked on.  These are considered “conditions of satisfaction” for the product owner, or acceptance criteria.

Example: As a player, I want to see a Message-of-the-Day (MOTD) from the developer so that I learn useful things about the game.

Some acceptance criteria for this story might be:
  • The MOTD window may only take half the screen.
  • It’s only shown the first time someone starts the game during a calendar day.
Keep in mind that acceptance criteria shouldn’t be a huge list of design specifications.  These are a limited set of specific “must do” criteria that the team needs to know up front.

Types of acceptance criteria:
  • Specific functionality known up front (above).
  • Special, non-functional requirements (e.g. memory or CPU performance limits), for example “The game should have ten of these effects running with less than 2 milliseconds per frame cost on the target platform”.
  • A set of expected behaviors for a subset of features, such as needing persistence in saved games or needing specific tests for online gameplay.   
  • Known areas that will be incomplete.  For example, if you’re implementing a jump feature, but don’t have enough animation support (and have failed to get more), an acceptance criteria might say “some animation transitions will pop”.
Good acceptance criteria will:
  • Have a clean pass/fail, testable state.
  • Won’t dictate implementation details.
Who and When?
Acceptance criteria can be added to a story any time before, and during, sprint planning.  Usually they’ll be added in release planning, grooming and sprint planning with greater detail added as the sprint work approaches.  The product owner doesn’t have to be the one to identify them all, but should be present when they are captured.  I prefer to have the team there as well asking questions and suggesting acceptance criteria along the way.  One useful approach is to have a member of QA or a developer capture any acceptance criteria during discussions of the story, especially in sprint planning.

The Point
Acceptance testing will focus the team on the outcome as much as the means of getting there.  It allows the business side to build trust with the team and to improve how they communicate their needs.  When trust exists, the business is less likely to worry about the sprint and interfere.

Wednesday, May 20, 2015

Help! My Scrum Master is Running the Sprint!

“Hi, I had a question.  Our Scrum Master is acting like a manager, running all the meetings and assigning work.  This is what they did before Scrum.  I thought Scrum meant the team did this stuff?” - A developer

The values of Scrum apply here.
Scrum! Scrum! Scrum! Scrum!

The sprint is the team’s commitment to a goal.  True commitment to the game and continuous improvement requires a level of ownership, otherwise it’s an environment of compliance and people don’t engage as well with that.

The barrier to allowing ownership is fear.  I often hear the micro-managing Scrum Master’s fear of what will happen if the team doesn’t follow their leadership.  "They might fail!", they cry.   There is often an underlying suspicion that others on the team are not competent because they'll make mistakes.   The news is that competence doesn't mean you don't make mistakes!

Allowing teams to make mistakes and safely fail from time-to-time requires trust and courage.  Trust requires respect.  A lack of trust and respect is a cultural root of evil.  When I see locked supply cabinets, dictated core hours or time-sheets, web filters, people referred to as “resources” etc. I know that the true adoption of Scrum will be a bit harder.

How can this be done?
Courage is required
Shifting from a mindset of individual compliance to team commitment requires courage all around.  The Scrum Master and the team must summon the courage to take chances and fail every once in awhile.  Emboldening the team to find this courage is more challenging than micromanaging via a task spreadsheet, but it’s also more fulfilling.  It doesn’t happen overnight, but trust and respect can be grown.

Scrum Masters can start with some simple things:
  1. Silence yourself! Stop doing all the talking.  Learn to allow awkward silences.  The team will fill the silence with their own ideas and solutions.
  2. Ask questions, even when you know the answer.  Teach them to catch fish rather than giving them fish to eat.
  3. Be courageous.  Demonstrate respect and protect the team.
The development team assumes a part of this challenge as well.  It’s often hard for them to shift from being assigned tasks to accepting responsibly for planning and executing their own work.  Respect and openness among each other is critical.

For example: if an artist is having problems getting code to work with a model, the entire sprint goal is threatened, but the team's focus on their shared goal should guide them to solve this problem on their own.   There is no need to blame the artist for not solving their own problem.  There is no need for someone wearing a manager’s hat to assign an engineer to fix the problem.

Does this mean managers/producers should be fired?
Don't expect leadership to
be overthrown, but leveraged
An underlying fear from managers or producers jumping into the Scrum Master role is that they should continue doing what they've been good at in the past or they'll be let go.

In the ten years that I have been training teams to use Scrum, I haven’t seen this happen.  I’m not saying it never will, but most stories I hear reflect my own as a manager in a studio that adopted Scrum.

I found that as teams became more self-organizing, there was a growing part of my time that needed to be filled with other things.  Eventually, about half my day was freed up by not micro-managing.  This wasn’t bad because the work that teams took from me was never the work I enjoyed doing.  So, I joined a team as their product owner and ended up enjoying the work far more as I was closer to the game and the people making it.

This might not be the path for every manager.  Agile should lead a studio to ask what practices and roles are benefitting their games and the people making them and what parts are not.  Again, this can threaten the status quo, which is why a lot of larger, established studios have a harder time adopting agile.

"Now I can make cool movies about
monkeys taking over the world!"
The bottom line is that these values have to win out for you to have full success with agile.

Tuesday, May 19, 2015

The Scrum Values

The Scrum Values

  • Focus. Because we focus on only a few things at a time, we work well together and produce excellent work. We deliver valuable items sooner.
  • Courage. Because we are not alone, we feel supported and have more resources at our disposal. This gives us the courage to undertake greater challenges.
  • Openness. As we work together, we practice expressing how we're doing, and what's in our way. We learn that it is good to express concerns, so that they can be addressed.
  • Commitment. Because we have great control over our own destiny, we become more committed to success.
  • Respect. As we work together, sharing successes and failures, we come to respect each other, and to help each other become worthy of respect.