CRITICAL STAGE ANALYSIS

1
CRITICAL STAGE ANALYSIS

• Integrating CSA into the game development process

2
What is Critical Stage Analysis?

• Simple mid-project feedback
• Designed to get feedback from all team members
• Requires team to take responsibility for cited
  problems ASAP
• Intended to improve game quality and reduce
  crunch time by fixing problems early

3
The Origin of the Process

• Wolfgang Hamann of Radical Entertainment
• Article published in the July 17, 2003 issue of
  Game Developers Magazine (available at
  gamasutra.com)
• Tutorial session at 2003 GDC (slides are archived
  at the GDC website)

4
Wolfgangs Theory

• We plan for our game production process to be
  very linear, but it never really is.
• Were unprepared for the loops were thrown for,
  and we have no tools to help us see whats coming
  in advance.
• CSA adds a feedback loop that allows us to see
  our true production path more clearly and be
  better prepared.

5
Who am I to talk to you about Critical Stage
Analysis?

• Im just a producer using the process
• Amaze Entertainment small Austin studio
• Handheld projects on Nintendo DS
• Licensed properties, 3rd party developer
• Small teams, about 15 people or less
• About a year of development time (or less!)

6
CSA A couple of caveats

• Hamann thinks postmortems should be scrapped I
  disagree.
• The literature presents this as a very simple
  process and while the doing a survey is simple,
  executing solutions for problems identified
  mid-project is not.

7
Surveying the team

• The big three questions, with answers ranked in
  order of importance
• Name five things that went well during the stage
• Name five things that went wrong during the stage
• Name five things that could be improved for the
  future.
• The rating process is considered critical.

8
Analyze data and deal with problems

• Data should be gathered and distributed to all
  team leads.
• Team leads should meet to analyze the data,
  determine solutions, assign ownership.
• Data and solutions should be presented to the
  entire team for discussion.
• Everything should happen within one week of the
  milestone or stage that has just past.

9
The Ideal Timeline

• Survey made available no more than three days
  after a milestone.
• Team members will spend about 10-15 minutes
  answering survey.
• Information should be compiled and distributed to
  leads no more than two days after it is gathered,
  and should be discussed immediately after
  receipt.
• Discuss the results in the next team meeting,
  within the week.

10
Tips and tricks

• Give team members multiple ways to fill out
  surveys.
• In team meetings, bring up remaining issues from
  the last CSA first.
• Lead with the positives.
• Always deal openly and honestly about all issues
  dont hide things from the team.
• Unsolvable problems should be broached.
• Unedited CSAs should remain accessible.
• Follow up is essential.

11
Measures of success

• Fewer issues are raised repeatedly
• Fewer things ranked with high importance over the
  course of the project

12
What I wanted out of CSA

• Feedback before the game was over
• Feedback from the quietest team members
• Feedback on both big and little issues
• A new communication tool
• Morale measurement
• A chance for team members to bestow accolades
  upon one another.

13
Things I changed

• I added a couple more questions, essentially to
  help get a measure of morale
• How satisified were you with the look and feel of
  this milestone, and why?
• How satisfied were you with the deliverable we
  sent overall, and why?

14
Things I changed

• Everybody was required to fill out the survey in
  the same format.
• Surveymonkey.com
• Zoomerang.com
• I didnt follow the suggested timeline.

15
My CSA Timeline

• Monday after milestone send out CSA survey to
  team. Deadline to fill out noon on Thursday.
• Wednesday afternoon send out reminder to do
  survey.
• Thursday afternoon producer compiles data and
  sends to all leads and directors.
• Friday morning meeting with producer, leads and
  directors to determine what kind of solutions can
  be offered for all problems identified.
• Friday afternoon producer creates a nifty CSA
  presentation
• The following weeks team meeting team
  participates in a CSA presentation

16
Things Ive learned

• The CSA process has been a great thing most of
  the time.
• People did, indeed, laud their coworkers.
• We did, indeed, identify problems we were able to
  solve earlier in the process.
• I started getting a whole new level of feedback
  from team members about the project both macro
  and micro issues.
• Some project teams have become more emotionally
  invested in the process than others.
• Its very hard to keep things anonymous.

17
Things Ive learned

• The producer has a huge influence on gets
  discussed in the CSA meeting.
• Its difficult to tell what the rankings mean.
• The two questions about what went wrong and what
  could be improved basically always ended up being
  10 things that went wrong.
• People who dont communicate well in meetings
  dont always communicate well on the surveys
  either.
• Its hard to give people unlimited ways to
  participate.
• It takes a lot of time.

18
Things Ive learned

• People may wait until a CSA survey to present a
  problem, and thats a problem.
• CSA results really give you a leg up when youre
  getting ready for a postmortem!
• Opposing viewpoints come up all the time thus,
  you cant make everybody happy.
• Some team members may be triggered in unexpected
  ways by the process.

19
But the most important thing is

• Doing a bunch of surveys doesnt do a thing
  unless you as a team commit to fixing the issues
  that are raised by them.
• Team members need to feel that team leads and
  directors have got their back, and are willing to
  make changes to the production schedule, game
  design and perhaps even the staffing and budget
  plan if an important problem sited needs to be
  solved.

20
Future Plans

• Start an online repository of CSAs to be used
  company-wide.
• Ask even fewer questions.
• Ask contract vendors to participate.
• Get the client involved.

21
Thats it!

• Questions? Comments?
• Do you have a CSA-type process youve used? How
  does it work for you?