Playtesting

1
Playtesting
2
Playtesting is not

• Focus groups gathering opinions on proposed
  game/feature ideas
• QA testing the hunt for bugs
• Usability testing is the user interface easy to
  understand and intuitive how difficult is it to
  learn how to play
• Playtesting does my game achieve its goal (very
  often, is it fun?)

3
Early and often

• The feedback from playtests is useless when it
  comes too late in the project to make changes
• Infrequent playtests generate less revealing and
  less important discoveries
• You always need to playtest more often than you
  think

4
Why playtest?

• Quote from Peter Molyneux
• (Fable, Black White, Populous)

5
Outside observers

• Whenever possible, hire outsiders to conduct your
  playtests
• 1gt Relieves playtesters from having to be
  tactful (you want brutal honesty)
• 2gt Keeps your from justifying and defending your
  work (they dont care)
• If not, at least invite playtesters to help you
  most by being critical

6
Outside observers

• If not outsiders, then conduct tests using those
  farthest from the work or youre asking for
  debates (you can even divide the game between
  project teams)

7
Why? We love our babies

• Its hard to watch someone cut up your baby, we
  defend it ruthlessly
• There are even social/psychological terms for
  this and other related phenomenon, such as sunk
  cost fallacy and loss aversion
• The more you invest, the more you irrationally
  protect the investment

8
Dollar Auction

• I am auctioning off a 100 bill
• It goes to the highest bidder, who simply pays
  their bid and receives the 100
• But the second highest bidder must also pay their
  highest bid, yet gets nothing
• ANY BIDS?

9
Environment of honesty

• Encourage your playtesters to be brutally honest
  and unforgiving in their comments (helps if they
  know they are not going to hurt your personal
  feelings)
• Sets an environment where they wont hold back
  nit-picky comments
• People dont like to be mean, you must give them
  permission / force them to

10
How to begin

• IDEAL say nothing and just let them play
• Concession explain the controls
• Concession explain the goal
• Anything else will completely subvert your
  attempt to see what the player can discern on
  their own
• Remember that you cant be there when they
  actually play your game

11
Dont taint the results

• Its hard NOT to subliminally tip your hand and
  ask leading questions
• The second level was pretty easy, right? Makes
  them embarrassed to say it was way too hard,
  theyll say it wasnt that easy
• How did you like the large open world?
• Large and open are positive adjectives, they
  think they are supposed to naturally agree
• Never make a statement with a question

12
Keep em separated

• If conducting long playtests where there are
  breaks or you have an open space, playtesters
  will want to chat with each other about the game.
  DONT LET THEM!
• You dont want to muddy someones perspective, or
  allow assertive testers to dominate other
  peoples opinions
• Unless your game RELIES on groups of people
  playing together in the same space, dont let
  playtesters mingle

13
Let them suffer

• Its irresistible to help a playtester figure it
  out, rather than sitting in agony as they become
  frustrated and hate your game
• You want to see the things they will try
• Ask them what they are thinking as they fail,
  what they THOUGHT should happen (then change your
  game to make them right next time, fail to
  success)

14
What is unclear?

• Find out what images, icons, text, missions,
  features, etc. are not clear
• If they dont use certain buttons, ask what they
  think would happen if they pressed them (based on
  the icons)
• Be invisible as much as possible, but actively
  solicit feedback when they DONT try things you
  need to be tested

15
They dont care

• Never explain or justify your decisions to the
  playtesters. Its instinctual when they
  criticize but you dont need to convince them
  your choices were wise, you just need their
  honest feedback
• The more you defend the less they attack, and you
  WANT the onslaught (joining in actually helps
  them speak out)

16
Target demographics

• Test the target thoroughly
• but also test outside to get
• things you hadnt realized
• Example Any user interface that can be
  understood by 40-50 year olds will be understood
  by 20-30 year olds. But the older testers
  provide great value in showing what is possibly
  on the fence for the younger testers

17
Get statistics

• Not just what doesnt work, but how many people
  did it not work for
• You can ignore problems that only strike a
  minimal percentage of testers and you MUST
  address problems that most of them had
• Breadth is more valuable than depth (more
  sessions with more people rather than longer
  sessions with less)

18
Statistics in game

• Where possible, log important events
• Its useful to know where your players are
  struggling, how long certain tasks take, etc.

19
Statistics in game

• Refer to Microsofts processes for a look at how
  detailed statistics, and filtering thereof,
  benefits trouble spots
• WIRED article
• MS playtest from tester perspective
• GDC Presentation

20
Kleenex testers

• You want lots of fresh testers Kleenex testers
  who have never played your game
• Getting testers through the first five minutes of
  your game is the toughest part
• Experts in your genre are less valuable since
  they know the territory and guess well when in
  doubt (cant rely on them)

21
Recurring testers

• Kleenex testers show you where your game should
  improve for first-timers
• Recurring testers are required to discover
  replayability and when your game tires
• Crucial for longer game experiences, particularly
  for MMOs

22
Prepare questions ahead

• Do NOT improvise a post-play interview
• Follow surprises and interestingly unique aspects
  of a playtesters experience
• But have your crucial questions prepared in order
  to make sure they are included

23

• Most important question to ask
• What are the three worst
• things about this experience?