Lecture 2B Experimental Methods for Business Strategy

1
Lecture 2BExperimental Methods for Business
Strategy

• In this session you will design a game on your
  own laptop and have your colleagues log on as
  subjects. I will then provide some guidance on
  how to analyze experimental data.

2
Designing an experiment that uses the extensive
or strategic form

• Open a browser and visit http//www.comlabgames.c
  om/
• Click Old discrete and Strategic Form Module
  http//www.comlabgames.com/tree/index.html
• Click Edit a Tree to design a game in extensive
  form or Edit a Matrix to design a game in
  strategic form.

3
The rudiments of constructing a simultaneous move
game

• The mechanics of designing your own two player
  simultaneous move game are easy
• Determine the dimensions of the matrix.
• Enumerate the strategies.
• Define the payoffs in the cells.
• Name the players.
• Give your game a title.
• Undo your work and revise your game.
• Save your game in a directory.

4
The rudiments of constructing an extensive form

• The mechanics of designing your own extensive
  form game are almost as easy
• Draw the moves of the players and nature.
• Label the moves and define the probabilities.
• Name the players and define the payoffs.
• Draw the information sets.
• Undo your work and revise your game.
• Save your game in a directory

5
Conducting an Experiment

• Disable all firewalls on your laptop. Otherwise
  your experimental subjects will be prevented from
  participating by the firewall.
• Use a wired connection to the internet to avoid
  congestion. If you use a wireless connection,
  your subjects may be disconnected while waiting
  to join your game.
• Open your game in the Comlabgames module and
  provide your subjects with your internet IP
  address and port number.

6
Analyzing the data

• The experimental results are automatically saved
  in the same directory as your game, and can be
  opened as an HTML file or in Excel.
• We now review methods for analyzing categorical
  data from finite games played in the extensive
  and strategic forms.
• We discuss measures for evaluating performance,
  ways of summarizing the data, and statistical
  methods for forming confidence intervals and
  testing hypotheses.
• Chapter 2 of Strategic Play provides a more
  detailed analysis.

7
Learning strategic behavior using experimental
methods

• Applying experimental methods is a self-contained
  training tool for learning strategic behavior
• Design a variety of games that capture parts of
  the strategic issue you are trying to understand.
• Conduct experiments with human subjects in the
  area using small monetary stakes as motivation.
  Your subjects do not need to have any training in
  game theory or experimental methods.
• Analyze the results from the experiments seeking
  behavioral patterns that might apply in real life.

8
Four advantages of experimental methods over
theory

• One way of formulating strategy is to find
  empirical behavioral patterns that emerge from
  repeatedly conducting experiments
• The game might be too complex to solve.
• The game might have multiple solutions.
• Experimental subjects and also managers sometimes
  make irrational decisions.
• Managers learn more from experience than theory,
  so managers in training might learn more quickly
  by artificially recreating strategic situations
  rather than theorizing about them.

9
Empirically optimal strategiesfor strategic form
games

• Given the behavior of the subject population,
  what is optimal play?
• To answer this question for the row player in a
  strategic form game, we weight each cell payoff
  by the relative frequency its column was visited
  by the column player, and form the average payoff
  the row player would have received from playing
  any given strategy.
• The best reply to the empirical distribution
  generated by the column players maximizes the
  average payoff calculated in this fashion.

10
Empirically optimal strategiesfor extensive form
games

• In the extensive form game, a similar approach is
  used to evaluate a any given move that taken be
  taken from a designated information set.
• First we compute the expected payoff from making
  a particular move from a given node, weighting
  the payoffs with their relative frequency of
  occurrence conditional on making that move.
• Then we calculate the relative frequency of
  arriving at any node belonging to the same
  information set.
• In this way we form the empirical expected payoff
  from making any given move from any given
  information set.

11
Two uses for the data

• The results from the experiment can be
  interpreted as a comment on the rationality of
  the subjects, and also the usefulness of the
  theory.
• Data from experiments can be used to
• Evaluate the performance of subjects, and thus
  link their incentives to play the game with the
  payoffs that face them in the game.
• Investigate whether subjects follow the
  predictions of theory, whether the empirically
  optimally strategies match the predictions, and
  whether different characteristics of subjects are
  significant.

12
An example
You may recall playing this game in the first
lecture
13
Expected value maximization

• This is an example of a game with perfect
  information.
• In such games each player sees exactly how the
  game progresses to her decision node. There are
  no dotted lines connecting decision nodes.
• Perfect information games can be readily solved
  if the players maximize their expected value ,
  and there are not too many nodes.

14
Solution

• Expected value maximizers use the principle of
  backwards induction to solve the problem
• At node 4, NATURE selects node 6 with probability
  0.5 and node 7 with probability 0.5. On reaching
  that node, the expected value for INNOVATOR is 5
  and the expected value for VENTURE CAPITALIST is
  6.
• Anticipating this, the VENTURE CAPITALIST would
  fund project at node 2 because 6 exceeds 5.
• At node 1, INNOVATOR will request funding
  because 5 is more than 2.
• So in the solution to this game the INNOVATOR
  requests funding and the VENTURE CAPITALIST will
  fund the project.

15
Conduct of the experiment

• 23 subjects from the 2003 undergraduate economics
  class participated in the experiment.
• No knowledge of game theory was explained to the
  subjects before the experiment. The solution of
  the game was not explained.
• The subjects were randomly assigned player roles
  upon logging on the game, and pairs were
  anonymously matched.
• Subjects were told that they should play the game
  at least once, and were permitted to play more
  than once.
• 13 subjects played the game once. The remaining
  10 played it twice or three times.

16
Criteria for rewards and grading

• In this experiment subjects were told there were
  no rewards from playing the game well.
• An alternative scheme is to sum the points each
  player gets in total and pay them at the rate of
  a dollar a point.
• Or we get reward subjects by playing the game
  correctly. For example we could award each
  subject one point every time the game in which he
  is participating ends in the terminal node that
  is solution to the game, and zero otherwise, and
  sum the points for each subject and divide
  through by the number of games played.

17
List of subjects
18
Test results
How would subjects have fared under this
alternative scheme for awarding points?
19
Further notes on assessment

• Should we let the grade a particular subject
  receives be affected by other subjects behavior
  or chance?
• Under the alternative grading scheme in this
  example the VENTURE CAPITALIST is automatically
  punished if the INNOVATOR makes a mistake.
• In the alternative scheme, it is implicitly
  assumed that both players are net present value
  maximizers, although very risk averse subjects
  might rationally choose to pass on the project.
• If we do not make points proportional to the
  payoffs in the game, the game is being modified.

20
Trials and outcomes

• We now turn to the second use of the data, for
  analyzing the game and its solution.
• We could treat each game played by a pair of
  subjects as a trial, and the full history of play
  as an outcome.
• Alternatively we could treat each time a subject
  plays one game as a trial. Then his moves would
  be the trial outcome.
• In both cases the number of trials is the size of
  the sample.

21
Analyzing behavior

• The sample of trials and their associated
  outcomes is used as evidence to inform us about
  the underlying population pool from which the
  sample is drawn.
• We compare the predictions of the theory to the
  sample behavior observed, to see whether the
  theory applies to the underlying population.
• Similarly the behavior of different sub-samples
  are compared with each other, to see whether
  different types of subjects behave the same way
  or not.

22
Summary statistics
As a first cut a histogram shows the predicted
and observed outcomes
23
Other graphics

• Bar graphs, pie charts and Venn diagrams are also
  useful ways of graphically depicting the data.
• An advantage of the pie chart is that it
  automatically incorporates the normalization that
  the proportions implied by a partitioning must
  sum to one. If one of K outcomes occurs each
  trial, their relative frequencies are easy to
  read off a pie chart.
• A Venn diagram is quite useful in showing sets of
  outcomes that have nonempty intersections. For
  example we could illustrate the number of times
  the INNOVATOR maximized expected value, the
  number both players did, and the remaining times,
  when the INNOVATOR did not maximize expected
  value.

24
Statistical inference

• We might consider each outcome of a trial as a
  random draw from a probability distribution.
• The characteristics of the sample then provide us
  with information to estimate the parameters
  describing the distribution, and to test
  hypotheses of interests to us.
• In our example, we define each trial as a move by
  a subject and ask what is the probability that
  subjects maximize expected value.

25
Estimating the mean of a Bernoulli random
variable
26
Are there gender differences?

• The numbers in brackets predict the number who
  would have ended up on a terminal node if both
  genders behaved exactly the same way.
• For example, considering females who played in a
  game ending on node 5, note that 3 9 divided by
  39 times 13.
• Are the corresponding numbers in brackets
  statistically different from the actual outcomes?

27
Estimated expected cell frequency for testing
gender differences
28
Are juniors and seniors different?
The test statistic is 4.18, which is less than
the Chi-square critical value, for an 0.5 test
with 2 degrees of freedom, of 5.99. We cannot
reject the null hypothesis that juniors and
seniors are the same.
29
Lecture Summary

• We designed games in the extensive and strategic
  forms.
• We explained how to conduct experiments for
  analyzing strategic behavior with invited
  subjects who have internet access.
• We showed how to analyze categorical data
  generated by experimental sessions.
• Finally, we discussed the merits of using
  experimental methods to learn strategy.