Experimentelle Wirtschaftsforschung

1
Experimentelle Wirtschaftsforschung

• Armin Falk
• Konstanze Albrecht
• Steffen Altmann
• Matthias Wibral

2
Roadmap

• Why experiments?
• Advantages and Objectives
• Limits and objections
• How to do an experiment?
• Designing an experiment
• Conducting an experiment
• Data analysis
• How to write a paper?

3
Data Analysis

• Mann-Whitney U-test
• Wilcoxon test
• Regressions

4
U-test

• Many different names
• Mann-Whitney test
• Mann-Whitney U-test
• Mann-Whitney two-sample statistic
• Wilcoxon-Mann-Whitney test
• Wilcoxon-Mann-Whitney U-test
• Rank-sum test
• Wilcoxon rank-sum test
• Two-sample Wilcoxon rank-sum test
• U-test

5
When can/should we use a U-test

• Question
• Are two random samples drawn from the same
  population?
• A.k.a. Is there a treatment effect?
• Independent observations
• Between subject design
• We come back to this in a minute

6
When can/should we use a U-test II

• Data has to be at least ordinal
• Ordinal we can use lt/gt(e.g. grades)
• Interval we can use lt/gt//- (e.g. temperature in
  C)
• Ratio we can use lt/gt//-/// (e.g. length)
• Not nominal (e.g., colors)
• No assumptions about error distribution, its a
  non-parametric test

7
How is a U-test done for small samples (n2lt20)?

• Count number of cases in the smaller (n1) and
  larger sample (n2)
• Do one ranking for both samples together
• Smallest value gets rank 1
• Ties get the average rank
• Calculate U For each value of the smaller
  sample count how many values of the larger
  sample are ahead in the ranking sum this up
  this is U
• Look up corresponding p-value in a table (if U
  too large U n1n2 U)

8
Table with (one-sided) p-values of U-test (n24
or n25)
p-values are for one-sided tests for two-sided
tests, p-value has to be doubled (source Siegel
2001)
9
Example A Dictator Game

• Dictator game, subjects can give between 0 and 15
• Treatments
• D Game is called dictator game
• G Game is called giving game
• Question Is giving different between treatments?
• Distribution of giving?
• Null hypothesis Giving does not differ between
  treatments, i.e.
• H0 DistributionD DistributionG
• Observations (5 subjects each)
• D 1, 2, 0, 6, 4
• G 12, 5, 7, 8, 15

10
Example

• TreatmentG is a treatment dummy (0 if treatment
  D, 1 if treatment G)
• Count number of observations n1 n2 5

11
Example (II)

• Do one ranking for both samples (ascending)

12
Example (III)

• Do one ranking for both samples (ascending)
• For each value of the smaller sample count how
  many values of the larger sample are ahead in the
  ranking
• Sum this up to get U 24

13
Example (IV)

• U 24
• Look up corresponding p-value in a table (if U
  too large U n1n2 U 5 5 24 1)
• P-value of 1-sided test 0.008
• Remember H0 DistributionD DistributionG
• P-value of 2-sided test 0.0082 0.016
• ?Hypothesis that giving does not differ between
  treatments can be rejected on 5 confidence
  level

14
How is a U-test done for larger samples (n2gt20)?

• Count number of cases in the smaller (n1) and
  larger sample (n2)
• Do one ranking for both samples together
• Smallest value gets rank 1
• Ties get the average rank
• Add rankings for each sample (R1 and R2)
• then U n1n2 0.5n1(n11) R1
• or U n1n2 0.5n2(n21) R2

15
How is a U-test done for larger samples (n2gt20)?
(II)

• If n2gt20, U is distributed approximately normal
• Calculate p-value
• Mean mu 0.5n1n2
• S.d. sigma sqrt((1/12)n1n2(n1n21))
• z (U mu)/sigma
• Look up p-value in a table of normal distribution
• For your empirical paper its ok to use this
  approximation already for n2gt9

16
U-test on the Internet

• http//elegans.swmed.edu/leon/stats/utest.html

17
U-test in Stata

• Input data
• Make a variable TreatmentG
• Make a variable giving
• ranksum giving, by(TreatmentG)

18
When are observations not independent and how
can we use a U-test anyway?

• Repeated game (partner matching)
• Averages for (fixed) groups over all periods are
  the independent observations
• For first period, individual observations are
  still independent
• Interactive game, subjects are matched with
  different other subjects in different periods
  (i.e. stranger matching)
• Every matching group is one independent
  observation, take average over all subjects and
  all periods in one matching group

19
When are observations not independent and how
can we use a U-test anyway? (II)

• Game with perfect stranger matching
• Average for each individual subject over all
  periods is one independent observation
• One-shot game
• Individual subject is one independent observation
• Digression The role of feedback / information
• One can argue that in a repeated game (stranger /
  partner matching) where subjects do not receive
  feedback on others behavior (and on own and
  others interim payoffs), each subject is an
  independent observation

20
Data Analysis

• Mann-Whitney U-test
• Wilcoxon test
• Regressions

21
Wilcoxon test

• Different names
• Wilcoxon test
• Wilcoxon T-test
• Signed-rank test
• Signrank test
• Wilcoxon signed-rank test
• Wilcoxon signrank test for dependent pairs
• Wilcoxon matched-pairs signed-ranks test

22
Wilcoxon Signed Rank Test

• 2 Treatments Within-Subject
• H0 treatments are the same
• Procedure
• Take differences
• Sort differences according to size of difference
• Allocate ranks (smallest Diff. ? rank 1)
• Sum of ranks for positive differences determines
  T
• Evaluate p-value for T (Table)
• H0 can be rejected on 5 level if p 0.05
• H0 cannot be rejected if p gt 0.05

23
When can/should we use a Wilcoxon test?

• Pairs of observations
• I.e., every observation is actually two
• Examples
• Length of right and left arm
• Behavior in first and last period of a subject
• Strategy method
• Within subject design behavior in treatment 1
  and behavior in treatment 2

24
When can/should we use a Wilcoxon test? (II)

• Data has to be more than ordinal but less than
  interval
• We have to be able rank differences in pairs
• Interval/ratio data is also ok
• No assumptions about error distribution,
  non-parametric test

25
How is a Wilcoxon test done for samples Nlt25?

• Calculate for each pair the signed distance d
• Drop all observations with d0
• N is number of observations with dgt0
• Rank all ds (without regard of the sign)
• Smallest value gets rank 1
• Ties get the average rank
• Calculate T as sum of all ranks of one sign (T is
  the smaller sum of the two possible sums)
• Look up corresponding p-value in a table (T has
  to be smaller than the given number

26
Table with p-values for Wilcoxon test
27
Example

• Dictator game, subjects can give between 0 and 15
• Treatments (within subject design)
• Y Recipient is young
• O Recipient is old
• Question Is giving different acc. to recipient?
• H0 Giving does not differ acc. To recepient
• Observations (8 subjects) (Y, O)
• (3,6) (4,9) (15,14) (12,3) (1,7) (4,5)
  (10,6) (3,2)

28
Solution example

• Calculate differences and absolute value of
  difference

29
Solution example (II)

• Rank data by absolute difference (ascending)
• Calculate sum of ranks for one sign (T smaller
  sum, here positive ranks)
• T17
• P0.945
• H0 (Giving does not differ) cannot be rejected

30
How is a Wilcoxon test done for samples Ngt25?

• Calculate for each pair the signed distance d
• Drop all observations with d0
• N is number of observations with dgt0
• Rank all ds (without regard of the sign)
• Ties get the average rank
• Calculate T as sum of all ranks of one sign (T is
  the smaller sum of the two possible sume)

31
How is a Wilcoxon test done for samples Ngt25?

• If Ngt25, T is distributed approximately normal
• Calculate p-value
• Mean mu 0.25N(N1)
• S.d. sigma sqrt((1/24)N(N1)(2N1))
• z (T mu)/sigma
• Look up p-value in a table of normal distribution

32
Wilcoxon test on the Internet

• http//www.fon.hum.uva.nl/Service/Statistics/Signe
  d_Rank_Test.html

33
Wilcoxon test in Stata

• Input data
• Two variables per observation
• signrank YO

34
Data Analysis

• Mann-Whitney U-test
• Wilcoxon test
• Regressions

35
Regressions

• Different names
• Regression
• OLS regression
• ordinary least squares
• OLS
• Method of least squares
• Multivariate regression

36
When can/should we do a regression?

• More assumptions
• Independent observations
• Normally distributed error term
• Gauss-Markov assumptions
• Main advantage
• Possibility to control for other variables
• Higher efficiency (U-test 95 of t-test)

37
Regressions in Stata

• Example Giving Game vs. Dictator Game (U-test)
• regress giving TreatmentG
• giving is variable containing the giving info.
• TreatmentG is variable containing the treatment
  info (i.e., explanatory / exogenous variable).
• Significance tested by t-test
• Example Giving acc. to recipient (Wilcoxon test)
• Observations not independent!
• Cannot use OLS

38
Regressions in Stata (II)

• Repeated individual decision (giving)
• Data is dependent within each person (person)
• But independent across persons
• regress giving treat, cluster(person)
• Matching groups (action)
• Data is dependent within each MG (group)
• But independent across matching groups
• regress action treat, cluster(group)

39
Example

• Dictator game, giving between 0 and 15
• Every subject gives repeatedly
• Subjects (dictators) are both male and female
• Treatments
• M all recipients are male
• F all recipients are female

40
Example (II)

• Treatment M
• Treatment F
• Female Gender of dictator
• How to analyze data of repeated game?

41
Example (III)

• Every subject in each period treated as
  independent observation (which they are not)
• regress giving TreatmentF
• ranksum giving, by(TreatmentF)
• Taking dependence on dictator-level into account
• regress giving TreatmentF, cluster(Subject)
• ranksum mean_giving, by(TreatmentF)
• bysort Subject egen mean_givingmean(giving)
• Analyzing additional control variables (dictator
  gender)
• regress giving TreatmentF female, cluster(person)

42
Roadmap

• Why experiments?
• Advantages and Objectives
• Limits and objections
• How to do an experiment?
• Designing an experiment
• Conducting an experiment
• Data analysis
• How to write a paper?

43
How to structure a paper about an experiment

• Title page
• Title
• Authors
• Abstract (short summary, ca. 100 words)
• Sections (chapters of the paper)
• Introduction
• Design Predictions
• Results
• Conclusions
• References

44
How to structure a paper about an experiment II

• Introduction
• Introduction (whats this about)
• Research question (In this paper, we ) as
  early as possible
• Design (no p-values)
• Results
• Interpretation
• Related literature
• Readers should be able to stop after the
  introduction and know the important things
• Rest of the paper only shows that introduction
  was correct

45
How to structure a paper about an experiment III

• Design Section
• Section title is usually Experimental design
  although that is grammatically not correct
• General design (incl. payoff table)
• Treatments (additional control treatments can be
  mentioned later)
• If there is a crucial design detail, explain it
• But you dont have to explain why you chose every
  parameter the way you did
• Predictions (or as separate chapter)
• Standard prediction (rational selfish) SGPNE
• Alternative hypothesis/hypotheses

46
How to structure a paper about an experiment IV

• Design
• Instructions as appendix (or available upon
  request)
• Procedural details at the end of design
• subjects
• information about subjects undergrad? which
  university? which fields?
• average earnings
• duration of experiment
• ztree disclaimer
• etc.

47
How to structure a paper about an experiment V

• Results
• Do formulate explicit results (Result 1 )
• Usually, every result needs a statistical test of
  the hypothesis (i.e., p-value)
• Keep this in mind when designing the experiment
• State which test you use
• Short conclusion
• The appendix (Blinddarm) of economics papers not
  needed anymore, will probably die out
• Abstract again main design idea, main results
• Caveats
• More interpretation possible extensions

48
Frontloading

• Use a newspaper style, not a joke style
• Important things first, then the details
• A joke works the other way round
• Suspense is not necessary

49
Present or past tense?

• Design past tense
• Predictions present tense/future
• Results present tense or past tense
• Present tense makes results seem more general
• But sometimes akward

50
Layout

• No bold text
• Italics only rarely if it helps understand a
  sentence
• Use as few footnotes as possible
• Footnotes destroy readability of the paper
• If something is important, mention it in the text
  body
• If it is not important, its not important

51
Citations

• Thoughts which someone else already brought
  forward or findings already found somewhere else
  have to be cited
• Usually, the main thought is cited, not the
  literal statement, i.e.
• Fehr (1999) has argued that sanctions enforce
  norms instead of
• Fehr (1999) wrote sanctions enforce norms
• Citations in the text (NOT in footnotes)
• Fehr (1999) shows that
• It has been argued that sanctions enforce norms
  (Fehr 1999)

52
Figures and tables

• Figures have to be readable if printed in black
  and white
• Notes below figure/table must explain details,
  interpretation is done in the text

53
Useful books for next Christmas

• Deirdre McCloskey Rhetoric of Economics
• William Zinsser On Writing Well