Theory testing and internal validity

1
Theory testing and internal validity
2
Theory testing or phenomena discovery

• Approach explaining data
• Looking at data, discovering a pattern and
  constructing a theory to explain it
  
• Such a theory can always be constructed
• Solution
• Validity of this theory must be tested on new
  grounds
  
• Prediction criterion
• Dont let the data tell the story
• Theory driven experimental design

3

• Approach theory driven experimental design
• Field anecdotes or casual observations generates
  priors in a theory
  
• Hypotheses developed based on theory
• Reject theory or not?
• Bayesian approach
• If reject look for auxiliary factors
• If not reject be content
• Priors have no scientific foundation
• Solution
• Let previous phenomena generate priors
• If not reject be sure that internal validity
  has been achieved
  
• Ceteris paribus
• Control of auxiliary (confounding) factors

4
Trust game

• Observed propensity to Roll is increasing in
  Bs beliefs about As expectations
  
• Prop(R)g(tB) g0
• sBf(tB) f0
• Internal validity
• Truth telling incentive compatibility of belief
  elicitation mechanism
  
• Confounds by auxiliary factors that are
  correlated with beliefs

5
Internal validity

• We have appropriately modeled the Ceteris Paribus
  assumption
  
• Incentive compatible belief elicitation
• Insurance motive
• If B believes that A has a high probability on
  the event that B will Roll
  
• If he truth tells his payoffs will be
• 10 or 14 minus guilt plus payoffs from accurate
  beliefs if he is right
  
• 5 if he is wrong
• If he understates his beliefs his payoffs will
  be
  
• 10 or 14 minus guilt if he is right
• 5 plus payoffs from accurate beliefs if he is
  wrong
  

6
Auxiliary factors

• Psychological values
• Honesty, sincerity, integrity
• Social preferences implying Roll
• Efficiency
• Distribution
• Cognitive capital
• Perception of the game
• The danger of testing for the effect of one
  individual characteristic at a time
  
• Priors based on field anecdotes, personal
  experiences, and introspection in favor of guilt
  over other possibilities