PS 0700 RESEARCH METHODS IN POLITICAL SCIENCE

1
PS 0700 RESEARCH METHODS IN POLITICAL SCIENCE

• UNIT 2 FUNDAMENTALS OF EMPIRICAL INQUIRY

2
1. SPECIFYING THE RESEARCH QUESTION

• WHY? Why are Some Bureaucratic Agencies More
  Responsive to Political Influence than Others?
• HOW? How does Information Flow Through the Mass
  Public?
• WHEN? Does the Timing of Position Taking on
  Legislation Affect Material Benefits Obtained by
  MCs

3
1. SPECIFYING THE RESEARCH QUESTION

• Common Features
• Grounded in Empirical Observation(s)
• Falsification is a Must
• Aim is to Make Empirical Generalizations

4
1. SPECIFYING THE RESEARCH QUESTION

• The Research Question Must Be Compelling That
  is, find an Object (i.e., Phenomenon) of Broad
  Interest that you seek to Better Understand.
  Explanadum/Dependent Variable

5
1. SPECIFYING THE RESEARCH QUESTION

• Arrive at a Plausible Explanation of the Object
  of Interest. Explanan/ Independent Variable
• Test your explanation against conventional wisdom
  and/or alternative explanations for your Object
  of Interest.

6
1. SPECIFYING THE RESEARCH QUESTION

• PASSING THE SO WHAT? TEST
• Does it contribute to the cumulative knowledge on
  the Object of Interest?
• Does it Yield Any Theoretical Advances/
  Implications?
• Does it Yield Any Practical/Substantive
  Implications?

7
2. PROPOSING CAUSAL EXPLANATIONS

• Does X (Independent variable)
• ? (Cause)
• Y (Dependent variable)
• Antecedent Variable Causally Prior to the
  Independent Variable
• e.g. Religious Attitudes ? Attitudes on Abortion
  ? Presidential Vote Choice

8
2. PROPOSING CAUSAL EXPLANATIONS

• Intervening Variable Causally Between the
  Independent and Dependent Variables i.e.,
  Conditioning or Contextual Factors
• e.g. Presidential Election Year
• ?
• Divided Government
• ?
• Government Spending /or Taxes

9
3. FORMULATING HYPOTHESES(i.e., EDUCATED
GUESSES)

• Derived From Either Theoretical Logic or
  Empirical Conjecture
• Characteristics of a Good Hypothesis
• An Empirical Statement
• Generalization
• Plausible
• Specific
• Appropriate
• Testable

10
3. FORMULATING HYPOTHESES(i.e., EDUCATED
GUESSES)

• Directional Relationship X has a positive
  impact on Y
• Positive Relationship X and Y Covary in the Same
  Direction
• Negative/Inverse Relationship X and Y Covary in
  the Opposite Direction

11
3. FORMULATING HYPOTHESES(i.e., EDUCATED
GUESSES)

• Tautological Relationship A Hypothesis that
  contains two concepts (X Y) which are
  essentially identical.
• e.g., As political insulation of bureaucratic
  agencies increases, these institutions act in a
  more independent fashion.

12
3. FORMULATING HYPOTHESES(i.e., EDUCATED
GUESSES)

• Spurious Relationship When X appears to Cause Y,
  but actually Z is the true source of the causal
  relationship involving Y.
• e.g., Conference Winning the Super Bowl Affects
  Stock Market Performance

13
3. FORMULATING HYPOTHESES(i.e., EDUCATED
GUESSES)

• Endogenous (Two-Way) Relationship
• When X ? Y, but Y ? X, too!
• e.g., Democratization ? Economic Growth
• Economic Growth ? Democratization

14
4. SPECIFYING UNITS OF ANALYSIS

• Unit of Analysis Type or Level of Political
  Actors (or Policy) which the hypothesis is
  thought to apply.
• E.G. 1 The impact of ideology on the
  legislators vote choice on a Budget bill.
• Unit of Analysis Legislator

15
4. SPECIFYING UNITS OF ANALYSIS

• E.G. 2 The impact of professionalized state
  legislatures on state economic growth.
• Unit of Analysis Each State Legislator

16
5. ECOLOGICAL INFERENCE

• EI Use of Aggregate Data to Study Individual
  Level Behavior.
• EI should be avoided at all costs i.e., data
  should match the unit of analysis
• WHY? Ecological Fallacy Problem
• Erroneously claiming a relationship (or lack
  thereof) when one does not exist (or does exist)

17
6. CONCEPTS

• The DNA of Empirical Inquiry
• Needed! Operational Definitions
• Conversion of Abstract Concepts into Measurable
  Concepts
• PURPOSES
• Transmissible ? Replication ? Extension
• Empirical ? Inference ? Evidence

18
6. CONCEPTS

• E.G. 1 ECONOMIC PERFORMANCE
• Income Growth
• Low Unemployment
• Low Inflation
• Trade Surplus
• Exchange Rates

19
6. CONCEPTS

• E.G. 2 SOCIAL ATTITUDES
• Abortion
• Race
• Gay/Lesbian Marriage/Civil Unions
• School Prayer
• Guns/2nd Amendment

20
6. CONCEPTS

• E.G. 3 REPRESENTATION
• Descriptive Representation Personal
  Characteristics e.g., race, gender, religion
• Symbolic Representation God and Country or
  Fighting for Working Americans - e.g.,
  speeches

21
6. CONCEPTS

• Substantive Representation To what extent do
  elected representatives mirror the wishes of the
  majority of constituents (i.e., median
  constituent)?
• e.g., legislators ideology
• median constituent ideology

22
7. HYPOTHESIS TESTING

• Scientific Generalization Expresses a
  Relationship Between Concepts.
• Hypothesis An educated guess about
  relationships.
• Well-Confirmed Hypothesis A hypothesis that is
  found to be true.
• Laws Relationships confirmed 99-99.99999 of
  the time

23
7. HYPOTHESIS TESTING

• Conditional Generalizations
• Statistical Most or Tends (a generalization
  that relates to most of a population.
• Universal All (a generalization that pertains
  to an entire population)

24
7. HYPOTHESIS TESTING

• Characteristics of Generalizations
• Empirical Import Grounded in real-world
  observation so that one must be able to confirm
  or deny relationships (i.e., falsifiable)
• Systematic Import Concepts must be related in
  some meaningful manner (i.e., plausible
  relationship)

25
7. HYPOTHESIS TESTING

• Null Hypothesis Assumption of no relationship
  (or difference) between variables.
• Alternative Hypothesis Null hypothesis is false.
• Directional Hypothesis An educated guess about
  the direction of the relationship (one-tailed
  hypothesis testing)

26
7. HYPOTHESIS TESTING

• Non-Directional Hypothesis Null hypothesis is
  false but do not choose a direction for the
  relationship (two-tailed hypothesis testing)

27
7. HYPOTHESIS TESTING

• Hypothesis Testing (Inferential) Errors
• Type I Error Incorrectly rejecting the null
  hypothesis (i.e., falsely concluding existence of
  a relationship)
• Type II Error Incorrectly accepting the null
  hypothesis (i.e., falsely concluding absence of a
  relationship)

28
8. MEASUREMENT

• Measurement Systematic Observation and
  Representation by Quantitative Values (i.e.,
  Numbers)
• Operational Definition of Concepts Deciding what
  kinds of empirical observations should be made to
  measure the occurrence of an attribute or
  behavior. These are important since different
  people mean different things by examining the
  same concept -- it is important to agree on some
  basic measure(s) of a concept so that some
  consistency occurs.

29
8. MEASUREMENT

• E.G. 1 How to Measure the Ideology of Supreme
  Court Justices?
• Rulings/Decisions Made on the SC
• Expert Surveys of Legal Scholars
• Justices Past Writings or Cases Prior to
  becoming a member of the SC

30
8. MEASUREMENT

• E.G. 2 Measurement of U.S. Influence in the
  Middle East
• Economic Trade
• Economic Non-Military Aid (Humanitarian and
  Development)
• Military Military Aid

31
8. MEASUREMENT

• Operational definitions are seldom absolutely
  correct or incorrect but rather should be
  evaluated in terms of how well they correspond to
  the concept that is attempted to being measured.
  

32
8. MEASUREMENT

• Factors that Plague Measurement
• Properly Designed Instruments
• e.g. Using different polls (with different
  formats) to make inferences about the same
  phenomenon.
• Data Constraints
• Analyzing data on Campaign Financing prior to
  1972 is impossible since Federal Election Laws
  did not require public disclosure

33
8. MEASUREMENT

• Dependence on Secondary Sources
• e.g. use of expert surveys or government
  documents which relies on the assessments of
  others.

34
8. MEASUREMENT

• Levels of Measurement
• Nominal Measurement A variable that assigns
  numerical values based upon discrete
  classification in mutually exclusive categories
• e.g., gender, race, binary party affiliation,
  religion

35
8. MEASUREMENT

• Ordinal Measurement More or Less comparisons
  can be made regarding different numerical values
  of a given variable. However, these values do
  not tell us anything about relative comparisons
  of how much more or less
• e.g., candidate thermometer rankings,
  categorical assessments of education and icnome

36
8. MEASUREMENT

• Interval Measurement Intervals between ordinal
  categories/values has meaning. That, is, we can
  assess how much larger or smaller in precise
  terms. Rankordered items, but places equal
  intervals between its categories. It cannot make
  statements such as candidate is twice as popular
  as candidate Y since there is no absolute zero
  point.
• e.g., inflation, budget deficits

37
8. MEASUREMENT

• Ratio Measurement Most complete form of
  measurement that states the exact magnitude
  differences among categories by having the same
  properties of interval measurement, plus absolute
  comparisons based upon zero baseline.
• e.g., vote share in an election,
• unemployment rate

38
8. MEASUREMENT

• Reliability The extent to which a measure yields
  the same results on repeated trials/samples from
  a population.
• Goal Ensure consistency among results
• from different trials

39
8. MEASUREMENT

• Different Methods for Analyzing Reliability
• Test-Retest Method Applying the same "test"
  (i.e., research instrument(s) / question(s)) to
  the same observations after a period of time and
  comparing the results of the different
  measurements
• (e.g., SAT example)
• Drawback Potential for Contagion between
  measurements taken at two different points in
  time (e.g., learning effects)

40
8. MEASUREMENT

• Alternative Form Method Using two different
  measures to gauge the same concept at two
  different points in time.
• (e.g., two surveys on policy liberalism that
  have different questions on them).
• Drawback Potential for Contagion between
  measurements taken at two different points in
  time as well as a lack of comparability
  problem between using different measures of the
  same concept

41
8. MEASUREMENT

• Split-Halves Method Assesses two measures of the
  same concept simultaneously. The results of the
  two measures are compared. (e.g., half of
  liberalism questions are given to one group while
  the remaining half are given to another group).
• Drawback While it overcomes the temporal
  contagion problem, it requires that the two
  subgroups are representative of one another
  (i.e., the means of breaking down the sample into
  two groups is unbiased

42
8. MEASUREMENT

• Validity The extent to which a measure is
  representing what it is supposed to measure.
• Goal Ensure accurate measurement of concepts by
  exhibiting a strong association between the
  measure and the related concept.

43
8. MEASUREMENT

• The validity of a measure is more difficult to
  demonstrate empirically than its reliability
  because it is difficult to obtain information
  regarding the correspondence between the
  measurement of a concept and the actual presence
  or amount of the concept itself

44
8. MEASUREMENT

• Different Methods for Analyzing Validity
• Face Validity Does it pass the smell test? A
  matter of judgment, not empirical proof
• Drawback Unless consensus exists for a
  particular measure of a given concept, it is
  difficult to establish face validity

45
8. MEASUREMENT

• Content Validity involves determining the full
  domain or meaning of a particular concept and
  then making sure that measures all portions of
  the domain are included in the measuring scheme.
  (e.g., measuring the full domain of policy
  liberalism by the American public)
• Drawback Must ensure that full domain of a
  particular concept is both defined and accounted
  for in the measurement scheme (difficult reaching
  agreement on this matter)

46
8. MEASUREMENT

• Construct Validity When a measure of a concept
  is related to a measure of another concept with
  which the original concept is thought to be
  related (e.g., strength of partisan
  identification and voting behavior Freshman
  undergraduate GPA and SAT scores).
• Drawback Failure to establish a relationship
  could pertain to many things (i.e., theoretical
  relationship is in error, poor measures of a
  concept, or inappropriate testing procedures)

47
8. MEASUREMENT

• Interitem Association relies on the similarity
  of outcomes of more than one measure of a concept
  to demonstrate the validity of the entire
  measurement scheme. (i.e., a multivariate analog
  to assessing construct validity)
• Drawback Same as for Construct Validity, except
  less problematic given reliance on multiple
  measures as a means to assess valid measurement
  of a concept

48
(No Transcript)
49
9. RESEARCH DESIGN

• ELEMENTS OF CAUSALITY IN A RESEARCH DESIGN
• Covariation Does X covary with Y?
• Time Order Does X precede Y?
• Confounding Factors Do A B also cause Y so as
  to make X moot?

50
9. RESEARCH DESIGN

• Example The impact of race on voting Y Vote
  Choice XRace ZPartisan ID

51
9. RESEARCH DESIGN

• Experimental Research Designs
• Researcher has control over stimulus applied to
  the experimental group
• Researcher controls assignment of subjects into
  experimental and control groups i.e.,
  randomization
• Observe/Measure responses or behavior

52
9. RESEARCH DESIGN

• Researcher does not have to control for
  extraneous factors (Zs)

53
9. RESEARCH DESIGN

• (1) Classic PreTest and PostTest Design
• R? Pre-Test E ? X ? Post-Test E
• R? Pre-Test C ? P ? Post-Test C
• Measurement of the dependent variable are taken
  both before and after the treatment.
• This allows one to examine differences that exist
  between the groups before the treatment in order
  to see whether these differences are attributable
  to the treatment, or inherent group differences.

54
9. RESEARCH DESIGN

• WEAKNESS Only a single group receives a
  pretest, a stimulus, and a posttest. One
  problem with this method is that there is no sure
  way of knowing whether the change in the
  dependent variable was due to the experimental
  factor and not to other factors. Also, there is
  no way to check for pretest and posttest based
  stimulus interaction.

55
9. RESEARCH DESIGN

• Simple Post-Test Design
• Two groups (E C) and two variables (X Y)
• R ? E ? X ? Post-Test E
• R ? C ? P ? Post-Test C
• Weakness Random Assignment is more Uncertain
  since no pre-testing occurs

56
9. RESEARCH DESIGN

• Repeated Measurement Design
• Multiple Pre-Test and Post-Test measurement
  with both groups being administered the stimulus
  at the same time.
• E.G. Focus groups viewing a presidential debate.

57
9. RESEARCH DESIGN

• Multi-Group Experimental Design
• Akin to the Classic Pre-Test and Post-Test
  Design, except with multiple experimental groups
  and altering the stimulus treatments.
• Three Group Example
• R ? Pre-Test E1 ? X ? Post-Test E1
• R ? Pre-Test E2 ? X ? Post-Test E2
• R ? Pre-Test E3 ? X ? Post-Test E3
• R ? Pre-Test C ? P ? Post-Test C

58
9. RESEARCH DESIGN

• Strength Overcomes the weakness associated with
  Classic PreTest and PostTest Design

59
9. RESEARCH DESIGN
60
9. RESEARCH DESIGN

• Field Experiments experimental design in a
  natural settings, whereby the researcher cannot
  randomly assigns subjects to experimental and
  control groups, but can manipulate the
  experimental variable.

61
9. RESEARCH DESIGN

• One cannot control for non-experimental factors
  relating to extraneous factors (e.g., historical
  effects) that naturally occur outside the lab.
• Pro Improves external validity from being in
  real world setting
• Con Lowers internal validity ? spurious effects
  more likely

62
9. RESEARCH DESIGN

• E.G. Voter Turnout Do Early Voting Rules
  Improve Political Participation?
• NO Random Assignment of Eligible Citizens to
  Early Voting (experimental) and Election Day
  Voting (control) Groups within a state that
  allows for early voting.
• Researcher controls who votes early and who
  votes on election day

63
9. RESEARCH DESIGN

• Need to examine a state with early voting laws
  for comparability purposes
• Must select a common feature(s) so that the
  groups mirror one another. (e.g., college age
  voters representing as the population of
  interest)
• Must be concerned about environmental
  influences that may lower internal validity

64

• NON-EXPERIMENTAL DESIGNS

65
9. RESEARCH DESIGN

• Case Study Design Small-N Design (single,
  comparative, or focus group)
• In-depth investigation of one or a handful of
  observations
• Most often used for exploratory or descriptive
  purposes

66
9. RESEARCH DESIGN

• Cons Limited Generalizability Sample Selection
  and Spurious Relationships
• Pro Deep Understanding of Causality

67
9. RESEARCH DESIGN

• Cross-Sectional Design Measurements of both the
  independent and dependent variables taken at
  approximately the same time
• Individuals (surveys) or Aggregate (groups,
  institutions, states, nations)

68
9. RESEARCH DESIGN

• Pros Improves External Validity due to large N
  -- especially generalizability across populations
• Cons Lowers Internal Validity (i.e., determining
  true cause and effect)

69
9. RESEARCH DESIGN

• E.G. Early Voting Laws and Vote Choice in 2008
  Presidential Election
• Pro A sample of thousands of voters compare
  individuals from early voting to non early
  voting states
• Con Do not have pre-test observations on the
  impact of switching to early voting laws.

70
9. RESEARCH DESIGN

• Time Series (Longitudinal) Design
• Repeated Measurement for a single cross-sectional
  unit through time
• Pros (1) Cause?Effect (2) Dynamic Effects
• Cons (1) Omitted Factors
  
• (2) Period/Transitory effects

71
9. RESEARCH DESIGN

• E.G., Stimsons Policy Mood Measure
• Did Reagan Lead the Republican Revolution?

72
9. RESEARCH DESIGN

• Interrupted Time Series Design
• Analogous to a Classic Pre-Test / Post-Test
  Design
• Researcher does not have control over group
  assignment nor application of stimulus

73
9. RESEARCH DESIGN

• Con Threat to Internal Validity due to omitted
  factors that may drive change independent of the
  intervention event
• Pro Analysis f dynamic political change
• E.G., Mass Voter Mobilization of
  African-Americans in the American South

74
9. RESEARCH DESIGN

• Panel Design measurements both cross-section and
  through time. Change in individuals or aggregate
  units through time
• Pro Overcomes Internal Validity problems
  associated with Cross-Sectional Design
• Con Panel Mortality

75
9. RESEARCH DESIGN

• E.G. Cross-National Relationship between
  Democratization and Economic Growth

76
(No Transcript)
77
10. SAMPLING

• What is a Sample? A subset of observations/units
  derived from a population.
• What is a Population? A well-defined set of
  observations that encompasses a particular
  hypothesis
• The Costs of Population Analysis time, money,
  etc

78
10. SAMPLING

• The Basics of Sampling
• Sample Statistics () Statistics derived from a
  sample used to approximate corresponding
  population values/ parameters (e.g., mean,
  median, variance/standard deviation)
• Sample Statistic

79
10. SAMPLING

• How well does the sample estimate approximate
  population parameter?
• Sample Bias Bias attributable to systematic
  exclusion of elements from a sample
• Sample Bias (Desirable)
• 
  (Undesirable)

80
10. SAMPLING

• E.G., Sampling Bias in Election Polling (Likely
  Voters vs. Registered Voters)

81
10. SAMPLING

• Sampling Error The amount of error attributable
  to a sample estimate
• Sampling Error ,
• where the expected (i.e., average) value of the
  sample estimator equals the corresponding
  population parameter

82
10. SAMPLING

• Elements of Statistical Inference
• Expected Value the average (long-run) value of a
  sample statistic based on repeated samples from a
  population
• Standard Error the measure of dispersion (i.e.,
  standard deviation) of a sampling distribution
  surrounding the expected value of the sample
  estimator ( )
• Confidence Interval (Sampling Distribution) What
  of the time that we observe the sample
  statistic ( ) if we were to replicate the
  sample k times.

83
10. SAMPLING

• Confidence Interval Equation
• 90 C.I.
• 95 C.I.
• 99 C.I.
• Note The scalars above assume a Standard Normal
  Probability Distribution.

84
10. SAMPLING

• How Large A Sample?
• As sample size ?, Sampling Error ?
• Cost ?
• See Table 7.4 in Textbook (p. 237) for details.

85
10. SAMPLING

• Probability Samples Each element in the
  population has a known probability of being
  selected.
• Simple Random Sample each element has an equal
  chance of selection
• Systematic Sample Elements are selected at
  predetermined intervals (as opposed to at random)

86
10. SAMPLING

• Stratified Sample Elements share one or more
  characteristics are grouped (e.g., gender, race,
  religion), and elements are selected from each
  group in proportion to each groups proportion in
  the total population.
• Cluster Sample Used to circumvent not having a
  list of elements in the sample population. Use
  only a partial list of elements.

87
10. SAMPLING

• Non-Probability Samples Each element in the
  population has an unknown probability of being
  selected.
• Purposive Sample Researcher selects cases of
  interest

88
10. SAMPLING

• Convenience Sample inclusion of elements based
  upon ease
• Quota Sample Elements selected based upon their
  proportion to their representation to the
  population (nonprobability sampling analog to
  proportionate stratified sampling)
• Snowball Sample elements are chosen through
  word of mouth contact by other elements

89

• END OF UNIT 2 MATERIAL