Research Methods in Politics 6

1
Research Methods in Politics6

• Critically Evaluating Published Information

2
Teaching and Learning Objectives

1. to learn how to critically evaluate published
   research using research component analysis and,
2. to apply Roses ABCDE model (1982) for
   deciphering research

3
Research Component Analysis

1. when
2. where
3. who
4. abstract
5. research question and rationale
6. theory and literature review
7. Hypothesis
8. methodology

• data collection what,
• when, how
• data presentation
• data analysis, interpretation and
  discussion
• conclusions
• implications
• style
• referencing and bibliography.

4
Roses ABCDE deciphering model

• A Theory an explanatory statement about social
  phenomena. ?
  ? ?
• B Theoretical Proposition specific propositions
  to be ?
• investigated in the study. ?
• ? ?
• C Operationalisation decisions made on how to
  carry out ?
• empirical work techniques of data collection
  sampling ?
• concepts and indicators variables units.
  ?
• ?
• D Fieldwork collecting data practical problems
  of ? implementing Stage C
  decisions. ?
  ? ?
• E Results data analysis leads to findings
  interpretation leads back to C, B, and A

5
Three-Stage Framework for Evaluating Research

• Relationship to other theory and research
• ?
  External validity
  
• A Theory ?
• ? Internal theoretical
  validity
• B Theoretical Proposition ?
• ?
• C Operationalisation ?
• ? Internal empirical validity
• D Fieldwork ? ?
  
• E Results ?

6
Roses Strategy for Evaluating Theory-Testing,
Quantitative Research

• 1. summarise the research into ABCDE stages
• 2. assess the operationalisation in C
  concept-indicators, samples, units and variables
• 3. add further descriptive details of the
  researchers theoretical exposition and
  theory-evidence linkages and, tables of data
  presented, reliability and accuracy, and
  interpretation
• 4. assess the internal empirical validity by
  examining the fit between the data and its
  interpretation
• 5. assess the internal theoretical validity, i.e.
  the relationship between A (theory) and B
  (hypothesis) and,
• 6. conclude with a general assessment which
  consolidates the separate assessments made and
  makes an assessment of the overall contribution
  of the research to the discipline. (Rose, 1982
  104-05)

7
Roses Strategy for Evaluating Theory-Building
Research Utilising Qualitative Data

• summarise the research report into its ABCDE
  stages stating its natural history (origins and
  development) the data and methods of collection
  sampling method of analysis and, presentation
  of results
• evaluate in turn the validity of the
  concept-indicator links theory and, sampling
  and generalisation and,
• conclude by reviewing the consistency (or
  otherwise) of the various validities and other
  factors and assessing of the overall contribution
  of the new theory to the discipline (Rose, 1982
  130-02)

8
Questions for Discussion or Assignments

• Consider the concepts of conflict freedom
  liberty hegemony and, consciousness of
  identity. What concept-indicators would you
  suggest for these? Why? What data would you
  collect?
• For undergraduates You are a distinguished
  academic. You have been asked by the editor of a
  refereed journal to act as a referee for draft
  research report. Select one of the two published
  reports
• Bevir, M., and Rhodes, R A W. (2006) Prime
  Ministers, Presidentialism and Westminster
  Smokescreens IN Political Studies, vol. 54,
  Number 4, pp. 671-90 OR,
• Chaney, P. (2006) Critical Mass, Deliberation
  and the Substantive Representation of Women
  Evidence from the UKs Devolution Programme IN
  Political Studies, vol. 54, Number 4, pp.
  691-714.
• Write your assessment using the letter
  (below)

9

• Dr Arial Helvetica USR
• Editor, New Politics Digest Department of
  Politics
• University of Watersea
• Dear Arial
• title of draft research paper
• Thank you for inviting me to referee this paper.
• Overall, my view is that, whilst this excellent
  paper breaks new ground, there are a number of
  areas where minor revisions and additions would
  be helpful.
• Let me begin by summarising what I believe are
  the research question, theoretical framework,
  hypothesis and key concepts. The research
  question, etc are 200-300 words
• The strengths of the paper are 300-400 words
• The weaknesses are 300-400 words
• I would also like to comment on the style of the
  paper 200-300 words
• The paper is, of course, suitable for publication
  in New Politics Digest. However, I would suggest
  a number of changes. They are 200-300 words

10
Questions for Discussion or Assignments

• For graduates
• 3. Select a research report on a subject that
  interests you from a refereed journal in your
  university library. Evaluate it critically using
  research component analysis or Roses strategies.
  Attach a copy of the research paper to your
  assignment showing what you concluded were the
  various analytical components. How could the
  research and the report be improved?