Reading and Critiquing Research

1
Reading and Critiquing Research

• March 1, 2005
• N295

2
Research Critique

• A careful, critical appraisal of the strengths
  and limitations of a study
• A reviewer examines a study for
• Merits
• Limitations
• Meaning
• significance

3
Research Critique

• Researchers designing a study make several
  methodological decisions that affect the quality
  and integrity of the research.
• Consumers preparing a critique evaluate the
  decisions the researchers made to determine how
  much faith can be placed in the results.

4
Guidelines for Conducting a Research Critique

• Read and critique entire study
• Examine the reports organization and
  presentation

• Identify strengths and weaknesses
• Be objective in identifying strengths and
  weaknesses
• Suggest modifications for future studies

5
Steps of a Research Critique

• Comprehension
• Read the article carefully
• Identify terms you do not understand and look up
  their meaning in glossary of research text
• Read the article a second time
• Highlight each step of the research process

6
Steps of a Research Critique

• Comparison
• How well did the researcher follow the rules for
  an ideal study?
• Analysis
• Involves a critique of the logical links
  connecting one study element with another

7
Guidelines for Comparison and Analysis

• Research Problem and Purpose
• Is the problem significant to nursing?
• Does the purpose clarify the aim of the study?

• Review of Literature
• Organized?
• Shows progressive development of ideas through
  previous research?
• Provides rationale and direction to the study?

8
Guidelines for Comparison and Analysis

• Study Framework
• Is the framework clear and linked to the research
  purpose?
• Are maps/models logical?

• Research questions or hypotheses
• Expressed clearly?
• Logically linked to the research purpose and
  framework?

9
Guidelines for Comparison and Analysis

• Variables
• Reflect the concepts identified in the framework?
• Clearly defined?
• Conceptual definition consistent with operational
  definition?

• Design
• Appropriate?
• Evidence of a clearly defined protocol?
• Treatment closely monitored to ensure
  consistency?
• Threats to internal validity minimized?
• Logically linked to sampling method and
  statistics used?

10
Guidelines for Comparison and Analysis

• Sample, Population, and Setting
• Sampling method adequate to produce a
  representative sample?
• Biases?
• Human rights protected?
• Setting appropriate?

• Measurements
• Instruments adequately measure the study
  variables?
• Reliability and validity described?
• Instruments and scoring techniques clearly
  described?

11
Guidelines for Comparison and Analysis

• Observation
• Is what is being observed clearly defined?
• Inter-rater reliability described?
• Techniques for recording observations described?

• Physiological Measures
• Appropriate for the research questions?
• Data recording consistent?
• Methods of recording measures clearly described?

12
Guidelines for Comparison and Analysis

• Data Collection
• Process clearly described?
• Consistent?
• Ethical?
• Training of data collectors addressed?

• Data Analysis
• Procedures appropriate to the type of data
  collected?
• Analysis procedures clearly described?
• Results presented in an understandable way?

13
Interpretation of Findings

• Are findings clinically significant?
• Do the conclusions fit the findings from the
  analyses?
• Conclusions based on statistically significant
  and clinically significant results?
• Study limitations identified?

14
Guidelines for Evaluation

• Examine the links between study process, study
  findings and previous studies.
• What rival hypotheses can be suggested for the
  findings?
• How much confidence can be placed in the study
  findings?
• To what populations can the findings be
  generalized?

15
Guidelines for Evaluation

• Do the findings build on findings of previous
  studies?
• Current knowledge in the area identified?
• Author indicates implication of the findings for
  practice?
• What is the next logical research question from
  this study?

16
Quick Checklist

• Purpose of Study
• Clarity easy to understand?
• Practical significance results will impact
  practice?

• Statement of Problem
• Clarity easy to understand?
• Conceptual definitions a word picture of the
  problem
• Operational definitions description of how
  people or things will be measured?

17
Quick Checklist

• Literature Review
• Current citations published within the past 5
  years?
• Classic studies important works included in the
  review?
• Relevance works pertain to the topic being
  investigated?

• Methodology
• Population identified?
• Adequate sample enough subjects?
• Representativeness sampling method identified?
• Research design described?
• Could be easily replicated?

18
Quick Checklist

• Methodology (cont.)
• Data collection data recorded carefully tools
  reliable and valid?
• Data analysis appropriate tests or statistics
  were used?

• Results and conclusions
• Accuracy data interpretation consistent with
  results?
• Conclusions addressed research questions?

19
Quick Checklist

• Results and conclusions (cont.)
• Significance to nursing implications for
  nursing practice addressed?
• Clarity results were easy to understand and
  presented clearly?
• Limitations identified?

• Overall concern
• Objectivity biases were recognized and stated
  in report
• Ethical considerations informed consent was
  obtained IRB approval?
• Readability study makes sense valuable to
  practice?