How to Review and Evaluate a Scientific Article

1
How to Review and Evaluatea Scientific Article

• AEE Graduate Student Seminar Series
• January 16, 2002
• Revised February 5, 2003
• Mikel Woods, Ph.D.

2
The Basics of a Critique

• What is the specific question (s) being
  addressed?
• What general research methods are used?
• List the major findings of the paper and identify
  the figures and tables (data) which support those
  findings.
• Are you confident about the data presented? If
  not, why not? What might be wrong?
• What major conclusions do the authors make in
  their discussion? Are their conclusions supported
  by the data? How are these data important to the
  field (That is, what do these data tell us which
  is new?)?
• Are there other possible interpretations
  (explanations) for the data? What are they?

Girden, E.R. (1996). Evaluating research articles
from start to finish. One Thousand Oaks, CA.
Sage Press.
3
Reviewing the Purpose

• What were the objectives in performing the study?
• What were the research questions being asked?

4
Study Design

• What type of study was performed?
• How was the study population sampled?
• Are there potential sources of selection bias?

5
Measurement and Observation

• Are clear definitions of terms used?
• What outcome criteria were used?
• What measurements were taken?
• How?
• What methods were used to assess validity or
  reliability of measurements?

6
Results

• How are the data presented?
• Is the data relevant to the study question?
• Are there data that were not presented?

7
Conclusions

• What were the main conclusions?
• Is the study significant?
• Is the study relevant to other populations?
• What questions remain unresolved?

8
Reading the Fun Way

• First thing to do is look at the four corners of
  the picture. (That way you don't miss anything.)
• Then look at the center and find what catches
  your eye,
• Then the foreground and background.
• Then ask yourself, what is going on here?

9
Evaluating Your Peers

• What to Look For
• How to Say it Constructively

10
Writing Evaluation Grid

• Content
• Organization, Rhetorical Logical Development
• Organization Sentence Structure
• Word Choice
• Tone
• Spelling
• Format

11
Use of Supporting Evidence

• This point is underdeveloped.
• The details here dont really help me see your
  point.
• Im not convinced whats your authority?
• The X (three) reasons you offer are very
  persuasive.
• Good Examples. Can you offer more?

12
Use of sources.

• Only two sources? You need several more.
• Who said this?
• One of the clearest paraphrases Ive seen of this
  crucial passage.
• Your summary leaves out three of the writers
  main points.
• Your summary is just repletion it doesnt add
  anything new.
• This quotation beautifully sums up your argument.
• Why do you quote at such length here? Why not
  paraphrase?
• You cite only sources that support your claim
  citing one or two with differing views would help
  show me youve considered other opinions.

13
The Purpose

• Why are you telling us all this?
• What is the issue here, and what is your stand on
  it?
• Very efficient and thorough discussion! You
  explain the content very clearly and thus reveal
  your understanding of the issue/topic?

14
Attention to Audience

• Careful you dont talk down to your readers.
• Youve left me behind here. I cant follow.
• Your level of jargon is perfect for relating to
  the audience.
• Im really enjoying reading this!
• Dont assume everyone shares your opinion about
  this issue.

15
Overall Impression

• I was looking for more critical analysis from
  you, and Ive found it!
• Your grasp of the material here is truly
  impressive.
• What happened here? I cant understand your point
  in this article/section.
• Good job -- you convinced me!

16
Overall Originations

• Im confused here what does this point have to
  do with the one before it?
• Your most important point is buried here in the
  middle. Why not move it up front.
• Organization is chronological rather than
  topical as a result, you summarize but do not
  analyze.
• How did we get here? You need a transition.

17
Sentence Structure Style

• The passing of your sentences here really keeps
  me reading excellent variation of length and
  type.
• Can you combine sentences to make a logical
  connection explicit here?
• Your use of questions helps clarify this complex
  issue.
• This is not effective word order for a closing
  sentence Ive forgotten your main point. Can
  you find a better sentence?
• These sentences all begin with nouns the result
  is a kind of dull clip-clop, clip-clop.

18
Paragraph Structure

• Why the one- and two-sentences paragraphs?
  Elaborate!
• Your introductory paragraph immediately gets my
  attention and gives an overview of the essay
  good!
• I cant follow the information in this paragraph.
  Can you reorder it?
• What is the main idea of this paragraph?
• Very effective ordering of details in this
  paragraph.
• This paragraph skips around two or three points.
  It has enough ideas for three paragraphs.

19
Format

• You need a title, one that gets across your
  meaning.
• Your headings helped me follow this report. Why
  not use subheadings?

20
Documentation

• I checked that source, and this quotations not
  on the page you list.
• What are you paraphrasing here? Your introduction
  merely drops readers into the middle of things.
  Introduce the material paraphrased.
• What are you summarizing here? Where do these
  ideas come from?
• I cant tell where this quotation ends.
• Keep in-text citations as simple as possible
  see information in style manual.
• Why arent works listed in alphabetical order?
• This is not correct APA citation style. Check
  your manual!
• What is the date of this publication?