Peer review

1
Peer review

• Critique of a scientific paper

2
Key elements

• Scientific argument
• Methods
• Writing
• Tables and figures

3
Background Asks a specific question
?

• What is the research question?
• What are the global issues?
• How do they apply to the local context?

Argument
4
Results Reports sound scientific data that meet
the objectives
?

• Was the research question addressed?
• Is there enough detail?
• Do results allow statistical inference?

Argument
5
Discussion Interprets results, builds a case
based on data and the literature
?

• Are the results interpreted on the basis of what
  is already known?
• Is the case for the conclusions well made?

Argument
6
Limitations Described, analyzed,impact on
conclusions clear
?

• Key limitations, could influence main conclusions
  discussed?
• Limitations?
• Description
• Description of the consequences
• What may be done to address the limitation
• Interpretation of the results in light of the
  limitation

Argument
7
Suggests next steps in intervention and research
?

• Next steps suggested in terms of
• Action? Research?
• Are they relevant?
• Are they based upon the data?

Argument
8
Design adequate to meet objectives
?

• What are the study objectives?
• Measure a quantity?
• Test a hypothesis?
• What kind of indicators does the study objectives
  call for?
• Is the study design explicit?
• Can the study design lead to the indicator needed?

Methods
9
Population well defined and relevant to the
objectives
?

• What are the inclusion criteria?
• What are the exclusion criteria?
• Is the population relevant to the objectives?

Methods
10
Definitions are specified, clear and based upon
standard criteria
?

• Are the outcomes defined?
• Are the exposures/interventions defined?
• Are standard definitions used?

Methods
11
Sampling methods are sound
?

• Is the sample described?
• How was it selected?
• Are sampling methods compatible with classical
  methods?
• Will the sample address the objectives?

Methods
12
Sample size Estimated appropriately and is
sufficient
?

• Mention of a sample size calculation?
• Are the methods/formula used relevant?
• Are the assumptions making sense?
• Will the power be sufficient to address the
  objectives?

Methods
13
Design is free of bias
?

• Selection biases
• Information biases

Methods
14
Data collected is relevant and well described
?

• Is it clear what data was collected?
• Is the data collection relevant to the study
  objectives?
• Unnecessary information?
• Key piece of information missing?

Methods
15
Data collected with methods ensuring sufficient
quality
?

• What kind of instruments were used?
• Who collected the data?
• What were the quality assurance procedures?

Methods
16
Analysis thought of beforehand and appropriate
?

• Is there a mention of an analysis plan?
• Does the data analysis address the objectives?
• Are there multiple/excessive comparisons?

Methods
17
Indicators appropriate and well calculated
?

• Are indicators appropriate for study design?
• Disease frequency
• Incidence
• Prevalence
• Association
• Odds ratio
• Prevalence ratio
• Risk ratio
• Are indicators calculated correctly?

Methods
18
Statistical tests are appropriate and well
computed
?

• Were the tests chosen appropriate?
• Were computations correct?
• Is the interpretation right?

Methods
19
Appropriate attention to protection of human
subjects
?

• Does the study involve human subjects?
• Mention of possible risks/benefits?
• Confidentiality of data?
• Informed consent?
• Ethics committee approval (if applicable)?

Methods
20
Content well distributed
?

• IMRAD
• Does each section contain what it is supposed to
  contain?
• Does any section contain irrelevant material?
• Are there elements placed in the wrong section?

Writing
21
Language is simple and clear
?

• Is the language simple, avoiding jargon?
• Are sentences short?
• Is the argument structured?
• Are the meanings specific?

Writing
22
Writing is sequential, reads well
?

• Are sentences starting where the previous ended?
• Is the writing taking the reader by the hand from
  the introduction to the conclusion?
• Active voice?

Writing
23
Words Precise and consistent
?

• Are appropriate words chosen?
• Are identical words used to refer to the same
  concepts?
• Are words used in the context of their accepted
  scientific meanings?
• No use of significant if does not to refer to
  statistics

Writing
24
Only relevant and useful tables and/or figures
?

• How many tables and figures are there?
• Are the tables redundant with the figures?
• Are the tables/figures all needed?

Tables and figures
25
Choice of graph/table to display information is
appropriate
?

• Are the graph(s) effective at capturing a piece
  of information?
• Are the table(s) effective at displaying the data
  to
• Point to the main trends?
• Point to the main exceptions to these trends?

Tables and figures
26
Tables are clear, exact and the totals add up
?

• Are the tables readable?
• Are the titles explicit?
• Are there row / column summaries?
• Do the totals add up?
• Are there footnotes to explain unclear points?

Tables and figures
27
Graphs appropriate, can be understood
?

• Is there a clear message behind every graph?
• Are the graphs drawn in a way that communicate
  the message effectively?
• Is the ink-to-data ratio low?

Tables and figures
28
Take home message

• Use a checklist to evaluate
• Assess the strength of the argument
• Review the methods point by point
• Evaluate the effectiveness of the writing
• Check appropriateness of the tables/figures