Undertaking a Literature Review

1
Undertaking a Literature Review

• David Parkinson

2
What is a Review?

• According to Cooper (1988) '... a literature
  review uses as its database reports of primary or
  original scholarship, and does not report new
  primary scholarship itself... The types of
  scholarship may be empirical, theoretical,
  critical/analytic, or methodological in nature.
  Secondly a literature review seeks to describe,
  summarise, evaluate, clarify and/or integrate the
  content of primary reports.'

3
Why do a Review?

• The Philosopher Rumsfeld once said
• Now what is the message there? The message is
  that there are known "knowns." There are things
  we know that we know. There are known unknowns.
  That is to say there are things that we now know
  we don't know. But there are also unknown
  unknowns. There are things we don't know we don't
  know. So when we do the best we can and we pull
  all this information together, and we then say
  well that's basically what we see as the
  situation, that is really only the known knowns
  and the known unknowns. And each year, we
  discover a few more of those unknown unknowns.
• (Press Conference at NATO Headquarters,
  Brussels, Belgium, June 6, 2002)

4
Why do a Review?

• Know what to do (before starting research)
• to identify gaps in the literature
• Know where to start (starting)
• to carry on from where others have already
  reached, or position your project relative to
  previous work
• to identify information, methods and ideas that
  may be relevant to your project (i.e. avoid
  reinventing the wheel)
• Know what you have done (finishing)
• to increase your breadth of knowledge of your
  subject area
• to put your work into perspective
• Other
• to identify opposing views
• to identify other people working in the same
  fields

5
Outline

• Specifying
• formulating the problem
• Searching
• collecting the data
• Collating
• evaluating the data
• Analysing
• interpretation of the data
• Writing
• presentation of results

6
1. Specifying the Review

• Topic
• What subject will the review cover
• Type of Review
• Integrative, Theoretical, Methodological
• Breadth of Review
• The range of subjects that are covered
• Depth of Review
• The amount of detail that it goes into

7
Topic

• Should be the easiest step, but remember
• Doesnt need to be too specific
• Has to be a subject covered in the literature
• Be prepared to make (minor) alterations to the
  scope of the review as it proceeds

8
Types of Review

• Integrative
• Drawing conclusions from many separate studies
• Theoretical
• Present different theories to explain a certain
  phenomenon, comparing breadth, consistency and
  predictions
• Methodological
• Examine research methods that have been applied
  to a particular problem. Often critical, arguing
  that results may be artificial

9
Outline

• Try to outline your review in advance
• Break down the topic into subheadings

• A Realistic Cosmological Model Based on
  Observations and Some Theory Developed over the
  Last 90 Years by Burbidge
• Introduction
• The Beginning 1915-1936
• The 1940s
• The 1950s onwards
• Active galaxies as the most likely site of
  creation
• Energetics
• Quasi-stellar objects
• Dark matter
• Acceleration in the Universe
• Summary
• Conclusion

10
Breadth and depth

• Need to specify the level of detail and the span
  of the survey
• If the survey is too broad and too detailed, it
  may never be completed
• Need to decide what sources to use (journals,
  books etc)
• Be prepared to be dynamic
• Change the scope of the review depending on the
  literature and your own constraints

11
Scope

• How do I know when I have done enough?
• If new papers keep appearing, it might be a good
  idea to ignore them. Limit yourself to when you
  started writing, unless something really
  important appears.
• More than 10 pages, but less than 50.
• How much background/historical material do I
  need?
• Think of the audience (it might be you!)
• Historical papers are only needed to introduce
  new or unfamiliar concepts, so rarely necessary
  (most modern papers on gravity do not cite Newton
  or Einstein for example).

12
2. Searching

• Informal channels
• Yourself, friends, colleagues, conferences
• Primary channels
• Books, review articles (but remember, dont
  plagiarize)
• Secondary sources
• Bibliographies, abstract archives
• Bad sources of information (dont use, or use
  carefully)
• Google, wikipedia

13
Web of Knowledge
14
ScienceDirect
15
arXiv
16
SPIRES
17
NASA ADS
18
NCBI PubMed
19
APA PsycNet
20
PROLA
21
The Library
22
Search Terms

• Standard search terms include
• Title
• Author
• Abstract
• Keywords
• Every search engine will have different syntax
• Use Boolean operators (AND, OR etc)
• Some search engines have online tutorials to
  teach you how to use them

23
Search Process

• Start with some relevant source
• Find all references
• Screen papers, selecting only those which are
  relevant and of high quality
• Repeat steps 1-3 multiple times, building up a
  bibliographic source list
• Remember to change the scope/detail of review,
  depending on literature that appears during
  search

24
Finding papers

• Some papers may not be available from the online
  archives. If so
• Google for it
• The Library
• Inter-Library Loan
• Email the author
• Ask someone who may have it, or access to it

25
Scanning vs. Reading

• Dont read through every paper you come across -
  it takes too long!
• Instead scan through papers quickly, then
  identify essential reading
• Keep a bibliographic record of every paper, book
  or website you use (either using paper or
  bibliographic software)

26
Reference management software

• There is a host of software available for helping
  find, and keep note of your references (and
  perhaps even the papers themselves e.g. Papers)
• Some will link to one or more bibliographic
  database (such as the arXiv, PubMed, U.S. Library
  of Congress etc)
• Others contribute to a community database, a kind
  of social bookmarking (such as BibSonomy,
  CiteULike, or Connotea).
• Beware, these will only be as good as the people
  using them
• Know what software you will be using to write in
  choosing your bibliographic tool

27
Papers
28
BibDesk
29
Zotero
30
3. Collating

• Critical analysis of available literature
• Practical (Is it going to be useful?)
• Is it in a language I can read?
• Is it from the area I am reviewing?
• It is from a peer-reviewed Journal I respect?
• Quality (Is it any good?)
• Did the authors sample the population fairly?
• Did they treat the errors consistently and well?

31
The Citations Fallacy

• Can we judge how important a paper is by the
  number of citations it has?
• A paper that creates a new subject area, or makes
  important progress in that area is likely to be
  cited a lot, however
• Many important papers are rarely cited, but that
  doesnt mean they are not important
• Many wrong papers are cited by people trying to
  disprove them (but that doesnt mean the paper is
  not important)
• A paper may be highly cited because the author is
  working in a fashionable area, and has a lot of
  friends
• So citations are not necessarily measure of
  quality

32
4. Analysing

• What does the data actually tell us?
• How does it apply to the topic of the review?
• What conclusions can we draw?
• Are there any major disagreements between
  different sources/studies?

33
Meta-analysis

• Many studies have a small data set (population),
  or data with low constraining power
• To get around this, it is common for reviews to
  contain some element of meta-analysis, bringing
  together the different studies and making some
  overall conclusions.
• Be very careful when doing this, as different
  authors may treat the errors differently.
• A good meta-analysis of bad studies will still
  give bad results. Sometimes the best conclusion
  will be that nothing can be concluded.

34
5. Writing

• Introduction
• Define topic, also parameters/terms
• Body, where the review proceeds
• Chronologically
• Thematically
• Methodologically
• Conclusions
• Include future questions to be answered

35
Writing Hints

• Dont confuse a literature review with an
  annotated bibliography discuss themes,
  referencing many sources simultaneously
• Likely to be read by a wider audience than a
  highly technical paper, so jargon and specialized
  abbreviations should be avoided or carefully
  explained.
• Dont make a statement without a reference
  (unless it is obvious or trivial)
• Include a bibliography (!)

36
Referencing BibTex vs. EndNote

• BibTex is best if you are already familiar with
  LaTeX, as the interface is similar.
• There is graphical software to help you
• EndNote is best if you will be writing you papers
  using Word/Open Office etc
• Though EndNote libraries can be turned into
  BibTex files

37
BibTex

• BibTex allows you to store references as an
  external file, and make citations in your own
  fashion
• The papers are stored in a .bib file with the
  following format
• Many journals have their own BibTex style files
  (for example American Physical Society apsrev.bst)

Label
_at_articlegreenwade93, author "George D.
Greenwade", title "The Comprehensive
Tex Archive Network (CTAN)", year
"1993", journal "TUGBoat", volume
"14", number "3", pages
"342--351"
Type
38
BibTeX Packages

• Natbib is a reimplementation of the LaTeX \cite
  command, which allows for textual (\citet) or
  parenthetical (\citep)citation

\citetjon90 --gt Jones et al. (1990)
\citetchap. 2jon90 --gt Jones et al. (1990,
chap. 2) \citepjon90 --gt (Jones et al.,
1990) \citepchap. 2jon90 --gt (Jones et al.,
1990, chap. 2) \citepseejon90 --gt (see
Jones et al., 1990) \citepseechap.
2jon90 --gt (see Jones et al., 1990, chap. 2)
\citetjon90 --gt Jones, Baker, and Williams
(1990) \citepjon90 --gt (Jones, Baker, and
Williams, 1990)
39
Conclusions

• The purpose of a review is not just to find what
  is in the literature, but also to draw
  conclusions from it. Have this in mind from the
  beginning.
• Be as thorough as you can, bearing your
  constraints in mind
• Be flexible, changing the scope of your review,
  or the amount of detail that you cover, as the
  review proceeds

40
Bibliography

• Conducting Research Literature Reviews, Arlene
  Fink, 1998, Sage Publications
• The Integrative Research Review - a systematic
  approach, Harris M. Cooper, 1984, Sage
  Publications
• How to Write and Publish a Scientific Paper by
  Robert A. Day and Barbara Gastel
• Performing a Literature Review, Lois E. Reed,
  http//fie.engrng.pitt.edu/fie98/papers/1298.pdf
• natbib reference sheet http//merkel.zoneo.net/Lat
  ex/natbib.php
• Prizes have been awarded for Scientific
  Reviewing, http//www.nasonline.org/site/PageServe
  r?pagenameAWARDS_scirev