Searching for Relevant Studies Interactive Case Study Quiz:

1
Searching for Relevant Studies Interactive Case
Study Quiz

• C. Michael White, Pharm.D., FCP, FCCP
• Professor and DirectorUniversity of Connecticut
  / Hartford HospitalEvidence-based Practice
  Center

Author has no actual or potential conflicts of
interest in relation to this activity
2
Interactive Case Quiz Instructions

• Open this presentation as a slideshow. This will
  activate the hyperlinks.
• When you come to a decision slide, choose the red
  box corresponding to the correct choice. If you
  are correct you will be directed forward in the
  case. If you are incorrect, you will be directed
  back to the decision slide to choose again.
• Click on forward hyperlinks (Red Boxes) to follow
  through case based on your responses
• Click on home hyperlink (Blue House Icon) to go
  back to the last correct step in the series

3
Case Study Quiz Searching the Literature

• You are interested in evaluating the benefits and
  harms associated with the use of thrombectomy
  devices in acute coronary syndromes. You need to
  determine what literature is available to conduct
  your review.

4
Participation Quiz Question 1

• You decide to conduct a systematic review (SR).
  What are two ways in which SRs differ from
  narrative reviews?
• They are the same thing
• Systematic reviews use comprehensive searches and
  explicit methods
• Systematic reviews always employ quantitative
  synthesis (meta-analysis

5
They are the Same

• This is incorrect, narrative reviews do not, as a
  rule, employ comprehensive literature searches or
  explicit methods for conducting the review

Click on Blue Box to go Back
6
Comprehensive Search and Explicit Methods

• You are correct! Systematic reviews employ
  comprehensive searches and use explicit methods
  for conducting the review.

SELECT RED BOX
7
Meta-Analysis

• This is incorrect. While some systematic reviews
  employ quantitative synthesis such as
  meta-analysis, others do not.

Click on Blue Box to go Back
8
Participation Quiz Question 2

• So a comprehensive search is required. When
  conducting this search, what do you need to
  assure?
• You balanced precision and recall
• You have gotten every possible relevant citation
  regardless of the workload involved
• That you limit the search so that only relevant
  citations are found

9
Precision and Recall

• This is correct! Precision is the proportion of
  retrieved articles that are relevant while recall
  is the proportion of potentially relevant
  articles retrieved by the search. You want to
  employ methods to maximize precision and recall.
  Overly restrictive searches increase recall but
  compromise precision. Overly broad searches
  increase precision but compromise recall.

SELECT RED BOX
10
Youve Gotten it All

• This is incorrect. While it is laudable to want
  to acquire every possible relevant citation, this
  approach compromises precision and makes the SR
  unnecessarily onerous. You need to balance
  precision and recall.

Click on Blue Box to go Back
11
You Only Have Relevant Citations in your Search

• This is incorrect. Every citation found is
  relevant but it is likely that many relevant
  citations were missed by this overly restrictive
  search strategy. You need to balance precision
  and recall.

Click on Blue Box to go Back
12
Participation Quiz Question 3

• Given the need to balance precision and recall.
  What should you do before initiating a search?
• Enter in the search as thrombectomy AND acute
  coronary syndromes
• Enter the search as thrombectomy OR acute
  coronary syndromes
• Understand the topic, devise an analytic
  framework, pose clearly defined key questions,
  and understand the scope of the review before
  devising your strategy

13
Thrombectomy AND ACS

• This is incorrect. Searching based only on
  readily known disease and treatment information
  would be problematic. You need to understand the
  topic area before starting your search.

Click on Blue Box to go Back
14
Thrombectomy OR ACS

• This is incorrect. This would be a voluminous
  search for citations for thrombectomy anywhere in
  the body in addition to any citation published on
  acute coronary syndromes.

Click on Blue Box to go Back
15
Due Diligence

• This is correct. Understanding the topic and
  devising an analytic framework drive clearly
  defined key questions. Combining this with a
  understanding the scope of the project allows an
  optimal balance between precision and recall when
  sculpting the search.

SELECT RED BOX
16
The Power of Due Diligence

• Your due diligence pays off, you now realize that
  your initial search thrombectomy AND acute
  coronary syndrome would have the compromised
  recall. ((Thrombectomy OR MerciClot.mp OR other
  devices names) AND (acute coronary syndromes OR
  myocardial infarction OR angina, unstable)) would
  assure better recall without appreciably
  impacting precision.

17
Participation Quiz Question 4

• You try to decide whether to do one search or two
  searches (one for benefits and another for
  harms). Which of the following would cause you
  to perform two searches?
• Performing two searches is redundant and should
  be avoided
• You decide a priori to only allow RCT data when
  evaluating benefits but allowing RCTs and
  observational trials for harms
• If the available literature base on the topic is
  extensive

18
Never Do Multiple Searches

• This is incorrect. If the size of the available
  literature is small, it may be appropriate to
  conduct one single broad search. However, if the
  literature is extensive and observational trials
  are allowed in the harms but not the benefits
  evaluations, a single broad search would reduce
  precision for studies reviewing efficacy
  endpoints without appreciably improving the
  recall.

Click on Blue Box to go Back
19
Separate Benefits and Harms Search

• This is correct! Benefits searches is usually
  limited to randomized controlled trials since
  they have greater internal validity. Searches
  usually include the disease in question, the
  intervention, and a hedge for controlled trials.
  Harms searches are usually broader and include
  observational trials. Searches usually include
  the harm (or subheadings for general harms) and
  the intervention. When the available literature
  is extensive, two searches improves precision
  without appreciably impacting recall.

SELECT RED BOX
20
Participation Quiz Question 5

• You need to decide which databases to use. The
  minimum requirement for a thorough search of the
  literature would include which two databases?
• MEDLINE TOXNET
• MEDLINE CINAHL
• MEDLINE Cochrane CENTRAL

21
MEDLINE TOXNET

• This is incorrect. Searching MEDLINE and the
  Cochrane CENTRAL databases are needed to fulfill
  the minimum broad search requirements. TOXNET is
  a database for adverse events or toxicology and
  can be used in addition to these others databases.

Click on Blue Box to go Back
22
MEDLINE CINAHL

• This is incorrect. Searching MEDLINE and the
  Cochrane CENTRAL databases are needed to fulfill
  the minimum broad search requirements. CINAHL is
  a database that focuses on nursing literature and
  can be added to these other databases.

Click on Blue Box to go Back
23
MEDLINE Cochrane CENTRAL

• This is correct. Combining MEDLINE with Cochrane
  CENTRAL would be a minimum requirement for a
  thorough search of the literature. Using MEDLINE
  alone may miss between 12 and 25 of relevant
  citations found in other databases and may
  produce a North American bias.

SELECT RED BOX
24
Participation Quiz Question 6

• You decide to use hand searching of references
  from identified systematic reviews, studies, and
  abstract booklets from prominent meetings within
  the field of interest. Why would you do this?
• Hand searching can capture citations that are not
  indexed or are improperly indexed.
• Limiting hand searching to these sources can
  target this time intensive activity to areas
  where the yield will be the greatest.

25
Both are Correct

• You are correct! Hand searching can yield up to
  13-25 of citations that do not come up in
  database searches. Since hand searching is the
  most time intensive search strategy, it should be
  targeted to maximize the yield.

SELECT RED BOX
26
Participation Quiz Question 7

• You decide to search the FDA website and
  ClinicalTrials.gov to identify studies that might
  be appropriate for your systematic review. These
  website are an example of
• Black literature
• Grey literature
• SIPs

27
Black Literature

• This is incorrect. You chose the wrong color.

Click on Blue Box to go Back
28
Grey Literature

• This is correct! Grey literature can be important
  for identifying trials that were conducted but
  are, as of now, unpublished or incompletely
  published. Searching for grey literature is a
  useful way to minimize publication bias.

SELECT RED BOX
29
SIPs

• This is incorrect. SIPs are Scientific
  Information Packets provided by the manufacturer
  of a drug or device.

Click on Blue Box to go Back
30
Participation Quiz Question 8

• You conduct a thorough search of MEDLINE and
  Cochrane CENTRAL, use two searches (one for
  benefits and another for harms, and employ
  targeted hand searching and grey literature
  evaluation. What is the last thing you need to
  remember to do germane to searching.
• Only choose citations that agree with your
  preconceived notions
• Selectively report aspects of your search
  strategy
• Provide transparent reporting of search
  strategies and the citations you identified

31
Preconceived Notions

• This is incorrect. This would be akin to a
  narrative review and should not be used in a
  systematic review.

Click on Blue Box to go Back
32
Selective Reporting

• This is incorrect. In order to build confidence
  in the systematic review results, transparent and
  defensible methods need to be employed. You need
  to report enough information about your search
  strategy so that a person, skilled in the art,
  could reproduce it.

Click on Blue Box to go Back
33
Transparent Reporting

• This is correct! Transparent reporting is
  important in supporting the credibility of
  systematic review results. Reporting of search
  strategies and disposition of the citations
  should allow a person, skilled in the art, to
  reproduce the search you conducted.

SELECT RED BOX
34
Congratulations!

• You have successfully maneuvered through the
  pitfalls of literature searching. A comprehensive
  understanding of the topic, the use of multiple
  searches, multiple databases, use of hedges, use
  of hand searching and grey literature, and
  development of a transparent reporting structure
  will give readers confidence in your systematic
  review.