Effective Search Strategies for Systematic Reviews of Medical Tests presentation

About This Presentation

Title:

Effective Search Strategies for Systematic Reviews of Medical Tests

Description:

Effective Search Strategies for Systematic Reviews of Medical Tests Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Medical ... –

Number of Views:276

Avg rating:3.0/5.0

Slides: 35

Provided by: AlexP172

Learn more at: http://www.baylorcme.org

Category:

more less

Transcript and Presenter's Notes

Title: Effective Search Strategies for Systematic Reviews of Medical Tests

1
Effective Search Strategies for Systematic
Reviews of Medical Tests

Prepared for
The Agency for Healthcare Research and Quality
(AHRQ)
Training Modules for Medical Test Reviews Methods
Guide
www.ahrq.gov

2
Overview of a Medical Test Review
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
3
Learning Objectives

Recognize the importance of a multiple-approach
method of searching
Understand the current state of indexing and
reporting and its effect on the most pragmatic
methods of conducting searches

The overall goal is to locate all published
studies relevant to key questions.
General guidance on searching for systematic
reviews also applies to reviews of medical tests.
A balance must be achieved between recall and
precision
Recall how much of the relevant literature is
retrieved.
Precision how much of the retrieved literature
is relevant.
An optimal balance depends on the context. For
comparative effectiveness reviews, the goal is to
conduct a comprehensive and exhaustive search
while minimizing the resources needed to review
retrieved citations.

Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Healthcare Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
5
General Search Guidance (2 of 3)

Develop the search strategy for the study with
the research team according to the analytic
framework and the inclusion and exclusion
criteria.
Search electronic databases for applicable
studies
MEDLINE? and the Cochrane Central Register of
Controlled Trials should always be included in
the search strategy.
Other useful databases include EMBASE ?, CINAHL?,
and PsycINFO?.
Tailor the search to each database.
Use both controlled (i.e., MeSH ? subject
headings) and uncontrolled vocabulary.
Use delimiters such as language only if a
specific case can be made.

Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Healthcare Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
6
General Search Guidance (3 of 3)

Identifying additional studies
References from studies included for review
Citation databases (e.g., SciVerse? Scopus or Web
of Science ?)
Unpublished or grey literature, such as
Clinical trial registries (e.g.,
ClinicalTrials.gov and the World Health
Organization International Clinical Trials
Registry Platform)
Conference proceedings/abstracts (e.g., ProQuest
COS Conference Papers Index)

Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
. Agency for Healthcare Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Reviews. Available at
www.effectivehealthcare.ahrq.gov/methodsguide.cfm.
7
Challenges Specific to Searches forStudies of
Medical Tests (1 of 3)

Searches are required on each relevant test
strategy being considered.
One of two approaches is used to conduct the
search
Search all possible tests used to evaluate the
disease. This requires knowledge of all possible
test strategies available.
Search on the disease or condition, and then
filter for the medical test.
Searching is often an iterative process,
requiring refinements to search criteria.

The first approach for identifying studies of
medical tests is to search for specific named
tests.
Benefits
It is straightforward test names are used to
locate studies.
A specific search for the concept of
diagnosis/screening/ prognosis may not be
necessary.
Drawbacks
New approaches may be missed because of evolving
test strategies.
Using the test name alone may be insufficient,
especially for gene product tests that are
associated with multiple diseases.

The second approach for identifying studies of
medical tests is to search for a specific disease
or condition.
Benefits
This approach is preferred if specific medical
tests are unknown.
It can be used in combination with searches for
known test names.
Drawback Searches are broad and increase the
amount of work needed to filter down to the
relevant studies.
This approach is used more often by reviewers.

Do not rely on search filters alone.
Do not rely on controlled vocabulary (subject
headings) alone.
Search in multiple locations.

Search filters (or hedges)
Are pre-prepared and tested searches.
Can be combined with searches on a particular
disease or condition.
Most search filters have been developed for
MEDLINE?.
Additional search filters have been developed
specifically for diagnostic imaging and EMBASE ?.
Currently, search filters are inappropriate for
systematic reviews because they
May miss relevant studies.
Do not perform as well for studies published
before 1990 because of nonstandardized reporting
and indexing of medical test studies, which may
improve in the future.

Example Filter used by PubMed? Clinical Queries
Category selected Diagnosis
Two choices of scope to optimize search Broad
or Narrow

Category Optimization of Search Sensitivity/Specificity of Search PubMed? Search String
Diagnosis Broad Sensitivity/breadth (of Diagnosis search) 98/74 (sensitivTitle/Abstract OR sensitivity and specificityMeSH Terms OR diagnosTitle/Abstract OR diagnosisMeSHnoexp OR diagnostic MeSHnoexp OR diagnosis,differentialMeSHnoexp OR diagnosisSubheadingnoexp)
Diagnosis Narrow Specificity/narrowness (of Diagnosis search) 64/98 (specificityTitle/Abstract)
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
13
Search Principle 2 Do Not Rely on Controlled
Vocabulary (Subject Headings) Alone

It is important to use all known variants of a
test name, as in the examples below that refer to
hematocrit
Abbreviations (Hct, Crit, PCV)
Generic names (hematocrit, packed cell volume)
Proprietary names (e.g., LighTouch HCT)
International terms/spellings (haematocrit)
Controlled vocabulary may not contain all
variants.
Controlled vocabulary is unreliable due to
nonstandardized reporting and indexing.
Textwords help identify tests not yet indexed or
indexed improperly.
Filters may suggest appropriate textwords.

Searching more than one database and tailoring
searches to each is always advisable. Otherwise,
there is a risk of bias due to little overlap
between the many databases.
Until reporting/indexing is standardized, the
best method is to combine highly sensitive
searches and manual article screening.
Articles can still miss relevant articles, so
additional sources of information are necessary.
Other sources of information
Additional bibliographic databases
Citation tracking (i.e., reading through
references of relevant articles)
Regulatory documents
Search for diagnostic documents at Devices_at_FDA, a
U.S. Food and Drug Administration Web site.

Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
15
Search Principle 3 Search in Multiple Locations
(2 of 4)
Examples of specialized databases that have free
access
Open-Access Databases Open-Access Databases Open-Access Databases
Database URL Topic Coverage
IBIDS (International Bibliographic Information on Dietary Supplements) www.ods.od.nih.gov/Health_ Information/IBIDS.aspx Dietary supplements
PubMed? www.ncbi.nlm.nih.gov/pubmed Biology Health sciences
TOXNET (Toxicology Data Network) www.toxnet.nlm.nih.gov Toxicology Environmental health adverse effects
World Health Organization Global Health Library www.who.int/ghl/medicus/en/ International biomedical topics Global Index Medicus
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
16
Search Principle 3 Search in Multiple Locations
(3 of 4)
Examples of specialized databases that require a
subscription
Subscription Databases Subscription Databases Subscription Databases
Database URL Topic Coverage
CINAHL? (Cumulative Index to Nursing and Allied Health) www.ebscohost.com/cinahl Nursing Allied health
EMBASE? www.embase.com Biomedical with emphases on drugs and pharmaceuticals More nonU.S. coverage than MEDLINE?
IPA (International Pharmaceutical Abstracts) www.csa.com/factsheets/ipa-set-c.php Drugs and pharmaceuticals
MANTIS (Manual Alternative and Natural Therapy Index System) www.healthindex.com/MANTIS.aspx Osteopathy Chiropractic Alternative medicine
PsycINFO? www.apa.org/pubs/databases/psycinfo/ index.aspx Psychological literature
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
17
Examples of citation-tracking databases
Search Principle 3 Search in Multiple Locations
(4 of 4)
Citation-Tracking Databases Citation-Tracking Databases Citation-Tracking Databases
Database URL Access
Google Scholar www.scholar.google.com Free
PubFocus www.pubfocus.com Free
PubReMiner bioinfo.amc.uva.nl/human-genetics/pubreminer Free
SciVerse? Scopus www.info.scopus.com Subscription required
Web of Science? thomsonreuters.com/products_services/science/ science_products/a-z/web_of_science Subscription required
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Methods
guide for medical test reviews. Available at
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
18
Examples of Search Strategies

AHRQ Evidence Report/Technology Assessment No.
142 Testing for BNP and NT-proBNP in the
Diagnosis and Prognosis of Heart Failure
All medical tests were known.
The search consisted of all possible variations
on test names.
Using a search string to capture the diagnostic
testing concept was not necessary.
AHRQ Comparative Effectiveness Review No. 2
Effectiveness of Noninvasive Diagnostic Tests for
Breast Abnormalities
All medical tests were not known.
The PubMed? search string relied on textwords
used to capture the diagnostic testing concept
diagnosis OR diagnose OR diagnostic OR dish OR
gold standard OR ROC OR receiver operating
characteristic OR sensitivity and
specificitymh OR likelihood OR false positive
OR false negative OR true positive OR true
negative OR predictive value OR accuracy OR
precision

Diagnostic search filters cannot be depended on
exclusively.
When the full range of tests is known, searching
for a specific test using all variant names may
be sufficient.
Combining highly sensitive database searches
(using textwords), hand searching, and searching
lists of cited references is currently the best
way to identify relevant studies.
Do not rely on controlled vocabulary alone.
Search multiple locations and databases.
Be sure to check the Devices_at_FDA Web site, which
is a useful source.

Generally, when designing a literature search,
higher precision is desirable relative to recall.
True
False

21
Practice Question 1 (2 of 2)

Explanation of Question 1
The statement is false. Maintaining balance
between recall and precision is usually
desirable.

22
Practice Question 2 (1 of 2)

Search filters are inappropriate for systematic
reviews.
True
False

23
Practice Question 2 (2 of 2)

Explanation of Question 2
The statement is true. Search filters may miss
relevant studies and do not perform as well for
studies published before 1990 because of
nonstandardized reporting and indexing.

24
Practice Question 3 (1 of 2)

Which of the following are true statements
regarding controlled vocabulary
A controlled vocabulary may not contain all
variants.
A controlled vocabulary is reliable thanks to
standardized reporting and indexing.
Textwords help identify tests not yet indexed or
indexed improperly.
Choices
a and b
a and c
b and c
All three statements are true.

25
Practice Question 3 (2 of 2)

Explanation of Question 3
The correct answer is b. Controlled vocabulary is
actually unreliable for identifying variants of a
test name because of nonstandardized reporting
and indexing.

26
Practice Question 4 (1 of 2)

Which of the following is not a principle of
searching for studies outlined in this module
Do not rely on search filters alone.
Do not rely on controlled vocabulary alone.
Search in multiple locations.
Avoid using Google Scholar as a search tool.

27
Practice Question 4 (2 of 2)

Explanation of Question 4
The correct answer is d. Google Scholar can be
useful for grey literature searches.

28
Authors

This presentation was prepared by Brooke
Heidenfelder, Rachael Posey, Lorraine Sease, Remy
Coeytaux, Gillian Sanders, and Alex Vaz, members
of the Duke University Evidence-based Practice
Center.
The module is based on Chapter 4, Effective
Search Strategies for Systematic Reviews of
Medical Tests. In Methods Guide for Medical Test
Reviews. AHRQ Publication No. 12-EC017.
Rockville, MD Agency for Healthcare Research and
Quality June 2012. www.effectivehealthcare.ahrq.g
ov/medtestsguide.cfm

29
References (1 of 6)

Agency for Healthcare Research and Quality.
Methods Guide for Effectiveness and Comparative
Effectiveness Review. Rockville, MD Agency for
Healthcare Research and Quality April 2012. AHRQ
Publication No. 10(12)-EHC063-EF. Chapters
available at www.effectivehealthcare.ahrq.gov/meth
odsguide.cfm.
Astin MP, Brazzelli MG, Fraser CM, et al.
Developing a sensitive search strategy in MEDLINE
to retrieve studies on assessment of the
diagnostic performance of imaging techniques.
Radiology. 2008 May247(2)365-73. PMID
18372447.
Bachmann LM, Coray R, Estermann P, et al.
Identifying diagnostic studies in MEDLINE
reducing the number needed to read. J Am Med
Inform Assoc. 2002 Nov-Dec9(6)653-8. PMID
12386115.
Bachmann LM, Estermann P, Kornenberg C, et al.
Identifying diagnostic accuracy studies in
EMBASE. J Med Libr Assoc. 2003 Jul91(3)341-6.
PMID 12883560.

30
References (2 of 6)

Balion C, Santaguida PL, Hill S, et al. Testing
for BNP and NT-proBNP in the Diagnosis and
Prognosis of Heart Failure. Evidence
Report/Technology Assessment No. 142 (Prepared by
the McMaster University Evidence-based Practice
Center under Contract No. 290-02-0020).
Rockville, MD Agency for Healthcare Research and
Quality September 2006. AHRQ Publication No.
06-E014. Available at www.ncbi.nlm.nih.gov/books/N
BK38136.
Betran AP, Say L, Gulmezoglu AM, et al.
Effectiveness of different databases in
identifying studies for systematic reviews
experience from the WHO systematic review of
maternal morbidity and mortality. BMC Med Res
Methodol. 2005 Jan 285(1)6. PMID 15679886.
Bruening W, Launders J, Pinkney N, et al.
Effectiveness of Noninvasive Diagnostic Tests for
Breast Abnormalities. Comparative Effectiveness
Review No. 2 (Prepared by the ECRI Evidence-based
Practice Center under Contract No. 290-02-0019).
Rockville, MD Agency for Healthcare Research and
Quality February 2006. AHRQ Publication No.
06-EHC005-EF. www.effectivehealthcare.ahrq.gov/rep
Files/BrCADx20Final20Report.pdf.

31
References (3 of 6)

Conn VS, Isaramalai SA, Rath S, et al. Beyond
MEDLINE for literature searches. J Nurs
Scholarsh. 200335(2)177-82. PMID 12854300.
Deville WL, Bezemer PD, Bouter LM. Publications
on diagnostic test evaluation in family medicine
journals an optimal search strategy. J Clin
Epidemiol. 2000 Jan53(1)65-9. PMID 10693905.
Doust JA, Pietrzak E, Sanders S, et al.
Identifying studies for systematic reviews of
diagnostic tests was difficult due to the poor
sensitivity and precision of methodologic filters
and the lack of information in the abstract. J
Clin Epidemiol. 2005 May58(5)444-9. PMID
15845330.
Haynes RB, Wilczynski NL. Optimal search
strategies for retrieving scientifically strong
studies of diagnosis from Medline analytical
survey. MBJ. 2004 May 1328(7447)1040. PMID
15073027.
Haynes RB, Wilczynski N, McKibbon KA, et al.
Developing optimal search strategies for
detecting clinically sound studies in MEDLINE. J
Am Med Inform Assoc. 1994 Nov-Dec1(6)447-58.
PMID 7850570.

32
References (4 of 6)

Honest H, Bachmann LM, Khan K. Electronic
searching of the literature for systematic
reviews of screening and diagnostic tests for
preterm birth. Eur J Obstet Gynecol Reprod Biol.
2003 Mar 26107(1)19-23. PMID 12593888.
Leeflang MM, Scholten RJ, Rutjes AW, et al. Use
of methodological search filters to identify
diagnostic accuracy studies can lead to the
omission of relevant studies. J Clin Epidemiol.
2006 Mar59(3)234-40. PMID 16488353.
Relevo R. Effective search strategies for
systematic reviews of medical tests. In Chang SM
and Matchar DB, eds. Methods guide for medical
test reviews. Rockville, MD Agency for
Healthcare Research and Quality June 2012. p.
4.1-4.9. AHRQ Publication No. 12-EHC017.
www.effectivehealthcare.ahrq.gov/medtestsguide.cfm
.
Ritchie G, Glanville J, Lefebvre C. Do published
search filters to identify diagnostic test
accuracy studies perform adequately? Health Info
Libr J. 2007 Sep24(3)188-92. PMID 17714173.

33
References (5 of 6)

Sampson M, Barrowman NJ, Moher D, et al. Should
meta-analysts search EMBASE in addition to
MEDLINE? J Clin Epidemiol. 2003 Oct
56(10)943-55. PMID 14568625.
Stevinson C, Lawlor DA. Searching multiple
databases for systematic reviews added value or
diminishing returns? Complement Ther Med. 2004
Dec12(4)228-32. PMID 15649836.
Suarez-Almazor ME, Belseck E, Homik J, et al.
Identifying clinical trials in the medical
literature with electronic databases MEDLINE
alone is not enough. Control Clin Trials. 2000
Oct21(5)476-87. PMID 11018564.
van der Weijden T, Ijzermans CJ, Dinant GJ, et
al. Identifying relevant diagnostic studies in
MEDLINE. The diagnostic value of the erythrocyte
sedimentation rate (ESR) and dipstick as an
example. Fam Pract. 1997 Jun14(3)204-8. PMID
9201493.

34
References (6 of 6)

Vincent S, Greenley S, Beaven O. Clinical
evidence diagnosis developing a sensitive search
strategy to retrieve diagnostic studies on deep
vein thrombosis a pragmatic approach. Health
Info Libr J. 2003 Sep20(3)150-9. PMID
12919278.
Whiting P, Westwood M, Beynon R, et al. Inclusion
of methodological filters in searches for
diagnostic test accuracy studies misses relevant
studies. J Clin Epidemiol. 2011 Jun64(6)602-7.
PMID 21075596.
Whiting P, Westwood M, Burke M, et al. Systematic
reviews of test accuracy should search a range of
databases to identify primary studies. J Clin
Epidemiol. 2008 Apr61(4)357-64. PMID 18313560.
Wilczynski NL, Haynes RB. EMBASE search
strategies for identifying methodologically sound
diagnostic studies for use by clinicians and
researchers. BMC. Med 2005 Mar37. PMID
15796772.
Zheng MH, Zhang X, Ye Q, et al. Searching
additional databases except PubMed are necessary
for a systematic review. Stroke. 2008
Aug39(8)e139. PMID 18583557.

Write a Comment

User Comments (0)

About PowerShow.com