Research Utilization: and its many determinants

1
Research Utilization and its (many) determinants
Carole A. Estabrooks Professor, Faculty of
Nursing Canada Research Chair in Knowledge
Translation University of Alberta
University of Melbourne Melbourne, AU October,
2005
2
(No Transcript)
3
Edmonton, Alberta
4
University of Alberta
5
(No Transcript)
6
PARiHS Framework for Research Implementation
Evidence-based practice

Context


Facilitation
Evidence
PARiHS Model (Kitson, et al., 1998, QSHC
7
Policy Framework
Policy Analysis Framework
Lomas, 2000
8
First what do we know about the determinants of
research utilization?
9

• Common barriers encountered in knowledge transfer
  in hospital settings

• Commonly identified barriers include
• Time
• Resources
• Support
• Knowledge skills
• Adequate evidence
• Culture, context, environment.

• Less commonly identified barriers include
• Disincentives
• Irrationality of decision-making environment
• Decision-making errors
• Attitudes beliefs
• Inability to see consequences

10
Best strategies for transferring research
findings?

• Imperfect evidence base for decision makers
• Many current rigorous evaluations have
  methodological weaknesses
• Poor reporting of study settings, barriers to
  change, content and rationale of intervention
• Generalisability of study findings is frequently
  uncertain
• Reminders most consistently observed to be
  effective
• Educational outreach only led to modest effects
• Dissemination of educational materials may lead
  to modest but potentially important effects
  (similar effects to more intensive interventions)
• Multifaceted interventions not necessarily more
  effective than single interventions

Grimshaw JM, Thomas RE, MacLennan G, Fraser C,
Ramsay C, Vale L et al. Effectiveness and
efficiency of guideline dissemination and
implementation strategies. Health Technol Assess
2004. (Available from http//www.hta.nhsweb.nhs.u
k/)
11
From some of our work in KUSP

• In nursing
• Most common interventions tend to be education,
  protocol guideline implementation
• Little audit feedback, this is emphasized more
  among physicians
• Individual patient information, personal
  experience, and social interactions are main
  information sources
• Limited use of journals, textbooks, and other
  media, including the internet

1
12
Scott-Findlays doctoral work on culture and
RU (in progress)

• Structure of Authority
• Nature of nurses work
• location of work
• expected level of activity
• routinized work
• expected ways to get work done
• Workplace climate
• ethical issues
• the response to errors
• the receptivity to innovation
• Types of knowledge valued
• clinical knowledge (practical, often tacit)
• specialized knowledge (e.g., PICU)
• research

13
Scott-Findlay (in progress)
14
From work in KUSP
For interventions we also need to consider (from
ethnographic case studies)

• Information needs and how these relate to the
  sources of knowledge used in practice
• How the knowledge sources used bear on the
  packaging and dissemination of evidence
• How dissemination strategies need to focus on
  what is needed and useful regardless of what we
  perceive may be the most relevant approach
• A focus on those determinants of research use
  with the potential to be manipulated in order to
  develop interventions with potential utility

4
15
From work in KUSP
Organizationally (causal and multi-level
modeling)

• nursing control over practice
• opportunity for nurse-to-nurse collaboration
• support for innovation
• facilitation of research use
• positive culture (characterized by good
  leadership, evaluation and performance feedback)
• increased time to nurse (-ve)
• emotional exhaustion (-ve)

• Higher nurse education levels
• Greater control over practice
• Greater freedom to make decisions
• Having a good nursing leader on unit
• Low levels of emotional exhaustion
• A sense of personal accomplishment

5
16
KUSP Studies

• Theorizing the determinants of research
  utilization (2000 2005)
• KUPI - The knowledge utilization policy
  implementation program (2002 2007)
• The Bibliometric study (2004 2006)
• Measuring research utilization (2005 2008)
• New and future studies

17

• Theorizing the determinants of research
  utilization (2000 2005)

Research Utilization Studies
The Determinants of Research Utilization Pain
Management in Adults Children
Purpose To understand how nurses use research in
their practice and what influences their ability
to use or not use research.
18
METHODS Ethnographic case studies

• Four hospitals (two in Ontario, two in Alberta)
• Adult and pediatric surgical units
• Qualitative and quantitative data collected
• Six months of participant observation each unit
• Nurses, patients, physicians, social workers,
  physiotherapists, and administrators participated

19
Data Sources

• Documents and records
• Interviews (individual group)
• Participant observations
• Physical artifacts
• Measures (e.g., Research utilization, unit
  culture, critical thinking, pain, workload,
  environmental complexity)

20
Quantitative findings (selected - sources only)

• In this study the patient care unit was unit of
  analysis
• Our goal to understand the role of context in
  shaping RU
• We mapped relationship of contextual factors
  using correspondence analysis

21
Quantitative findings (contd)

• Units with highest RU clustered together on
  factors such as
• Critical thinking
• Unit culture

• Workload and people support clustered more
  closely to units with lowest RU scores

22
The 7 Units on knowledge sources
23
(No Transcript)
24
Number of Sources Used by Nurses

• 20 of nurses use 1 5 sources
• 50 of nurses uses 6 10 sources
• 25 of nurses use 11-15 sources

25
7 Unit Comparison

• Overall, few differences in the sources used by
  nurses across units

26
Sources of Knowledge Patterns Over Time
Note. Mean scores were transformed to fit a
five-point likert scale (0-4), with 0 as never
and 4 as always.
27
Qualitative findings (selected - sources only)
Main Categories of Knowledge Sources

• Social interactions
• Experience
• Documents
• Intra-Personal

28
Taxonomy of nurses sources of knowledge
29
Why?  

• Time
• Nurses typically work not only on multiple
  schedules high and frequent levels of
  resequencing are required
• Context
• Nurses tend to be motivated to seek knowledge
  when they have context specific patient care
  situations, problems to deal with
• Trust
• Nurses tend to seek information , knowledge from
  those they trust (knowledgeable, empathetic)
• Hierarchy
• An environment where norms of organizational
  hierarchy are strictly adhered to may negatively
  affect knowledge transfer between nurses and
  CNSs, clinical leaders, and physicians.

30
Summary Conclusions

• Individual patient information, personal
  experience, and social interactions are main
  information sources
• Limited use of journals, textbooks, and popular
  media, including the internet
• The structure and organization of nurses work
  has a major influence on their choice of
  knowledge sources
• Researchers need to merge nurses
  information-seeking behavior with dissemination
  strategies

31
Summary Conclusions

• The nurses in this study overwhelmingly relied on
  social interactions and experience as their
  primary sources of practice knowledge
• Time, trust, and unit environment played an
  important role in determining the nurses sources
  of knowledge

32

• KUPI The knowledge utilization policy
  implementation program (2002 2007)

33
Evidence-based practice

• PARISH
• Promoting Action on Research Implementation in
  Health Services
• SI f (E,C,F)
• Successful implementation is a function of
  evidence, context and facilitation
• Kitson et al, 1998
• Rycroft-Malone et al, 2002

Context


Facilitation
Evidence
34
Knowledge utilization and policy implementation
(KUPI)
A five year research program whose purpose is to
develop theory in the knowledge utilization field
that increases (research) knowledge use by
clinicians and decision-makers across multiple
levels - by using existing data

35
Objectives of the larger KUPI program

• To study
• how predictors of research use vary across levels
  of decision-making
• how research use is accomplished in
  organizations, and
• how research use shapes policy implementation

36
Modeling Sub-Projects
Designed to study factors influencing knowledge
use, specifically research utilization (RU), at
different levels of decision-making among nurses
in Alberta hospitals.
37
Modeling sub-project Five studies

• DV derivation outcome validation
• A structural equation (SEM) assessment of the
  PARiSH (Kitson) model
• Multi-level (HLM) modeling
• Hierarchical structural equation modeling (H-SEM)
• A methodological evaluation of HLM vs. H-SEM and
  their usefulness for the study of research
  utilization

38
Example Multi-level (HLM) study objectives

• To determine which nursing factors at the
  individual, unit, and hospital levels predict
  research utilization
• To measure the contribution of the significant
  factors to variation in research utilization
  across units and hospitals

39
The Alberta Nurse Survey (1998)
A census population of all 12,345 registered
nurses from 129 hospitals in Alberta, Canada.
Useable returns 6526 (52.8)
A Employment Characteristics B Nursing Work
Index (NWI) C Maslach Burnout Inventory(MBI)
D Job characteristics E Last shift F
Demographics G Site specific questions
40
The Research Utilization Survey (1996)
A randomly selected sample of staff nurses in
Alberta, Canada. Useable returns 600 (42)

• Research utilization
• Background and daily demands
• Individual and professional factors
• The research itself
• Your organization

41
Final Sample
Hospitals Nurses 109
6,526 90 5,228 Selection criteria
5 nurses per hospital
42
The 3 level model
Organization factors (hospital level)
Specialty factors (unit level)
Nurse factors (individual level)
43
Significant univariate predictors Individual,
specialty (unit), and hospital levels

• Individual level
• Higher nurse education
• Control over practice
• Freedom to make decisions
• Emotional exhaustion
• Lack of personal accomplishment
• Specialty (unit) level
• Leadership
• Relational capital
• Consulting opportunity
• Participate in policy
• Organizational level
• Climate
• Innovativeness
• Organizational slack

44
Summary
Greater research utilization among nurses was
predicted in the final model by

• Higher nurse education levels
• Greater control over practice
• Greater freedom to make decisions
• Having a good nursing leader on unit
• Low levels of emotional exhaustion
• A sense of personal accomplishment

45
Components () of research explained at the
individual, specialty (unit), and hospital levels
4.5
3.7
91.8
46
Summary
Variation in research utilization was mainly due
to differences in individual characteristics of
nurses, with organizational factors contributing
less. Of the latter, however, having good
leadership on the unit was the most significant
determinant of research utilization
47
Structural equation (SEM) modeling objective

• to develop a theoretical model of hospital
  characteristics that predict research use by
  nurses

48
The Structural Equation Model (SEM)
49
Structural Equation Modeling Results

• Positive impact on research use
• nursing control over practice
• opportunity for nurse-to-nurse collaboration
• support for innovation
• facilitation of research use
• positive culture (characterized by good
  leadership, evaluation and performance feedback)
• Negative impact on research use
• Increased time to nurse
• emotional exhaustion

50

• The Bibliometric study (2004 2006)

51

• The Bibliometric study (2004 2006)

• Objectives
• Map the development of the field of knowledge
  utilization
• Identify the structure of the scientific
  community
• Provide direction to investigators
  decision-makers in the field

52
Terminology Tangle
53
Pilot study A bibliometric analysis of the RU
literature in nursing

• 600 articles
• Findings/Conclusions
• Limited collaborations
• Primarily opinion pieces with little research or
  theoretical work
• UK US most prolific in the field
• Increased productivity since early 1990s with
  change in terminology

Estabrooks, C.A., Winther, C., Derksen, L.
(2004). Mapping the field. A bibliometric
analysis of the research utilization literature
in nursing. Nursing Research, 53(5), 293-303.
54
(No Transcript)
55
Larger Study Methods

• Overview
• Web of Science 1945-2004
• Articles related to KU in general
• Three case studies related to KU
• Guidelines
• Policy management
• Nursing

56
Preliminary findings
Key authors 15 most cited authors
57
Key journals

• Journal of Evaluation in Clinical Practice
• Journal of Advanced Nursing
• British Medical Journal
• Knowledge Creation, Diffusion, Utilization
• Lancet
• Journal of General Internal Medicine
• Research Policy
• Journal of the American Medical Association
• Science Communication
• Social Science Medicine

58

• Measuring research utilization (2005
  2008)

59
Systematic Review Findings
Existing measures

• Nursing Practice Questionnaire (NPQ)
• NPQ (Modified)
• Research Utilization Questionnaire (RUQ)
• Edmonton Research Orientation Survey (EROS)
• Other multi-item measures (N3)
• Single item measures (N19)

60
Systematic Review Findings

• Poor construct clarity
• Lack of theoretical framing
• Lack of measurement theory
• Lack of psychometric assessment
• Presumption of linearity
• Absence of longitudinal work

• Self-report
• Recall
• Social desirability
• Scaling
• Unit of analysis

61
Instrument Development Study

• Objectives
• to clarify and validate the construct
• to develop observable indicators
• to develop a set of items that measure research
  use
• to conduct a pilot test of the instrument

62
Instrument Development study

• Approach
• Construct clarity ? Series of focus groups with
  three sets of experts
• Local (research team) expert panel (complete)
• International panel (May 2005)
• Managers and educators
• Providers (RNs, LPNs, assistants, aides)
• Indicator and item development ? Series of focus
  groups with two sets of experts
• Managers and educators
• Providers of nursing care (RNs, LPNs, RPNs,
  assistants, aides)
• Pilot test the draft instrument(s)

63
Action in Practice
Action
Thinking (i.e., Clinical Decision
Making Clinical Reasoning )
Instrumental Use
Task completion
Research Use
Instrumental Use
Task completion Relational Assessment Screening
Research Non-Use
Non-use
A. Informed (non-defensible)
Non-use
B. Informed (defensible)
Non-use
Uninformed
64
HI
Research utilization
Conceptual
Instrumental
HI
LO
LO
65

• New and future studies

• Health research utilization in hospitals (start
  up)
• Under review
• SHARP (Share and House Accessible Research
  Products)
• TROPIC team grant
• (KUSP) Team grant (letter of intent stage) The
  influence of context on research use (long term
  care settings)

66
Health research utilization in hospitals (start
up) (2nd in a series of 3-4 studies)
Objectives of the first (completed) study

• To describe two groups (researchers and users)
• To compare research dissemination behaviour among
  the researcher sub-groups
• To compare research utilization behaviour among
  the users sub-groups
• To explore which factors contribute to research
  dissemination (researchers) and research use
  (users)

67
Key Messages (from the first study)

• About Researchers
• Research infrastructure matters
• Not all academic researchers are equal
• Researchers do contribute to research use
• About Users
• Organizational capacity matters
• Intermediaries matter
• Talking with researchers matters

68
Engaged Dissemination By Group
Plain Dissemination By Group
Number of Publication By Group
69
Correlations
Correlation is significant at the 0.01 level
(2-tailed). Correlation is significant at the
0.05 level (2-tailed).
70
Research Utilization Index

• Summary of Comparisons
• There is significant difference across groups.
• Multiple comparison test revealed 4 homogeneous
  subsets nurse providers, nurse decision-makers
  and other decision-makers, other decision-makers
  and intermediaries and intermediaries and
  physician providers.
• The physicians are significantly different from
  all nurses and other decision-makers.

71
Factors Influencing Dissemination and Publication
Plain Dissemination Contribution by users,
Perceived impact, Perceived importance of
dissemination activities (B), Recognition of
expertise, Research focus users need Engaged
Dissemination Perceived impact, Perceived
importance of dissemination activities (B),
Research focus users need Publication Number
of research personnel, Research focus scholarly
advancement, Academic rank, Cost of utilization
72
Present Study

• Objectives
• to determine the feasibility of undertaking a
  larger scale study in Western Canada
• To validate a typology of providers To validate a
  revised survey instrument assessing individual
  and organizational characteristics
• To identify predictors of research utilization
• To assess provider group differences and
  similarities

73
The SHARP grant (infrastructure)
74
Why SHARP?
Preserves data for future analysesFacilitates
researcher collaborationIncreases effectiveness
of fundingProvides opportunities for mentoring
and education in data managementReduces
duplication of data collection
75
TROPIC
Purpose to ameliorate pain in children by
narrowing the gap between clinical practice and
the research evidence supporting optimal patient
care. Using the PARIHS framework
Main applicants Bonnie Stevens (PI), University
of Toronto Celeste Johnston, McGill
University Shoo Lee, University of Alberta Carole
Estabrooks, University of Alberta Shannon
Scott-Findlay, University of Alberta Patrick
McGrath, Dalhousie University Christine Chambers,
Dalhousie University
76
Figure 1. Working model of research
utilization (Estabrooks, Scott-Findlay
Cummings, 2005)
77
Three major TROPIC projects

• The creation of a centralized Canadian Pediatric
  Pain Research Network Database (e.g., collect
  demographics and information about pain)
• Assess organizational context and determine its
  impact on the implementation of pain research in
  pediatric clinical practice
• Evaluate the efficacy and effectiveness of (a)
  Evidence-based Practice In Change (EPIC Lee et
  al., 2002) and (b) Audit and Feedback

78

• Culture
• OCI, Cooke Lafferty (1987)
• Leadership
• Leadership Practice Inventory (LPI) Kouzes,
  Posner (2002)
• Leadership Empowering Behaviours (LEB) Hui (1994)
• Evaluation
• Focused qualitative data
• Other overall contextual measures
• Maslach Burnout Inventory
• Nursing Work Index - Revised (NWI R)
• Environmental Complexity Scale (ECS)
• Project Research in Nursing (PRN) Workload
  Measurement Instrument
• Research Utilization (revised)
• Research Utilization Survey (Estabrooks 1997)

79

• A main goal of the analyses will be to assess the
  role of unit contextual variables on nurse
  research utilization behaviors
• Additionally, to estimate the independent effects
  of these contextual variables, we have proposed
  Hierarchical Linear Modeling (HLM) and Structural
  Equation Modeling (SEM)

80
(KUSP) Team LOI (tentative and evolving)

• Carole Estabrooks (PI, Canada Research Chair)
• Lesley Degner (CHSRF Chair)
• Michael Leiter (Canada Research Chair)
• Heather Laschinger (UWO, leadership)
• Greta Cummings (UA, leadership)
• Joanne Profetto-McGrath (UA, facilitation)
• Malcolm Smith (Business, Univ of Manitoba)
• Laurel Strain (Gerontology, sociology)
• UK
• Joanne Rycroft-Malone
• Sue Dopson
• USA
• Anne Sales
• And others tba

81
Examining the context dimension of the PARIHS
framework in LTC settings
82
www.ualberta.ca/kusp