Assessing Organizational Ability for Technology Transfer:

1
Assessing Organizational Ability for Technology
Transfer The Medical Organizational Readiness
for Change Survey TM Bohman, RT Spence, M
Murphy-Smith, M Blount, A Yeung, M Wilkins, V
Waters, K McQueen University of Texas at Austin,
Baylor College of Medicine Harris County
Hospital District
Introduction
Findings
This study explores the psychometric properties
of the Medical Organizational Readiness for
Change (MORC) survey which was adapted from the
Organizational Readiness for Change (ORC) survey1
to assess the implementation of an alcohol and
drug screening intervention across several health
care settings. The ORC was developed for use with
substance abuse treatment programs. The
Screening, Brief Intervention, Referral, and
Treatment (SBIRT) model involves medical staff
screening patients and health specialists
providing services to those who screen positive.
RQ2 Can new, more reliable scales with construct
validity be identified?
RQ3 Is the MORC able to discriminate between
environments with different orientations to
change?
Methods
MORC surveys were administered on-site utilizing
a convenience sample of 186 staff at three
community health care clinics (CHC) and a level 1
trauma hospital emergency center (EC). All sites
were in the beginning stages of implementing an
SBIRT program. The response scales for the MORC
items ranged from 1 (Disagree Strongly) to 5
(Agree Strongly). The MORC utilized 48 of 129
items contained in the ORC. ORC scale reliability
data was obtained from http//www.ibr.tcu.edu/pubs
/datacoll/orcstats.htm
RQ3 Results T-tests were used to compare mean
responses between medical settings for the
MORC-2s 7 scales and 3 individual items.
Statistically significant differences between the
EC and CHCs were found on four scales
(Organizational Functioning, Work Group
Functioning, Autonomy Support, Supportive
Management) and two individual items (Leadership
and Skills). Actual differences between sites
were corroborated through staff responses
collected by interviews. Overall, the EC
appeared to be a more challenging environment for
program implementation than the CHCs. The
biggest differences between sites are reflected
in the Organizational and Work Group Functioning
Scale scores.
Findings
RQ1 When administered in healthcare settings, do
the MORCs scales show similar reliabilities to
the same scales from the ORC instrument
administered to substance abuse programs?
Summary
RQ2 Results A factor analysis was conducted
using 36 items for eight of the scales which
showed six factors represented the best solution
based on simple structure criteria. MORC-2 scales
(n7) including the two dimensions not included
in the factor analysis showed an overall average
reliability of .8 and ranged from 0.74 to 0.90.
Reliabilities by site were between 0.70 and 0.94
at the CHCs and slightly lower between 0.63 and
0.83 at the EC. Reliabilities are acceptable (at
the .6 level) for both EC and CHC settings.

• In summary, this study highlights several key
  points. First, effective implementation is
  critical to utilizing evidence-based practices
  successfully. Secondly, understanding
  organizational functioning is important in
  guiding implementation processes and requires
  valid and reliable assessment tools. Thirdly,
  assessment instruments need to be tailored to
  their context and technology. For example, we
  found that an instrument developed for substance
  abuse treatment centers did not perform well
  within medical settings. Finally, the new MORC
  instrument developed through this process
  provides a step forward in assisting future SBIRT
  implementations be successful. For example, our
  assessment indicates different strengths and
  challenges across sites requiring different
  implementation strategies. Adapting existing
  instruments to new settings and contexts may
  result in more useful assessment tools.

RQ1 Results Co-efficient alphas were computed
for scales separately for both CHCs and EC to
assess the internal consistency of the
established ORC scales. These alphas were also
compared to previously reported ORC data from
TCU. Analysis of the ORC scales contained in the
MORC showed an average reliability of 0.65 and
ranged from 0.34 to 0.91. Overall, only four of
ten scales (pressure for change, guidance,
change, cohesion) demonstrated acceptable
reliability (alpha gt .6) across settings
suggesting the conceptual structure of the ORC
scales did not have strong construct validity in
medical settings.
1 1 See D. D. Simpson (2002). A conceptual
framework for transferring research to practice.
Journal of Substance Abuse Treatment, 22(4),
171-182. This poster was presented at the
Research Society on Alcoholism Conference in
Baltimore, MD on June, 28 2006 . This study
was supported by SAMHSA grant number 26-3202-56.