Evidenced Based Research, Best Practices, and Success Strategies

1
Evidenced Based Research, Best Practices, and
Success Strategies

• Mary J. Yoho, PhD., R.N., CNE
• Director of Faculty Development Consultation
• Elsevier Health Science
• m.yoho_at_elsevier.com

2
(No Transcript)
3
Evidence Based Education (EBE) Best Practices
Evidenced-Based Research Success Strategies
4
Evidenced-Based EducationWhats to Goal?

• Select retain students most likely to succeed
• Minimize attrition
• Identify and remediate at-risk students
• Prepare students for safe and effective nursing
  practice
• Maximize NCLEX pass rates
• Impact the nursing shortage
• Other goals?

5
Evidence Based Education (EBE)Defined

• An approach which argues that policy and practice
  should be capable of being justified in terms of
  sound evidence about their likely effects
• Robert Coe, 1999 A manifesto for evidence-based
  education.

6
EBE Policy Practice

• EBE signifies the idea that educational policy
  and practice should be guided by the best
  evidence about what works
• Specific intervention strategies and policies
  should be rigorously evaluated before they are
  advocated or required
• Where this is not possible they should be adopted
  experimentally, in such a way that their impact
  can be properly evaluated

7
EBE Culture

• Evidence is valued over unfounded opinion
• Appropriate evidence-based action is valued over
  action for actions sake
• Reclaim debates about policy and practice for the
  professionals who know the most about the topic.

8
Current EBE Concerns

• Opinion has the status of equal to or greater
  than that of evidence as a basis for action
• Evidence is not value-free educational research
  carries its own values and objectives
• Honest debate and discussion is often skewed by
  concealed evidence

9
Current EBE Hurdles

• No quick-fixes
• Time, funding, IRB, research, expertise
• All strategies are not universal
• Evidence can be complex, especially when
  interpreting statistical analyses

10
Greatest EBE Hurdle

• The need to act often
• precedes the evidence
• Leading us to more
• questions than answers!

11
Why should Nursing Education be Evidenced-Based?

• The need evidence-based
• Funding
• Local, State, National
• Institutional funds
• Policy and practice change need justification
• Can be difficult and time-consuming
• Needs support by faculty and leadership

12
Most Compelling Outcome of EBE

• The evidence to
• support change must
• be stronger than the
• evidence or choice
• not to change

13
How can Nursing Education be Evidenced-Based?

• Implement an EBE culture
• First question Where is the evidence?
• Access the evidence
• Set time for serious and informed debate
• Be accountable.be able to justify and account
  for decisions based on sound evidence
• Disseminate!

14
National Curriculum Revision Initiatives

• State-wide
• Admission Assessments
• Reverting Clinical to Simulation and Virtual Lab
  hours
• Consortiums
• ADN to BSN initiatives
• Proprietary-based Nursing programsa force in
  nursing education

15
Catalysts in Curriculum Evaluation

• Accommodate more students
• Who will accommodate the 41,000-92,000 qualified
  applicants to entry level nursing who were turned
  away? (NLN)
• Compensate for the escalating demand for more
  nursing faculty
• Increasing demand in service for nurse
  researchers, patient safety and quality, Magnet
  leaders
• Potential decrease with the closure of
  non-expansion of MSN programs and the additional
  of more DNP and PhD programs

16
More Catalysts

• Remediate-to-Retain
• Lecture/Clinical/Lab contact hours
• Restructuring
• Move clinical to simulation and eLearning
• Add online courses
• Accommodate additional content in the NCLEX-RN
  Test Plan

17
Coast-2-Coast Curriculum Initiatives

• What are states doing?
• Partnerships academic practice
• Developing Consortiums
• Program Entry Policies
• State-wide curriculum standardization
• Entry into practice

18
Coast-2-Coast Curriculum Initiatives

• What is available to support curriculum
  initiatives?
• E-books
• Curriculum online
• I-pod downloads
• Evaluation tools
• VCE and simulation scenarios
• Online case studies
• Online skills
• eLearning Initiatives are becoming more available

19
Partnerships Maryland

• D D of Maryland SONs, Maryland Hospital
  Assoc., and Health Services Review Commission
• Nurse Support Program II Increased the rate
  structure of all hospitals by 0.1, funds to be
  used for nursing education
• Generates 8.8 mil annually
• Funds used to support nursing programs,
  scholarships, stipends, fellowships for new
  nursing faculty (RWJF, 2007)

20
Partnerships Oregon

• Providence Health System, OR Region University
  of Portland SON
• Created Providence Scholars program scholarship
  funding for 75 juniors and 75 seniors, each year
  for 5 yrs16 mil
• Students commit to work for Providence for 3
  years after graduation
• Costs offset by savings of 100,000 per RN
  decreased orientation costs, turnover
  recruiting
• Funds 12 masters students...hospital staff RNs
  to academia (RWJF, 2007)

21
Partnerships Loyola University Chicago SON

• Partnering with clinical agencies
• SON sponsored nurse executive breakfasts
• Provided a forum for sharing education and
  practice issues
• Networking and updating
• Creation of a Nursing Partnering Community
• Magnet application
• Grant writing
• Consultation
• Share clinical agency staff and resources

22
Consortium Massachusetts

• 11 competencies agreed upon by an education and
  practice consortium
• Patient training
• Do not ask a nurse what their degree isbut if
  they are competent
• Competency is the priority!!

23
Consortium California

• Foundation for CA Community Colleges
• 2.3 mil in funds to track new nurse graduates
• Closed Cookie Jar 2000 emergency funds to keep
  a student in school
• Documentation Training for nurses to nurse the
  patient, not the computer.

24
Program Entry Policies CA (cont)

• California Community College Chancellors Office
  (CCCCO)
• Initiatives for state-funded ADN programs
• Moving from open enrollment-lottery system to
  selecting the most qualified students prepared
  for the rigors of the nursing curriculum

25
Program Entry Policies CA

• 14 mil for admission initiatives8 mil for
  standardized admission testing
• Elsevier A2 selected as one of 3 pre-admission
  exams
• Had to demonstrate psychometrics and EBE
• Cut score for the exam established by the CCCCO
• Students not meeting the score must be remediate
  by the SON, and re-evaluated for admission.

26
A² Study 1 The Predictive Accuracy of the HESI
Exams for an ADN Program

• Study Outcomes
• A² reading comprehension sub-test scores were
  predictive of academic success and significantly
  related to MC scores (p .001) and to E2 scores
  (p .05)
• The MC scores were predictive of nursing student
  success on the E²
• (p .001).
• The E² was highly predictive of student success
  on the NCLEX-RN with 94.82 Predictive Accuracy
  to the NCLEX-RN

27
A² Study 2 Evaluating success and curricular
outcomes using HESI exams in ADN and BSN programs

• Examined both ADN (N 68) and BSN (N 69)
  nursing students who took the A² exam.
• Study Outcomes
• A statistically significant correlation with the
  A² scores and the beginning-level nursing course
  grades in the ADN and BSN programs (p .05) with
  final course grades in the Sophomore, Junior and
  Senior year.
• High level of predictive accuracy of E² with
  NCLEX-RN success documented in both programs
• ADN (96.43)
• BSN (98.53)
• A² validated as a significant predictor of
  success in nursing school and E² NCLEX-RN
  success

28
Program Entry Policies Georgia Alabama

• Georgia statewide entry exam for ADN programs
• Alabama considering a statewide entry exam for
  ADN and PN programs
• Already have a state-wide Biology exam
• Flattens-out the applicants
• Using EBE to support decision

29
State-wide Curriculum Standardization Oregon

• Oregon Consortium for Nursing Education (OCNE), a
  partnership of Oregon nursing programs dedicated
  to educating future nurses.
• Faculty from 8 community colleges and Oregon
  Health Science University School of Nursing
  have created a shared curriculum taught on all
  consortium campuses
• Students can complete coursework for the BSN from
  OHSU without leaving their home community

30
State-wide Curriculum Standardization North
Carolina

North Carolina ADN Curriculum Project North
Carolina ADN Curriculum Project 2 yr. grant from
the NC Community College System Plan Redesign
and restructure the ADN curricula to reflect one
curriculum standard to include major advances in
nursing and healthcare practices. Seamless
articulation for students 55 institutions. Include
s online hybrid nursing courses, PDAs iPods,
streaming videos.
31
State-wide Curriculum Standardization North
Carolina (cont)

• State-wide Concept-based Curriculum
  Standardization North Carolina
• Goal Increase nursing student retention among
  SONs
• Technology supports
• eCourses
• iPod downloads
• Textbook on line
• Online evaluation tools

32
State-wide Curriculum Standardization Hawaii
Washington

• Hawaii Washington are considering state-wide
  curriculum
• Initiated through either an education consortium
  (HI) or state nursing workforce initiative (WA)
• How will they measure outcomes.besides NCLEX-RN?

33
National Curriculum
Standardization

• Proprietary nursing programs
• Picking up forgotten nursing applicants
• Starting with PNmoving to RN
• Curriculum-in-a-boxremember Doc-in-a-box?
• Smart, savvy, and business-minded
• Use total curriculum solutions

34
Entry into practice

• Should new graduates enter into practice before
  or after taking NCLEX-RN?
• Reiter, Young, Adamson (2007) Decrease New
  Graduate Nurse Orientation Costs by Using HESI
  Exit Exam Scores JONA, 37(10), pp 459-463.
• Goal to decrease cost of orientating new grads
  who then do not pass NCLEX-RN (costs
  39,000-65,000 per grad.
• Hired New graduates with a HESI Exit Exam score
  of 870 or gt
• The retention rate for new graduate nurses during
  this study was 98 after 1 year
• Conclusion Validated the use of HESI E2 scores
  as a predictor of employment success by nurse
  managers and educators was validated by this
  study.

35
Beyond the US.Foreign Grads

• NCLEX-RN pass rate for Internationally prepared
  RN grads 52.0
• As compared to 52.4 pass rate for repeat US
  educated testers
• 13 of all practicing RNs are from international
  programs

36
EB Research Foreign Grads

• Parrone, et/al (2008). An evidenced based
  teaching/learning strategy for foreign nurses
  involving the HESI exam as a predictor for
  NCLEX-RN success. www.JTNL.org.
• N67, 100 pass rates for students who achieved
  their cut score of 85 (850).

37
Beyond the USCanada

• Canadian Nurse Association Exam
• Multiple choice questions
• Not psychometrically evaluated as NCLEX
• Case studies
• Puts test-taker in the clinical situation

38
Beyond the USInternational

• Philippines and India
• Prepare graduates for NCLEX-RN before the new
  graduates come to the US
• Global nursing shortagecontroversy between
  bringing in foreign grads vs leaving qualified
  applicants behind
• Nursing shortage is international too!

39
Identifying best practices

• Partnerships
• Consortiums
• State-wide initiatives
• Online
• Simulation
• How do we evaluate?

40
EBE does not need to be hair-raising

• Closing Advise
• Set the culture
• Access the evidence
• Use global research but think local
• Use EBE at every opportunity
• Be prepared to justify your decisions on evidence
• Disseminate!

41
Best Practices for Student Success On-line
Testing
42
The Classroom Has Changed!

• Then and Now

43
Rationale for Implementing Online Testing

• Assess the preparedness of applicants
• Evaluate and rank applicants according to faculty
  standards
• Evaluate student knowledge and critical thinking
• Statistically analyze exam results with
  auto-analysis
• Internal and external curriculum evaluation
• Prepare for annual and accreditation reviews

44
Advantages for Implementing Online Testing

• Electronically bank test items
• Download faculty-made items
• Download text book items
• Easily add alternate format items
• Pilot test items
• Accumulative statistics
• Electronic exam blueprints

45
Advantages for Implementing Online Testing

• Electronically bank test items
• Download faculty-made items
• Download text book items
• Easily add alternate format items
• Pilot test items
• Accumulative statistics
• Electronic exam blueprints

46
Testing Software.what should faculty use?

• House and organize item and exam banks.
• Selective faculty access
• Item analysis
• Cumulative item data
• Report Test data

47
Online ExamsEvaluate Your Nursing Curriculum

• Internal Curriculum Evaluation
• Reliable Valid test items exams
• Coordination with
• the NCLEX-RN Test Plan
• QSEN Competencies
• NLNAC Competencies
• AACN Competencies
• Nursing Concepts

48
Testing Software.what should faculty use?

• Faculty access and tracking of scores
• Students can access and track their scores
• Provide individual student remediation.
• Track academic program performance

49
Creative Online ExamsUsing Textbook Items
49
50
Evidenced-Based Research Standardized Testing
51
HESI Evidence-Based Research Overview

• State of our Science
• Research In Press and Near Press
• Research Advisory Board

52
Exit Exam Validity Studies
N37,184
53
Predictive Accuracy HESI Exit Exam (E²)Validity
6 Results
54
Predictive Accuracy HESI Exit Exam (E²)
Validity 7 Results

• Stratified Random Sample
• 137 schools surveyed
• 72 returned questionnaires (53)

Schools
Students
55
E2 Version 1
56
Repeat E2 Version 2
57
Repeat E2 Version 3
58
Validity 7 StudyOnline Case Studies

• Does the utilization of Evolve Case Studies
  significantly increase baccalaureate and
  associate degree registered nurse student scores
  on the Evolve exit examination as compared to
  students who do not use Evolve Case studies?
• Random Stratified Sample
• Two-way Analysis of Variance

59
Findings Case Studies
59
Means and Standard Deviations of E2 Scores by
Program and Case Study Utilization
60
Findings Case Studies
60
Two-way ANOVA Summary for Interaction and main
Effects for Case Study Use and Program Type
61
Research Understudy Does Online Remediation
Increase Student Exam Scores?
62
Research Understudy Does Faculty Tracking of
Online Remediation Increase Student Exam Scores?
63
Best Practices for Student Success Simulation
64
(No Transcript)
65
Simulation and Med Administration

• There was compelling evidence that collectively,
  students in clinical placement generate fewer
  medication errors if they have had prior
  exposure to a related, simulation-based
  experience.

66

• Challenges in Clinical Nursing Education
• Curriculum integration
• Lack of faculty support / staffing
• Pressure to integrate / use
• Student recruitment
• Lack of clinical sites / competition
• Complexity of patients in hospitals
• Opportunity for new teaching/learning model

67
Will you lead the charge to implement EBE
throughout your curriculum?
68
Thank You!!

• Mary J. Yoho, PhD, RN, CNE
• m.yoho_at_elsevier.com