The Significance of Standards in Elearning: the MedBiquitous Experience

1
The Significance of Standards in E-learning the
MedBiquitous Experience

• Peter Greene, M.D.MedBiquitous
• Rachel Ellaway, Ph.D.University of Edinburgh
• SMILE 2006

2
Learning Objectives

• By the end of this workshop, you will be able to
• Describe standards related to the development and
  delivery of e-learning
• Identify strategies for using those standards in
  ways that increase the pedagogical value and
  cost-effectiveness of educational resource
  development.
• Describe standards related to the tracking and
  evaluation of learning activities.
• Identify opportunities for applying standards for
  tracking and evaluation.
• Describe the role that standards could play in
  transforming medical education.

3
How would you describe yourself?

• Educator (primarily)
• Technologist (primarily)
• Both

4
Are you using a learning management system or
other learning technologies that are
standards-based ?

• Yes
• No
• I have no idea

5
Technology standards are very important for
advancing healthcare education

• Strongly agree
• Agree
• Neutral
• Disagree
• Strongly disagree

6
I find educational technology standards
confusing

• Strongly agree
• Agree
• Neutral
• Disagree
• Strongly disagree

7
MedBiquitous Mission

• To advance healthcare education through
  technology standards that promote professional
  competence, collaboration, and better patient
  care.
• Non-profit, member-driven, standards development
  organization

8
Overview

• What is MedBiquitous?
• Development and delivery of e-learning
• Tracking and evaluation of learning activities
• Opportunities for application
• How to participate

9
MedBiquitous Technology Standards for
Healthcare Education

• 50 member organizations
• 7 Working Groups
• ANSI process
• Openness
• Transparency
• Consensus
• Due process
• Work with leading organizations that can drive
  adoption

Professional Profile
Learning Objects
Activity Reporting
Metrics
Virtual Patients
Point of Care Learning
Competencies
10
ANSI Accredited Standards Developer

• MedBiquitous develops information technology
  (XML) standards for
• Healthcare education and training
• Healthcare professional competence assessment,
  certification, and licensure
• Healthcare professional and scientific
  publications
• Healthcare professional online communities and
  portals

11
MedBiquitous Process

• Approves new standards projects
• Meet via telco, in person
• Develops specifications
• Consensus body
• Votes
• Final approval

Executive committee
Working groups
Standards committee
ANSI
12
Weaving the es TogetherDr Rachel Ellaway, In
Print, Medical Teacher

• .effective and productive standards and
  specifications development processes are most
  often marked by a sense of common purpose and a
  suspension of the usual contest of professional
  egos and exercising of organizational rivalries
  found in other collaborative activities

13
Development and Delivery of E-learning

• Healthcare LOM
• Finding learning activities
• SCORM for Healthcare
• Interoperability of self-directed Web-based
  learning
• Virtual Patients
• Interoperability of Web-based simulations of
  clinical encounters

14
Healthcare Learning Object Metadata (LOM)

• Metadata Definition the data that describes a
  resource.
• Example card catalog information for a book
  author, title, publication date, topic, etc.
• Problem many publishers, libraries, and faculty
  authors use completely different metadata
  schemes.
• Example Title Assistant Professor vs. Title
  Chronic Renal Failure

15
Healthcare LOM Uses

• Learner searches a continuing education course
  catalog. All records are organized by LOM.
• An education provider is sending a board or other
  regulatory entity information about a clinician
  that has completed a required educational
  resource.
• A learner has a clinical question and searches a
  knowledge base.
• An educator searches a repository of teaching
  materials.

16
Healthcare LOM

• Builds on existing IEEE Learning Objects Metadata
  Standard
• Provides healthcare specific information needed
  by healthcare educators
• CE Credit information
• Medical vocabularies
• Level of evidence
• Educational context
• And more

17
Healthcare LOM Structure
18
Healthcare LOM Structure
19
Content Interoperability
If I switch learning management systems, can I
take my content with me?

• Without standards, content is
• Not portable / interoperable
• Monolithic and not easily updated
• Unable to be shared

20
SCORM

• An e-learning model that enables accessibility,
  reuse, and interoperability of learning objects
  and tracking of learner progress.
• Standards for running, packaging, and describing
  learning content
• Broad international adoption

• Shareable
• Content
• Object
• Reference
• Model

21
SCORM for Healthcare

• Profile of SCORM
• Leverages Healthcare LOM extensions to SCORM
  metadata
• Developed by MedBiquitous
• All SCORM for Healthcare content is conformant
  with SCORM

22
Goals of SCORM
23
SCORM Uses a Learning Objects Approach

• Discrete unit of digital instruction typically
  focused on a single learning objective
• Benefits
• Access just the information/education you need
• Reuse in multiple contexts (standalone or as part
  of a course) and systems
• Eliminate single vendor dependence
• Stop reinventing the wheel

24
Virtual Patients

• Definition An interactive computer simulation of
  real-life clinical scenarios for the purpose of
  medical training, education, or assessment.
• Several schools are developing
• Very costly to create

25
does it radiate anywhere
26
(No Transcript)
27
Current educational use of Virtual Patients

• Used within a single institution or single
  subject
• Hand coded or dependent on a proprietary engine
• Unable to be shared with other institutions
• Reinventing wheels leads to low level
  implementations and gaps

28
MedBiquitous Virtual Patient Architecture

• Component based approach
• Enables sharing of entire virtual patient
• Independent of player technologies
• Mix of existing and new standards and
  specifications
• Virtual patient can be used as is, modified, or
  broken into component parts for reuse
• Currently in development

29
Short Term Goals

• Create an overall architecture that separates the
  patients clinical data from the data that
  manages the educational delivery
• Develop a version 1.0 Virtual Patient
  specification that will include a data model for
  a clinical snapshot
• Ensure that this may be rendered via an emerging
  Virtual Patient Player

30
Virtual Patient Data
31
Working Group Accomplishments

• Use cases defined
• Development roadmap
• Architecture document
• Draft schema and specification for Virtual
  Patient Data

32
Are you planning to develop virtual patients in
the next year?

• Yes
• No

33
Tracking and Evaluation of Learning Activities

• Competencies
• Coordinating competency data
• Activity Report
• Tracking CE and certification activities across
  organizations
• Medical Education Metrics
• Reporting aggregate evaluation data

34
Competencies Defined

• The objectives and the outcomes that result from
  knowledge, attitudes, skills, and beliefs.

35
Uses for a Competency Standard

• Allow learners to track their accomplishments
  against a list of relevant competencies
• Enable educators to see how their curriculum fits
  into a competency framework
• Enable content developers to tie educational
  activities to a competency framework

36
Accomplishments

• Existing IEEE and HR-XML standards identified
• Work plan defined
• High level use cases defined
• Research on association of content with
  competencies with Oregon Health and Science
  University, funded by NLM

37
Building a Competency Framework
Competency Map
CompetencyAssociations
CM
CM
CM
CM
Healthcare Learning Object Metadata
Learning Objects/Activities
38
Are you planning to use technology to help
connect learning resources and activities to
competencies?

• Yes
• No

39
Activity Report Working Group

• Mission develop XML standards and Web services
  requirements and descriptions to enable tracking
  of the learning and certification activities of
  physicians and other healthcare professionals.

40
Current Problems with CE/CPD Tracking and
Measurement

• We arent tracking
• We dont help the learner assess gaps
• We cant tell if activity matches learners
  practice-based needs
• We dont measure value consistently
• We dont verify claimed credits for certification
  or licensure

41
Reporting and Tracking CE/CPD
Provider A
Activity report
ltxmlgt
Dr. John Does CE Tracker A Asthma Management
.5 cr B Bronchodialators .5 cr A Pain
Management .5 cr Report to Board
CME Tracker
Activity report
ltxmlgt
Provider B
42
Benefits

• Enables centralized tracking of CE/CPD and
  Certification activities
• Increases accountability of all parties
• Reduces administrative burden on professional

43
MEMS Medical Education Metrics

• Technology standard for core Kirkpatrick level 1
  evaluation data

• Users
• Educators want best practices, ability to compare
• Funders want to measure reach and efficacy
• Accrediting Bodies want to measure success of
  activity and provider

44
MEMS Data

• Activity Description
• Whats being evaluated
• Participant Activity Evaluation
• What did participants think
• Participation Metrics
• How many people participated
• Learner Demographics
• Provider Profile (future use)

45
MEMS Use Case

• Best practices
• Standardize measurement of learning activities
  within a program
• Communication of participation and evaluation
  metrics to
• Accrediting bodies
• Funders

46
Accomplishments

• Currently researching validity and reliability of
  standard survey instrument (RSNA)
• Easy to compare metrics among programs
• Plan to include higher levels of evaluation

47
Activity 1

• You are a lead expert at your institution in
  leveraging technology for education. Your
  institution is reforming its undergraduate
  medical curriculum to be based on the Scottish
  Doctor learning outcomes. Outcomes-based
  education is new to your institution, and you
  want to make sure that students have the
  opportunity to apply their learning in realistic
  and relevant ways.
• Your Dean of Education wants to leverage learning
  technologies to achieve these goals, but the
  budget for these technologies is minimal. Your
  institution does own a SCORM conformant learning
  management system and virtual learning
  environment, but it has not been leveraged
  effectively by the faculty. How do you approach
  content development in this environment?

48
Activity 2

• Everyone is impressed by your success. The
  director of CPD at your institution contacts you
  regarding the development of CPD portfolios for
  faculty. The institution offers its own CPD but
  also partners with 6 other providers. How can the
  institution track faculty participation in these
  CPD activities and track the facultys reaction
  to the external programs?

49
Opportunities for Application

• Can existing standards address any of the
  challenges you face?
• Are there needs for other standards?

50
How to Participate

• Membership
• Standards Committee
• Working Groups
• Lurking (standards are free and available at
  www.medbiq.org )

51
Contact us

• Peter Greenepeter.greene_at_medbiq.org
• Rachel Ellawayrachel.ellaway_at_ed.ac.uk