Dia 1

1
E-XCELLENCE Benchmarking quality in European
HE e-learning
QA/QE in e-learning SIG Workshop Feb 27th 2009
2
Project Origins

• QA in HE is receiving much attention at
  institutional, national and European level.
• Still missing QA specific to e-learning
• -criteria based on ease of access
• -new forms of interaction (students and staff)
• -flexibility, personalisation and other
  pedagogical
• aspects that are now deliverable by e-learning
• These express the added value of e-learning in
  HE.
• EADTU established a members task force and
  secured EU funding for 2 year project through
  2005-2006

3
E-xcellence Benchmarking Quality in e-learning
Main Contributors EADTU OULU-University
(Finland) OUNL (Netherlands) OUUK (United
Kingdom) with support from CNED (France) UNED
(Spain) UOC (Spain) EITSA (Estonia) NETTUNO
(Italy) APERTUS (Hungary) NVAO
(Netherlands/Belgium) European University
Association (EUA)
4
General objective

• QA in e-learning that
• Optimises the learning process and offering
  assurance to
• stakeholders that e-learning provision is of high
  quality.
• Is complementary to existing national quality
  assurance systems
• related more to content, staff and
  infrastructure.
• Our main aim was to establish
• a framework of quality criteria for the
  development, operation and evaluation of
  e-learning programmes
• an appropriate set of performance indicators,
  parameters and guidelines by which the quality of
  e-learning programmes can be measured by
  assessment
• a definition of excellence

5
APPROACH

• Analysis of existing quality processes of
  partners and/or requirements of national quality
  organisations
• Development of a suite of criteria relevant to
  e-learning
• (using EADTU members experience)
• Stakeholder consultation procedure
• Translation of these parameters into a flexible
  quality assessment tool based on benchmarks
• Testing in 3 pilots the benchmarks, guidance
  manual and approach
• Univ Oulu Finland
• EITSA Estonia
• UNINETTUNO Italy

6
Options Explored

• Structured CMM style models vs flexible informal
  systems
• EADTU accreditation vs Integration with national
  systems
• Diversity of European systems favoured
  flexibility in design and implementation.

7
Outputs

• Set of (33) European benchmarks
• These cover institutional, pedagogical,
  technical,
• ethical and management aspects of e-learning
• organised into the three main categories of
• Management (institutional and programme levels)
• Products (curriculum, course design and
  delivery)
• Services (student and staff support)
• An important aspect of E-xcellence is that it
  offers a European-wide
• set of benchmarks, independent of particular
  institutional or national
• systems, and with guidance for educational
  improvement

8
Structure of Resources

• Benchmark Statements
• Define desired features
• Indicators
• Source of evidence
• Indicators of excellence
• Exemplars of evidence
• Assessors Notes
• Provide background guidance for assessors

9
The E-xcellence manual

• Mindmap QUALITY manual as backbone of web-tool

10

• Web http//www.eadtu.nl/e-xcellenceQS/

11

12
Modes of use

• Informal Self Assessment via Quickscan
• Identify hot spots (and cold spots)
• Internal Quality Assurance/Improvement
• embed selected benchmarks in internal process
• adopt and formalise indicators
• External Quality Assurance - informal
• embed use of benchmarks in Self Assessment
• Adapt to meet QA agency requirements
• External Quality Assurance - formal
• external QA agencies embed benchmarks within
  their processes

13
Quickscan

• Simplified version of the full e-xcellence
  assessment tool
• Quick self-assessment of your institutions
  e-learning performance
• Identifies areas in your e-learning
  programme/course that need more attention
• Score yourself on the most relevant aspects
• Results will show after completing quick-scan.
  These help you
• To identify elements to be improved
• To guide the internal discussion
• To learn if a full quality assessment procedure
  is useful
• http//www.eadtu.nl/e-xcellenceQS/

14
(No Transcript)
15
Full e-learning assessment

• Evaluate the institution/program/course using the
  E-xcellence
• method and tool.
• Manual Benchmark statements, in relation to
  criteria and indicators.
• Questionnaire comments on the specific issue
  in the box you can refer to documents or other
  references which can be used as evidence on that
  specific aspect of e-learning.
• The on-line questionnaire needs to be filled out
  by different disciplines in your organisation
  coming from management, course designers, tutors
  and students
• On-site visit of e-learning expert

16
Full assessment
SITE VISIT - SELF ASSESSMENT - ANALYSIS -
INTERVIEWS - CONCLUSIONS
PREPARE SELF ASSESSMENT USING THE TOOL
ACTION PLANNING
REVIEW TOOLAGAINST INSTITUTIONALSTRUCTURE
PROCESS
Support manual, proposed agenda, assessors
notes, glossary.
17
Activity Breakout

• Quickscan provides overview of benchmarks to
  consider
• Relevance
• Confidence
• Evidence

18
Pilot Experience

• 3 Internal Quality Assurance reviews with
• E-xcellence team members participating
• Some difficulties with language and
  interpretation of benchmarks
• Suggestions for additional factors and
  regrouping
• More needed on the process of the exercise
• More on best practice

19
Pilot Experience

• Conclusions
• an intensive and productive exercise
• relevant information for the project materials,
  process, procedure
• relevant for the institutions
• relevant for the piloting teams
• work in progress
• base document/material for different products

20
Review Extracts

• The E-xcellence tool has the potential for
  becoming a useful instrument in supporting
  European universities towards developing the
  quality of e-learning applications (EUA)
• E-xcellence has evolved into an attempt to work
  within existing QA frameworks rather than
  re-inventing them. In our view, rightly so. We
  are convinced that the approach and products of
  E-xcellence are useful for institutions that
  aspire to having the e-learning component of
  their Bachelors and Masters degree courses
  assessed within the existing NVAO accreditation
  framework (NVAO)

21
E-xcellence

• EU funding for dissemination and implementation
• Strengthened links with ENQA
• Additional project partners

22
E-xcellence Partners
Core Contributors EADTU OULU-University
(Finland) OUNL (Netherlands) OUUK (United
Kingdom) with support from CADUV Uni Hradec
Kralove ( CZ Rep) UNED (Spain) EITSA
(Estonia) NETTUNO (Italy) Kath Univ Leuven (
Belgium) APERTUS (Hungary) NVAO
(Netherlands/Belgium) NHSU Univ Lund (
Sweden) Slovak Univ Technology ( Slovakia) MESI (
Russia) IFEL ( Switzerland)
23
E-xcellence Activities

• European Seminar May 2008 Madrid
• Jointly organised with ENQA
• 50 participants
• National Seminars
• One in each partner country
• Participation of institutions and Accreditation
  agencies/ministries
• Trialling of Quickscan and Full Assessment

24
E-xcellence Activities

• Quickscan trials
• Use of Quickscan by each participating
  institution
• Full Assessment trial
• Two online PG programmes at Univ Lund in
  association with NHSU
• OUNL in association with NVAO

25
E-xcellence Activities

• Current outcomes
• Quickscan analysis at Univ Hradec Kralove
• Internal staff training programme in use of Web
  CT
• Identified discrepancies between departments in
  implementation of central policy
• Interesting exemplars of learner support
  provision and staff reward structures

26
E-xcellence Sustainability

• E-xcellence Associates label
• Commitment to continuous improvement in
  e-learning activity
• Use of E-xcellence tools to identify and review
  improvement
• Virtual Benchmarking community
• Peer review by community members
• Input to update and revision of E-xcellence tools

27

• Web http//www.eadtu.nl/e-xcellenceQS/
• Thank you for your attention