Evaluation and Quality Assurance

1
Evaluation and Quality Assurance

• Target groups
• Project member
• Content provider
• UoL developer
• Teacher
• Learner

2
Evaluation and Quality Assurance

• Project objectives
• Processes
• Results

3
Evaluation and Quality Assurance

• Processes
• Creating UoL
• Filling out IMS LD templates
• Internal cooperation
• Learning with UoL
• Peer reviewing
• Playing UoL

• Sharing content
• Teaching with UoL
• Working with the repository

4
Evaluation and Quality Assurance

• Results
• Learning activities
• Project guidelines
• Units of Learning
• User documentation
• Deliverables

5
Evaluation and Quality Assurance

• Creating UoL and filling out IMS LD templates
• Performance indicators
• Degree of difficulty
• Expenditure of time
• Perception of advantages/disadvantages of IMS LD

6
Evaluation and Quality Assurance

• Internal cooperation
• Performance indicators
• Individual perception of cooperation level
• Suitability of communication tools
• Individual perception of team structure
• Efficiency of project meetings

7
Evaluation and Quality Assurance

• Learning with UoL
• Performance indicators
• Perceived didactical value of IMS LD
• Peer reviewing
• Performance indicators
• Number of submitted peer reviews
• Perceived benefit of peer reviews
• Perceived level of objectiveness of peer reviews

8
Evaluation and Quality Assurance

• Playing UoL
• Performance indicators
• Degree of difficulty
• Expenditure of time
• Sharing content
• Performance indicators
• Perceived level of protection against plagiarism
• Disposition to share knowledge

9
Evaluation and Quality Assurance

• Teaching with UoL
• Performance indicators
• Perceived didactical value of IMS LD
• Working with the repository
• Performance indicators
• Perceived level of usability
• Quality of technical support

10
Evaluation and Quality Assurance

• Learning activities
• Performance indicators
• Number of trial/demonstration activities
• Number of authentic learning activities
• Project guidelines
• Performance indicators
• Completeness
• Comprehensibility
• Ease of finding relevant information

11
Evaluation and Quality Assurance
Units of Learning

• Performance indicators
• Number of UoL
• Number of items
• Learning time
• Number of subjects
• Number of languages
• Number of peer reviews
• Ratio of positive/negative reviews

• Number of downloads
• Distribution of CC licenses
• Didactical diversity
• Quality of learning materials

12
Evaluation and Quality Assurance

• User documentation
• Performance indicators
• Completeness
• Comprehensibility
• Ease of finding relevant information
• Deliverables
• Performance indicators
• Ratio of positive/negative peer reviews

13
Evaluation and Quality Assurance

• Evaluation methods

• Collecting quantitative data from the repository
  and corpus of learning materials
• Custom questionnaires for each target group
• Web-based survey for project members, content
  provider and UoL developer
• Print questionnaires for participants of
  demonstration events
• Analysis of internal peer reviews

14
Evaluation and Quality Assurance

• Time line for data collection
• 1st cycle (December 06)
• Statistical data from repository and corpus of
  learning materials
• Questionnaires from project members, content
  provider and UoL developer
• Peer reviews of deliverables
• Evaluation Report II
• Revised Evaluation Plan

15
Evaluation and Quality Assurance

• Time line for data collection
• 2nd cycle (March 07)
• Statistical data from repository and corpus of
  learning materials
• Questionnaires from project members, content
  provider and UoL developer
• Questionnaires from participants of
  trial/demonstration activities
• Peer reviews of deliverables
• Evaluation Report III

16
Evaluation and Quality Assurance

• Time line for data collection
• 3rd cycle (July 07)
• Statistical data from repository and corpus of
  learning materials
• Questionnaires from project members, content
  provider and UoL developer
• Questionnaires from participants of
  trial/demonstration activities
• Peer reviews of deliverables
• Final Evaluation Report

17
Evaluation and Quality Assurance

• TO DO
• Minor changes in questionnaires
• Translation of questionnaires for teacher and
  learner
• Collecting peer reviews
• Collecting e-mail addresses of project members,
  content provider and UoL developer
• Collecting statistical data from repository and
  corpus of materials Functionality implemented in
  repository?

18
Evaluation and Quality Assurance

• THANK YOU FOR YOUR ATTENTION!
• Contact
• Research Associate
• Chair Prof. Dr. Leidhold
• Political Science Department
• University of Cologne
• Fon 49.221.940.25.66
• Fax [phone]
• E-mail c.kozianka_at_uni-koeln.de