GCU Showcase: Large scale simultaneous online assessment

1
GLASGOW CALEDONIAN UNIVERSITY
CALEDONIAN BUSINESS SCHOOL INNOVATION AND
BUSINESS DEVELOPMENT GROUP
2

• Sheena MacArthur
• Gillian Roberts
• Linda Creanor

3
MARKETING FUNDMENTALS
4
MARKETING FUNDMENTALS

• 900 students over 2 semesters

5
MARKETING FUNDAMENTALS

• 900 students over 2 semesters
• ASSESSMENT
• 60 Course work
• 40 Multiple choice exam

6
MARKETING FUNDAMENTALS

• 900 students over 2 semesters
• ASSESSMENT
• 60 Course work
• 40 Multiple choice exam
• Exam on paper and manually marked

7
MARKETING FUNDAMENTALS

• January 2006

8
MARKETING FUNDAMENTALS

• January 2006
• 435 students
• 259 (60) online
• 176 (40) paper

9
MARKETING FUNDAMENTALS

• January 2006
• 435 students
• 259 (60) online
• 176 (40) paper
• Preparation
• IT staff, Exam office staff, Academic staff
• Security issues, Special procedures, Special
  invigilators instructions

10
EVALUATION
11
EVALUATION

• AIM - To identify students views of online
  assessment

12
EVALUATION

• AIM - To identify students views of online
  assessment
• Short questionnaire
• 212 responses (49 of those who took the exam)

13
EVALUATION

• AIM - To identify students views of online
  assessment
• Short questionnaire
• 212 responses (49 of those who took the
  exam)
• 96 students who had taken exam on paper
• 116 students who had taken exam online
• Two focus groups - 10 Online, 11 Paper

14
I value the ability to get immediate feedback on
my performance
15
Answering MCQs online is more stressful for me
than answering them on paper
16
REASONS GIVEN FOR ANSWER

• DISAGREED
• Easier to answer online and to change answers
• More relaxed atmosphere
• Comfortable with either
• I think everyone feels an affinity with
  computers cos were all so used to using them,
  so it does ease you in a bit

17
REASONS GIVEN FOR ANSWER

• NEITHER AGREED NOR DISAGREED
• Comfortable with either
• Each was equally stressful
• I was dreading that examcos it was online once
  I got going it was just easy panic over! I
  wouldnt have a problem doing it in future

18
REASONS GIVEN FOR ANSWER

• AGREED
• Possibility of technical difficulties
• Do not like computers
• Like to see questions on paper or write notes on
  paper
• Theres still something in the back of your mind
  saying, Where is it going? How is it being
  marked?

19
Assuming the technical difficulties experienced
by everyone at the start of the exam were
resolved, Id prefer to undertake MC exams online
20
REASONS FOR ANSWERS

• AGREED
• Instant marks
• Easier to answer online
• I would be glad to hear that we were doing more
  exams like this
• I feel as if you trust the computer better to
  give you right marks

21
REASONS FOR ANSWERS

• NEITHER AGREED NOR DISAGREED
• Comfortable with either
• Miscellaneous
• An exam is an exam . Doesnt matter how you
  take it

22
REASONS FOR ANSWERS

• DISAGREED
• Potential for technical difficulties
• If they could iron out these little mistakes.
  Then online examinations would be a really
  positive thing

23
LESSONS LEARNED

• Technically possible
• How to secure the online environment
• Accommodation solutions available
• Positive response from Students
• Staff development (Dundee online assessment
  course and experiential learning)

24
STUDENT PERFORMANCE

• No difference in mean mark between those who took
  exam online and paper
• No difference in failure rates
• No difference in mean mark compared with previous
  years

25
THE WAY FORWARD

• May 2006
• Two separate exams (150 students each)
• Fewer Labs
• Fewer Invigilators
• Next session Saltire Centre?