Dr Andrew Miles

1
Student Perceptions of the Use of Computer-Aided
Assessment

• Dr Andrew Miles
• School of Sport, PE and Recreation
• University of Wales Institute, Cardiff

2
Computer-Aided Assessment
CAA can be defined broadly as

• the use of computers in the assessment of student
  learning

This includes the use of computers to deliver,
mark and analyse assignments and examinations
3
Background

• Educational value of CAA has already been
  reported (Bull, 2002)
• UWIC adopting Blackboard as its VLE and wanting
  to pilot CAA effectiveness
• Explore practical issues by piloting CAA in
  Schools that already use Blackboard
• Draw up a set of recommendations for integrating
  CAA into assessment procedures across UWIC

4
Background

• Level One module taught in Semester 1
• Mixed A-Level profile
• All SSPER First Years (n300)
• Traditionally delivered as lead lectures plus 5
  Laboratory sessions
• Assessed by MCQ and Short Essay

5
Issues

• Large numbers (n300)
• New to Higher Education/ UWIC
• Work ethic
• Key skills
• Unfamiliarity Environment / Peers
• Variable previous subject experience
• Some have A-Level Biology / PE
• Outdoor and Adventure Residential

6
Blackboard as a solution

• Managed learning environment
• Permanent point of reference
• Allows non Biology / PE students access to
  additional notes
• Allows Biology / PE students to go further
• Flexible

7
Physiology Module Use of CAA

• Lead lecture PowerPoint Slides
• Additional reading material and Website links
• Quizzes
• Thematically organised
• Themed Formative Assessment Decathlon
• On-line assessment MCQ ?????????

8
Physiology Module Use of CAA

• Summative Assessment
• Intention to utilise CAA for final module
  assessment
• Decision dependent on students / resources
• Formative Assessment
• Themed approach
• 10 short answer quizzes
• Decathlon Theme
• Link to website
• Competitive nature

9
Student Usage

• Total students registered 261 (87)
• Total hits 21668
• Announcements 3878
• Information 3084
• Staff Information 924
• Documents 10906
• Assignments 1962
• Bibliography 914

10
Student Evaluation

• Student usage of the formative quizzes was high
  (95)
• 46 saying they were Very useful
• 48 saying they were Quite useful
• 59.6 of students felt the themed quizzes made
  learning more interesting

11
Student Evaluation

• 35 of the students would have preferred to take
  the Summative assessment via Blackboard rather
  than in paper format.
• 32.5 said they would have serious concerns if
  asked to take the assessment via Blackboard.

The most common concern of students was a
perceived lack of computer skills and the concern
that marks would be based on their computer
skills rather than their physiology knowledge.
12
Student Achievement
100
90
80
71.6
70
66.7
66.8
63.1
65.4
63.1
60
57.4
56.7
56.3
53.5
50
SDev
SPC
SPE
SES
SES
SDev
SPC
SPE
Pre-Blackboard
Post-Blackboard
13
Success?

• 8 improvement in student marks
• Better students ?
• Easier assessment ?
• Better teaching and learning ?
• BLACKBOARD ??????????

14
Issues Arising

• IT related
• IT Skills
• IT registered
• IT workstations
• Assessment implications
• Staff related
• Time to maintain and update Blackboard pages
• Student support

15
Issues Arising

• Blackboard related
• Student registration
• Student induction to Blackboard
• Monitoring student usage
• In classes
• Independently
• Quality of usage

16
CAA Guidelines
Summative Assessment

• Regulatory Issues
• UW regulations, Security etc
• Human Issues
• Student Issues IT Skills, Special needs,
  Errors, Groups
• Staff Issues Questions, Invigilation, Marking,
  Data
• Pedagogical Issues What is being tested? What
  can be tested?
• Systems Issues Hardware, Software, IT Support,
  Facilities

17
CAA Guidelines
Formative Assessment

• Regulatory Issues
• Less of an issue
• Human Issues
• Student Issues Adherence, Motivation,
  Frequency
• Staff Issues Updating, Monitoring, Feedback,
  Support
• Pedagogical Issues Question format, Feedback
• Systems Issues Remote Access, Hardware, Software