Maintaining student engagement with formative assessment using Questionmark

1
Maintaining student engagement with formative
assessment(using Questionmark)

• Tracey Wilkinson
• Centre for Biomedical Sciences Education
• School of Medicine, Dentistry and Biomedical
  Sciences

2
Plan

• why use objective assessments?
• advantages of regular formative assessment
• Questionmark
• types of questions available
• examples
• have a go!
• exchange ideas
• final word

3
Why use objective assessments?

• diagnostic (often early)
• self-assessment (feedback to students)
• continuous, formative (feedback to students and
  teachers)
• summative (counted towards final mark)

4
Formative assessment

• improves academic performance
• feedback element particularly important
• allows both teacher and student to monitor
  progress
• engagement summative element
• increasing requirement for online /
  computer-assisted assessment

5
Advantages of online tests

• advantages for lecturer
• reduced time commitment
• logistics of individual feedback (especially for
  large classes)
• allow testing of wide range of subjects
• more frequent assessment possible
• analysis of questions
• advantages for student
• convenient access
• immediate feedback (individual)
• monitor progress

6
Disadvantages of online tests

• time and effort to construct good questions
• higher order questions (Blooms taxonomy)
• written communication skills or creativity not
  easily tested

7
Questionmark Perception Assessment Management
System

• Queens pilot project
• easy to use online assessment tool
• more question types available (than QOL)

8
Examples from Questionmark
9
Have a go!
In pairs, or on your own, discuss the types of
questions that might be suitable in your
particular discipline. - true-false -
multiple choice / response - matching -
ranking - fill in / select a blank - text
match - hotspot / drag and drop - Likert
Now try constructing a few simple questions in
your own discipline(s).
Compare the questions you have constructed with
your neighbour. Any suggestions for improvement?
Any questions that might test higher cognitive
ability?
10
Logic and reasoning

• Train service suffers when a railroad combines
  commuter and freight service. By dividing its
  attention between its freight and commuter
  customers, a railroad serves neither particularly
  well. Therefore, if a railroad is going to be a
  successful business, then it must concentrate
  exclusively on one of these two markets.
• For the argument to be logically correct, it
  must make which one of the following assumptions?
• Commuter and freight services have little in
  common with each other.
• The first priority of a railroad is to be a
  successful business.
• Unless a railroad serves its customers well, it
  will not be a successful business.
• If a railroad concentrates on commuter service,
  it will be a successful business.
• Railroad commuters rarely want freight services
  as well.

Answer c
American LSAT (Law School Admissions Test)
11
Logic and reasoning

• A small software firm has four offices, numbered
  1, 2, 3 and 4. Each of its offices has exactly
  one computer and one printer. Each of these eight
  machines was bought in either 1987, 1988 or 1989.
  The eight machines were bought in a manner
  consistent with the following conditions
• the computer in each office was bought either in
  an earlier year than, or in the same year as, the
  printer in that office
• the computer in office 2 and the printer in
  office 1 were bought in the same year
• the computer in office 3 and the printer in
  office 4 were bought in the same year
• the computer in office 2 and the computer in
  office 3 were bought in different years
• the computer in office 1 and the printer in
  office 3 were bought in 1988
• If the computer in office 3 was bought in an
  earlier year than the printer in office 3, then
  which one of the following statements could be
  true?
• The computer in office 2 was bought in 1987.
• The computer in office 2 was bought in 1988.
• The computer in office 4 was bought in 1988.
• The printer in office 4 was bought in 1988.
• The printer in office 4 was bought in 1989.

American LSAT (Law School Admissions Test) Sample
Test, June 1999
Answer b
12
Regular assessments
Have any of you used regular formative assessment
(paper, personal response, Queens online, other)
with feedback? In groups of two to three,
discuss the ways in which you have used, or may
be planning to use, regular formative assessment
to enhance the learning in your classes. Be
prepared to give feedback to the whole workshop.
Feedback any areas of good practice?
13
Final word

• the students
• uptake
• find them useful
• ask for more
• pressure to do well as teacher sees them
• good revision for tests

14
References

• Brown G, Bull J and Pendlebury M (1997) Assessing
  student learning in higher education. London
  Routledge.
• Bull J and McKenna C (2004?) Good practice guide
  in question and test design.
• Fry H, Ketteridge S and Marshall S (2003) A
  handbook for teaching and learning in higher
  education. London RoutledgeFalmer.
• Miller T (2009) Formative computer-based
  assessment in higher education the effectiveness
  of feedback in supporting student learning.
  Assessment and Evaluation in Higher Education
  34181-192.
• Nicol D (2007) E-assessment by design using
  multiple choice tests to good effect. Journal of
  Further and Higher Education 31 53-64.
• Wiliam and Black (2003) In praise of educational
  research formative assessment. British
  Educational Research Journal 29623-637.