SPRInTA

1
(No Transcript)
2
SPRInTA

• Student Portal Resources for Innovative Targeted
  Assessment
• FDTL-5
• (bid written by Jo Jones, Guin G-B et al)
• November 2004 October 2006
• Martin Sellens
• Nicola Bryan
• Ben Steeples

3
OVERVIEW OF PRESENTATION

• Background to project
• (why ever did we bother?)
• Overview of project aims
• Progress to date (pilot module completed linked
  to portal)
• Beyond the pilot

4
PEDAGOGY vsPRAGMATISM

• Driven by pragmatism (overwork).
• Post-hoc Pedagogy.

5
BACKGROUND

• SPORTS SCIENCE AT ESSEX.
• A centre within Biological Sciences
• 50 staff 160 undergraduates p.a.
• 8 staff 75 undergraduates per year.
• Large classes (relatively).
• Lots of marking.

6
PROJECT DRIVER 1LOTS OF MARKING

• Assessment heavy (Biological Sciences ROA).
• Level 1 (12 modules)
• MCQ
• Exam (short and essay)
• Practical reports
• Level 2 (10 lecture modules practical module)
• Exam (short and essay)
• Continuous assessment (e.g. 2000 word essay)
• 30 practical reports (worksheets and SPF)

7
PROJECT DRIVER 1LOTS OF MARKING

• ASSESSMENT OVERLOAD (staff and students)
• Slow feedback
• Variable quality
• Waste of time (FAST)?
• Whats the solution?
• On-line marking
• Cut down on assessment (Smith, 2005, pers com)

8
PROJECT DRIVER 2PROGRAMME SPECIFICATIONS

• How do we know students have achieved learning
  outcomes?
• Threshold testing.
• Level1
• MCQs
• 80 (previously pass was 40 average 50)

9
PROJECT DRIVER 2PROGRAMME SPECIFICATIONS

• How do we get any of them through?
• Specify threshold objectives
• Opportunities to practice (self test formative
  assessment)
• Opportunities to resit!

10
PROJECT DRIVER 3IMPROVE STUDENT LEARNING
PROJECT OBJECTIVES

• Provide formative assessment and examination
  practice.
• Target it temporally and individually (link with
  student records tiered assessments)
• Provide targeted tutorial support
• Improve feedback
• Educate about plagiarism (link with JISC)
• Do it all seamlessly through the student portal

11
PROJECT PLAN
MIS
BS151
BS152
DEPT
BS153
STUDENT PORTAL
BS1nn
FEEDBACK
ADVANCED
CMR LINKS TO OTHER RESOURCES
THRESHOLD
FEEDBACK
BASIC
ASSESSMENT
MCQ
PRACTICALS
OCS
OCM
ESSAYS
12
PROGRESS TO DATE

• Pilot core module to hit many learning outcomes
  (year 1 Human Physiology).
• Formative Multiple Choice Questions (MCQs)-using
  QMP- bank of 600 questions.
• Split course materials into 6 sub modules.
• Questions tiered by difficulty.
• Summative Exam Online.
• Incentive-Summative Exam Questions from Formative
  Question Bank.
• Practice (mixed subtopic) exams 1 week before
  final exam.

13
DEMO ORIGINAL METHOD

• Demonstration of original method to access
  Perception.

14
DEMO ORIGINAL METHOD
15
DEMO ORIGINAL METHOD
16
DEMO ORIGINAL METHOD
17
DEMO ORIGINAL METHOD
18
DEMO ORIGINAL METHOD
19
STUDENT USE OF FORMATIVE ASSESSMENT

• 92.5 of the 106 students used the resource.
• Average of 45.5 - 48.6 assessments per student.
• Questions were accessed 6288 times with 4691
  completed assessments.
• Further analysis showed an uneven spread of
  students study time.

20
FORMATIVE ASSESSMENTS
Number of formative assessments completed per
week for the formative assessments.
Wk1 Sub-topic 1 opened Wk3 Sub-topic 2
opened Wk5 Sub-topic 3 opened Wk7 Sub-topic 4
opened Wk7 Trial Exam opened
21
FORMATIVE ASSESSMENTS
Number of completed assessments, by hour, of the
twenty four hour period leading up to the
summative exam at 200pm.
22
SUMMATIVE EXAM RESULTS

• BS155 Average Summative Exam Result
• 2004 46.5 15.5
• 2005 71.0 19.7
• The 2005 results were significantly higher than
  the 2004 results (plt0.001 t-test).
• There was also a significant positive correlation
  between the number of formative assessments a
  student completed and their summative exam score
  (plt0.01).

23
SUMMATIVE EXAM RESULTS
Correlation (positive) between the students final
exam score and the number of formative
assessments they completed.
24
SUMMATIVE EXAM RESULTS

• 76.4 of the students took one or more practice
  exam(s).
• Significant difference (plt0.01 ANOVA) between the
  summative exam score of students who had 0 or 1
  attempts at the practice exam and the students
  who had 3 or 4 attempts at the practice exam.

25
SUMMATIVE EXAM RESULTS
Number of practice assessments completed, by
student uptake, compared to summative exam mark.
26
POST-EXAM SURVEY

• An optional survey was made available after the
  summative exam.
• 44 of 106 students completed the survey
• Some of the results are as follows

27
POST-EXAM SURVEY

• The practice MCQs were a useful revision tool
• It made revision far less tedious

28
POST-EXAM SURVEY

• Using the practice MCQs left me feeling prepared
  for the final exam
• The practice questions were a good
• indicator of how hard the final exam questions
  were going to be (very!!)

29
POST-EXAM SURVEY

• Online practice MCQs should be made available to
  all 1st year modules
• The questions are great and helpful and should
  definitely be made available for other modules

30
BEYOND THE PILOT

• Apply formative question banks to 5 other first
  year modules.
• Encouraging uptake throughout term weighting?
• Interoperability.
• Dissemination Writtle pilot.

31
DEMO NEW METHOD

• Demonstration of new method to access Perception.
• We hope to have this up and running for October
  2005.
• Other departments welcome to use this resource.

32
DEMO NEW METHOD
33
DEMO NEW METHOD
34
DEMO NEW METHOD
35
DEMO NEW METHOD
36
QUESTIONS?
37
CONTACT

• Telephone [phone](4369)
• E-mail sprinta_at_essex.ac.uk
• Website http//www.essex.ac.uk/sprinta/
• Presentation available on our website.