Assessment and the First Year Experience

1
Assessment and the First Year Experience
Dr David Nicol, Project Director, REAP
www.reap.ac.uk Head of Research Development in
E-learning Centre for Academic Practice and
Learning Enhancement (CAPLE) University of
Strathclyde Joint Meeting of QAA Integrative
Assessment and First Year Experience Steering
Groups Edinburgh University, 30 May 2006
2
My Plan

• Two recent literature reviews on assessment
  (synthesis)
• Case study of first year assessment redesign
  (drawn from REAP project)
• Analysis of case study in relation to assessment
  principles
• Identify relevance to broad aims of IA and FYE
  work

3
First Year The academic experience

• What is important?
• Coping with transition
• Understanding what is required
• Engagement with academic programmes
• Support and feedback
• Experiences of success
• Agents in own learning
• Belief in self (ability) and motivation
• Social dynamics of learning (belonging)
• Yorke and Tintos research

4
Gibbs and Simpson (2004)
Gibbs, G. Simpson, C (2004) Conditions under
which assessment supports students learning,
Learning and Teaching in Higher Education, 1,
3-31. See Formative Assessment in Science
Teaching (FAST) project at http//www.open.ac.uk/
science/fdtl/
5
Gibbs and Simpson (2004)

• Assessment tasks Conditions 1-4
• Capture enough study time (in and out of class)
• Are spread out evenly across timeline of study
• Lead to productive activity (deep vs surface)
• Communicate clear and high expectations

6
Gibbs and Simpson (2004)

• Feedback Conditions 5-11
• Is sufficient (in frequency detail)
• Is provided quickly enough to be useful
• Focuses on learning rather than on marks
• Is linked to assessment criteria/expected
  outcomes
• Makes sense to students
• Is received by students and attended to
• Is acted upon, to improve work and/or learning

7
Nicol and Macfarlane-Dick (2006)

• Literature review
• Nicol, D. Macfarlane-Dick, D. (2006).
  Formative assessment and self-regulated learning
  A model and seven principles of good feedback
  practice. Studies in Higher Education, 34 (1),
  199-218
• See
• Student Enhanced Learning through Effective
  Feedback SENLEF project funded by HE Academy
• REAP project www.reap.ac.uk

8
Research on Assessment in HE

• Teaching/learning paradigm
• Transmission

Constructivist
student-centred
Assessment paradigm Transmission
teacher-centred
9
Some key research findings

• Students are always engaged in self-assessment/sel
  f-regulation of their own learning (Winne, 2005
  Black Wiliam, 2005). Logically entailed by
  constructivist ideas.
• The act of using teacher feedback implies that
  self-assessment must be present (Sadler, 1983,
  Nicol and Macfarlane-Dick, 2006)
• Feedback in HE is being significantly reduced so
  how are students still learning?
• The question is how can we scaffold students
  learning so they become better at self-regulation
  of learning? (Lajoie, 2005)

10
Scaffolding self regulation 7 principles of good
feedback

• Clarify what good performance is (goals,
  criteria, standards).
• Facilitate reflection and self-assessment in
  learning
• Deliver high quality feedback to students
  feedback that enables students to monitor and
  self-correct
• Encourage peer and tutor dialogue around learning
• Encourage positive motivational beliefs self
  esteem through assessment
• Provide opportunities to close the feedback loop
• Use feedback information to shape teaching
• Source Nicol and Macfarlane-Dick (2006)
  Formative assessment and self-regulated learning
  A model and seven principles of good feedback
  practice, Studies in Higher Education

11
Re-engineering Assessment Practices REAP

• Psychology Case Study
• Context
• 560 first year students
• Mixture of psychology majors (130) and those
  taking psychology only for one year (430)
• 6 topic areas, 48 lectures, 4 tutorials, 12
  practicals
• Assessment 2 x MCQs (25), tutorial attendance
  (4), taking part in experiment (5), essay exam
  (66)

12
Psychology Case Study

• Problems identified with the course
• Students got no practice in writing skills but
  required in the exam
• No feedback except on MCQs (percent correct)
• Didnt want to increase staff workload
• Wanted to improve overall exam marks
• And standard of entrant to second year

13
Psychology re-design
Stage 1 Question 1 moderate difficulty (50
words) Individual response post it discuss
answer in groups (of 6) Timed release model
answer to self-evaluate their response Stage 2
Question 2 difficult (100 words) Group response
discuss (online) agree post response Model
answer released for stage 2 Stage 3 Question 3
complex (300 word essay) Group response
discuss (online) agree post Model answer
released for stage 3
14
Psychology re-design key features

• Writing tasks related to current lecture delivery
  (on human memory).
• 78 volunteers induction task
• Question of increasing difficulty and length of
  answer (50 words, 100 words and 300 words)
• Moving from individual to group responses.
• Scaffolding of learning through peer discussion
  (and cognitive growth thro conflict resolution)
• Comparing work against model answers
• Use of assignment/ discussion tools in (WebCT)

15
Planned developments

• Replace half the lectures with discussion tasks
• Tasks will become progressively more demanding
  within and across topics as the year progresses.
• Participation will be compulsory and monitored
• Once a final group response has been agreed each
  student will submit a copy to VLE (WebCT)
• Purpose is to check on participation rates and
  ensure that all students have at least had
  sight of the group response.
• Source Jim Baxter, Psychology, Strathclyde
  University

16
The seven feedback principles

• Standard format and model answers provide
  progressive clarification of expectations (clear
  goals, principle 1)
• Students encouraged to self-assess against model
  answer (self-assessment, principle 2)
• Online peer discussion aimed at reaching
  consensus about response (dialogue, principle 4)
• Staged complexity and focus on learning rather
  marks (motivation, principle 5)
• Repeated cycle of topics and tasks (closing gap,
  principle 6)
• Tutors can monitor progress and adapt (shaping
  teaching, principle 7)

17
Gibbs Simpsons four assessment conditions

• Assessment tasks require significant study out of
  class (capture sufficient study time, condition
  1)
• They are distributed across topics and weeks (are
  spread out evenly, condition 2)
• They move students progressively to deeper levels
  of understanding (productive/ deep learning,
  condition 3)
• There are explicit goals and progressive increase
  in challenge (communicates clear and high
  expectations, condition 4)

18
What can we learn from this case study?

• Use of a simple technology (discussion board)
• Considerable thought gone into the pedagogical
  design which is transferable.
• Supported by robust assessment/learning
  principles
• Coordinated approach within the department
• Evaluation showed a lot of supportive
  socialisation during induction task
• Intention to compare year on year examination
  performance.

19
Integrative Assessment and the FYE

• Supporting the FYE through assessment requires
  that learners have opportunities to evaluate
  their own learning and achievement. Integration
  occurs in a number of ways there is a need to
• Integrate self, peer and tutor assessment
  processes
• Integrate opportunities for self-regulation
  within organised academic structures and
  activities.
• Integrate formative with summative processes
• Integrate assessment into teaching and learning
  activities.

20
First Year The academic experience

• What is important?
• Coping with transition
• Understanding what is required
• Engagement with academic programmes
• Support and feedback
• Experiences of success
• Sense of agency in own learning
• Belief in self (ability) and motivation
• Social dynamics of learning (belonging)
• Yorke and Tintos research

21
Other relevant papers
Nicol, D (draft), Laying the foundation for
lifelong learning cases studies of technology
supported assessment processes in large first
year classes, British Journal of Educational
Technology. Nicol, D. Milligan, C. (2006),
Rethinking technology-supported assessment in
relation to the seven principles of good feedback
practice. In C. Bryan and K. Clegg, Innovations
in Assessment, Routledge.
22
Questions and discussion
23
Psychology

• Task 1 Define and describe structural encoding,
  phonological encoding and semantic encoding.
  Provide an example of each construct. (50 words,
  individual)
• Task 2 Describe the serial position effect and
  its two separate components. Discuss the
  specific structural components of memory that are
  responsible. (100 word, group response)
• Task 3 Summarise the stage theory of memory.
  To what extent does it provide an adequate theory
  of memory? (300 word, group response)