Feedback: written product to relational process

1
Feedback written product to relational process

• Berry ODonovan
• Deputy Director ASKe Centre for Excellence
• Head of Learning and Teaching Development,
  Business School, Oxford Brookes
• ASKe Centre for Excellence
• (Assessment Standards Knowledge exchange)
• www.business.brookes.ac.uk/aske.html

2
Starting points assessment is a key driver of
student learning

• Assessment is at the heart of the student
  experience
• (Brown Knight, 1994)
• From our students point of view, assessment
  always defines the actual curriculum
• (Ramsden,1992)
• Assessment defines what students regard as
  important, how they spend their time and how they
  come to see themselves as students and then as
  graduates.........If you want to change student
  learning then change the methods of assessment
• (Brown, et al, 1997)

3
But there are problems

• QAA subject reviews
• National Student Satisfaction survey
• the Achilles heel of quality (Knight 2002a, p.
  107)
• Summative assessment practices in disarray
  (Knight 2002b, p. 275
• Broken (Race 2003, p. 5)
• There is considerable scope for professional
  development in the area of assessment (Yorke
  2000, p7)
• Rising concern about cheating and plagiarism

4
Problems contd.

• The types of assessment we currently use do not
  promote conceptual understanding and do not
  encourage a deep approach to learningOur means
  of assessing them seems to do little to encourage
  them to adopt anything other than a strategic or
  mechanical approach to their studies.
• (Newstead 2002, p3)
• Even when lecturers say that they want students
  to be creative and thoughtful, students often
  recognise that what is really necessary, or at
  least what is sufficient, is to memorise (Gibbs,
  1992, p.10)

5

• Feedback is the most powerful single pedagogic
  influence that makes a difference to student
  achievement
• Hattie (1987) - in a comprehensive review of 87
  meta-analyses of studies
• Feedback has extraordinarily high and
  consistently positive effects on learning
  compared with other aspects of teaching or other
  interventions designed to improve learning
• Black and Wiliam (1998) - in a comprehensive
  review of formative assessment
• Students are hungry for feedback to develop
  their learning
• (Higgins et al, 2002)

6
Again, there are problems

• Unhelpful feedback (Maclellan, 2001)
• Too vague (Higgins, 2000)
• Subject to interpretation (Ridsdale, 2003)
• Seen to be too subjective (Holmes Smith, 2003)
• Not understood (e.g. Lea and Street, 1998)
• Damage self-efficacy (Wotjas, 1998)
• Dont read it (Hounsell, 1987)
• Has no effect (Fritz et al, 2000)

7
Feedback, so what works in practice?

• My practice
• Large classes
• First year undergraduates
• Satellite campus
• A discipline that is not always positively
  selected

8
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but complex no single right
  response

9
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of work 24 hours
• Resource efficiencies?
• Large class approx 400
• one-hour lecture
• back-to-back one hour seminars (some tutors
  do 4)

10
Resource efficient ways of giving feedback

• Resource efficiencies more difficult to capture
  for feedback?
• Some useful techniques for unambiguous,
  instructional and directive feedback, more
  difficult for more open, interpretable,
  transformational feedback.
• There are times when students want / thirst /need
  to discuss their work with a trusted tutor.
• (Price et al, in submission ODonovan et al,
  2001)

11
There are some points in a programme of study
where students may require substantive/transformat
ional feedback from their tutors

• First year orientation (Yorke, 2007)
• Radical movement involving zones of discomfort,
  threshold concepts and troublesome knowledge
  (Meyer and Land, 2006)
• Changes in epistemology and knowledge
  structures (Basil Bernstein in Moore et al,
  2006)

Learning development
Year 1
Year 2
Year 3
12
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted end of week 3 with self-assessment

13
Setting students expectations through early
assessment and feedback

• A significant number of first year students say
  they have difficulty in coping with academic work
  (Yorke and London, 2007 Krause et al, 2005)
• Diagnostic assessment and formative feedback can
  help students appreciate what is expected (Yorke,
  2007)

14
Clarifiying expectations

• Frank In response to the question, Suggest
  how you would resolve the staging difficulties
  inherent in Ibsens Peer Gynt, you have
  written Do it on the radio
• Rita Precisely.
• Frank Well?
• Rita Well what?
• Educating Rita cited in Yorke 2007

15
Clarifying student expectations through early
assessment and feedbackAlso engender good study
habits diagnose areas of individual difficulty
promote engagement

• Setting students assignments as soon as they
  arrive at university could help cut dropout
  ratesintegrate students into university life
  as quickly as possible. This involves making them
  aware of the quality and quantity of work
  expected from them..The freedomis too much for
  some and they probably need more structure in the
  first year.
• Higher, 24/1/03
• Reporting Student Transition and Retention (STAR)
  project
• (in 5 universities)

16
Enhancing students self-evaluative ability

• Students holding an understanding of quality
  roughly similar to that held by their tutor(s) so
  that they can evaluate their work is an
  indispensable condition for enhancing students
  academic performance (Sadler, 1987 Sadler, 2008)
• At best, many programme assessment frameworks
  communicate understandings on quality to
  students serendipitously over time and often by
  an undermining process of elimination (ODonovan
  et al, 2004)

17
Developing self-evaluative ability

• Students need to understand the assessment
  standards and criteria to be able to
  self-evaluate their work in the act of production
  itself (Sadler,1987)
• The ability to make informed judgements on the
  work of self and others is a key graduate
  attribute (Boud, 2009)
• Marking practice with exemplars (ASKe 123
  leaflet)
• Facilitated self-assessment (e.g. generic
  feedback)
• Peer review and peer assessment
• Drafting and re-drafting

18
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted end of week 3 with self-assessment
• Formative in intent

19
Summative vs. formative assessment

• Measurement vs. development
• Summative contrasts with formative assessment
  in that it is concerned with summing up or
  summarising the achievement status of a students,
  and it is geared towards reporting at the end of
  a course of study especially for purposes of
  certification. It is essentially passive and
  does not normally have immediate impact on
  learning. (Sadler, 1989)
• Increased measurement - increased instrumentality
  - less learning (Gibbs Dunbar-Goddet, 2007). A
  progressive abandonment of time on tasks that
  carry no marks (Price et al., in press)
• Feedback without marks has the most effect on
  learning (Butler, 1988). OU has successfully
  increased retention by making early assignments
  feedback only
• However, formative assessment is not less
  resource intensive than summative assessment
  (Price et al, in press)

20
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted 3 weeks into a large business degree
  course
• Formative in intent
• Purpose diagnostic/forensic and feedforward

21
Purpose of feedback

• Correction
• Justification of a mark
• Benchmarking
• Feedforward/development
• Forensic diagnosis
• To be effective the purpose of feedback needs to
  be clarified and communicated to both staff and
  students.
• Price et al, (in submission) Feedback all that
  effort, but what is the effect? Assessment and
  Evaluation in Higher Education

22
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted 3 weeks into a large business degree
  course
• Formative in intent
• Purpose diagnostic/forensic and feedforward
• One week to complete marking and feedback

23
The Temporal Dimension of the Feedback
ProcessEngaging Students with Assessment
Feedback FDTL Project

• Feedback Moment or Process?

Opportunity to use feedback
Feedback moment
Submission
Feedforward
Feedback
Student still connected to assignment
Specific with immediate applicability
vs. Transferable aimed at longitudinal
development
24
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted 3 weeks into a large business degree
  course
• Formative in intent
• Purpose diagnostic/forensic and feedforward
• One week to complete marking and feedback
  before moderation processes
• Marking my own groups

25
The relational dimension

• To engage with feedback students need to
• Believe that the assessor has carefully read
  their piece of work
• Respect and trust the assessor
• Believe that the feedback will be useful
• Consequent difficulties of annonymised marking
• (FDTL Engaging Students with Assessment
  Feedback)
• Findings Song Another Tick in the Box

26
Barriers to Relationships?

• Programme level
• Anonymous marking
• Associate/part-time staff (no time or space)
• Academic staff have multiple roles teaching
  research admin
• Modular programme discontinuity of teaching
  staff
• Resource constraints time per student
• Modular level
• Focus on content delivery rather than development
  and dialogue

27
Feedback, what works in practice?

• Case study my practice last week
• Marking 46 pieces of first year UG work
• Short (750 words) but discursive and complex
• Submitted 3 weeks into a large business degree
  course
• Formative in intent
• Purpose diagnostic/forensic and feedforward
• One week to complete marking and feedback
  before moderation processes
• Marking my own groups
• Oral face-to-face feedback tutorials (selected
  students)

28
Importance of dialogue

• Even when written feedback is lovingly crafted,
  students may be disengaged
  (Gibbs Simpson, 2004, p. 20).
• The relational dimension within the process is
  key to student engagement. Dialogue supports
  understanding and engagement
• (Engaging Students with Assessment Feedback,
  FDTL)
• Limitations of written feedback
• Passivity
• Interpretation
• Feedback on complex, high level tasks require
  clarification and dialogue

29
Relational dimension and dialogue

• With staff
• Feedback on Self Assessment
• Case study 5mins for 140 students
• Personal tutor consultation
• With peers
• Peer review and peer assessment
• Peer assisted learning
• Feedback methods
• Oral rather than written
• Exemplars
• Generic feedback and feedback workshop

30
Feedback what works in practice?

• Preparation and setting expectations early in a
  programme
• Identifying feedback moments and application
  opportunities within the programme
• Emphasize the relational dimension of the
  feedback process
• Building in space for dialogue across a
  programme