Let

1
Lets Abolish Feedback

• Brian Whalley
• (Lately of Queens University of Belfast)

and put something better in its place
2
well be

• Looking at existing feedback
• Looking at what feedback is, variously
• Looking at some simple educational systems
• Marks, remarks and students needs
• Making suggestions for improvement

3

• Knowledge one has acquired without sufficient
  structure to tie it together is knowledge that is
  likely to be forgotten. An unconnected set of
  facts has a pitiably short half-life in memory.
  Organizing facts in terms of principles and ideas
  from which they may be inferred is the only known
  way of reducing the quick rate of loss of human
  memory . (But) designing curricula in a way that
  reflects the basic structure of the field
  requires the most fundamental understanding of
  that field.

(Bruner, J. S. (1960). The process of education.
Bruner, J. S. (1960). The process of education.
4
The NSS and the feedback issue

• What do the data really mean?
• An example of institutional panic
• I dont think they mean a great deal!
• even if they do then how can they be improved
  given that
• feedback is the best way to improve performance
  (see Nichol, Gibbs, Race, and especially Geoff
  Petty (Evidence-based Teaching)

5
What is feedback

• It depends upon whom you ask!
• Plenty of good statements so well look at a few
  consequences
• Nicol and Mcfarlane Dick
• Alastair Irons
• (Enhancing learning through formative assessment
  and feedback 2008)

6
The Ten Principles of Assessment and Feedback

• 1. Help clarify what good performance is
• via aims, criteria, standards
• 2. Encourage time and effort on challenging
  learning tasks.
• 3. Deliver high quality feedback information that
  helps learners self-correct.
• 4. Encourage positive motivational beliefs and
  self-esteem.
• 5. Encourage interaction and dialogue around
  learning (peer and teacher student)

(Nicol 2007)
7
10 principles continued

• 6. Facilitate the development of self-assessment
  and reflection in learning.
• 7. Give learners choice in assessment content
  and processes.
• 8. Involve students in decision-making about
  assessment policy and practice.
• 9. Support the development of learning
  communities.
• 10. Help teachers adapt teaching to student needs.

How do we use exams (and coursework) to deliver
these?
8
Nine disadvantages(From Irons, p 25-6)

• Students dont make use of feedback
• Doesnt actually contribute to learning
• Only there to justify the mark given
• May be categorial in tone and not explicit
• No opportunity to enter into dialogue
• Emphasises power relationships
• The potential to include bias
• May actually foster rote learning
• Might be inappropriate

To some, possibly large, degree these are because
of differences in what is meant by feedback.
9
If feedback is so important then why is it
referenced so infrequently? Where do you find a
definition?

• Well,
• we all know what feedback is, dont we?
• Dont we?

10
your module (swap)(5 minutes)

• Draw a time line (start of module to end)
• Tick for start (when do you start?)
• Tick for end (what denoted the end?)
• Note the hours to be spent (lectures etc)
• Put a symbol to mark any Assessment event
• Put a symbol to show when feedback is given
• Note below the type of feedback given (however
  you define it)

• Reflect, are you happy with this?
• Discuss your chart with a neighbour
• Are you happy with each others, and indeed, the
  feedback process?

11
when to give feedback

• When can you?
• When should you?
• When do you?
• Do others in your department agree on when you
  do?
• Do others in your department give proper
  feedback anyway (module scheme)?

12
A definition
13
Feedback and Feedforward in a control system
sense

Do exams provide a reference input (or even
output?) A MCQ might do this. Criterion
Referencing might too.
14
Educational ramp up

• To raise the level of attainment
• We need controlled input, ramping up.
• Output, achievement, results

Semester start
Semester end
Ee
Marks, no remarks for Exam events
15
Can I be more constructive?
How do we increase the gradient?
Task Activity Marks Remarks
Two Activity c/work
Two Activity c/work exam
Gradient learning rate
16
Allison Rossett(First Things First)
Performance analysis
Training needs assessment
Revised Courses
Current
Optimal
Performance Support
Key Areas of Need
Training needs assessment
Job Aids
Recommended Solution System
New Courses
E-Learning
Solution Partners
The seductive nature of the new media is another
good reason to add performance analysis right up
front. Performance analysis keeps the focus on
the customers and their purposes. The goal is to
find the right bundle of interventions, not to
take a spin with podcasts or Second Life.
17
Everyone needs trainers
Umm, sometimes it works
Teamwork Feed forward It is not judgemental.
The negative connotations of past failures are
banished There is no such thing as failure just
Feedforward. It is much easier to deliver.
People are less defensive when discussing future
performanceFeedforward is taken less personally,
provokes less resistance.
Marshall Goldsmith
18
Itiel Dror

• Control handing appropriate responsibilities to
  students
• Challenge student abilities
• Commitment student commitment to learning
  opportunities

What might be the best ways (note plural)
to develop these?
Using cognitive psychology
19
6 Competenciesstudents need to gain
Marcia Mentkowski
Mihály Csíkszentmihályi

• Competence encouragement by challenge and
  remarks to achieve skills levels
• Confidence promoting remarks to show
  themselves, and others, their achievements
• Critical thinking which is what we have been
  wanting all along in 'Thinking skills, used in
  problem solving
• Creativity in what students do and how they do
  it
• Collaboration bringing in team-working and
  ethics
• Commonality of purpose, to achieve specified
  (and unspecified) objectives
• Curtiosity which is more than curiosity.

20
Ways forward (rather than conclusions)

• Develop a broad skills base
• Include employability skills
• Use Problem-based learning
• Aim for the 6Cs
• Use the 3Cs
• And charting (etc)

Confidence Critical thinking Creativity Collabor
ation Commonality Curtiosity
Control Challenge Commitment
21
Assessment for Learning(Geoff Petty -
Evidence-Based Teaching)

• Seven principles common to high-quality
  learning and achievement (p164)
• Students must see the value of the learning
• Students must believe they can do it
• Challenging goals
• Feedback and dialogue on progress to the goal
• Establishing the structure of information and so
  its meaning
• Time and repetition
• Teach skills as well as content

22
Feedback and performance

• This is not as difficult as we might think (ahem)
• Changing the way we work at things
• Using informal learning and performance ideas

23
Ron Oliver et al. Schema, also
Task
Task
Problem specified
Manipulation of system
Task
Problem solution
S
Assessment
T
R
R
Rules System Examples Procedures
Clues Guidance Help
Provision of Feedback on solutions
Support
Support
Resources
T Tacit knowledge required
S Sticking point(s) likely
Fig 5.1 A temporal sequence describing a
rule-based learning design, In Oliver et al.
2007. Describing ICT-based learning designs that
promote quality learning outcomes
24
Communities of learning

• We need to develop these
• At a variety of scales of operation -
• In particular, at our own institutions -
• Student-centred but bringing in
• Tutors, pedagogy, employers, internet,
• Web 2.0, digital repositories,etc
• Wisdom of Crowds (Surowiecki, Shirkey) etc
• Using cognitive psychology

Collective Learning Consuming knowledge
Connecting knowledge Contributing knowledge
Charting knowledge
25
Assessment

• Alignment?
• To fit with diversity of skills as well as
  instruction methods etc
• E-portfolios
• Transcripts and record of achievements
• Leading back to employability

26
Reasons to be cheerful?

• If teaching was as simple as telling wed all
  be a lot smarter than we are. (Mark Twain)
• In Carol Dwecks terms, it takes an
  incrementalist approach to learning which
  emphasises effort and improving competence. This
  contrasts with an entity approach, which
  attributes learning to ability and is focused on
  proving competence through grades and
  comparisons.
• Gordon Stewart (2008)

27
Towards a proper educational feedback system

• What do I mean by this?
• Forget about Kolb cycles and Cowan loops or,
  perhaps better, use them to give a system that
  works for all students (and staff)
• Construct (with Alignment) a system (for a
  module) that maximises learning and understanding
  
• (rather than memory recall)
• Use Thinking Styles (Sternberg) rather than
  Learning Styles
• To produce proper alignment we really need more
  specific instructions per task/activity
• And to provide remarks with the marks is use a
  feedback in an educational control system

28
Delays and Troublesome Knowledge Sticking Points

• Troublesome knowledge (Land)
• Tacit knowledge (Polanyi)
• Delays doing activities Sticking Points
• Preflights (Just-in-Time Teaching G Novak)
• Feed-forward (controlled use of advance info)
• After Action Reviews
• ve Feedback or remarks

29
Things we can do, 1, general

• Recognise the needs of students and their
  antecedent conditions
• Too much winging it on what they (think they)
  know how and what to do.
• Recognise the tutors assessment nd feedback
  provision is by no means tried and trusted (was
  it ever satisfactory, let alone good)
• Recognise that timing is important

30
Preflights(Warm-ups)

• Tasks set before the event for students to do
• Could be assessed or not (gratuitous marks)
• Confidence building (in you, for themselves)
• Tackle Troublesome knowledge beforehand
• Gets them to start something required later
• helps to avoid last-minute syndrome
• Can reflect on the task and achievement
• Make them easy to assess (if required)
• Follow up with After-Action Review (reflection)
• Link in to Criteria-Referenced Assessment

31
PFL examples

• Look up definitions (e.g. on Wikipedia)
• Prepare a bit of a spreadsheet formulation
• Write a bit of HTML script
• Prepare a PDF of a document
• Prepare the basis for a digital poster
• Analyse a photograph (prepare metadata)
• Extract some critical data from a table
• Prepare a Concept Map of a problem
• Find an Open Resouce app for a task

32
Criteria Referencing

• Shows students what is wanted in advance
• Shows the mark scheme related to requirements
• Tutors Marks and Remarks (together or not)
  provide External feedback for the system
• Easy to get them to reflect (ie they provide the
  feedback) Internal Feedback for the system
• Easier to plan what is required from the
  assessment
• Important for cross-checking (eg demonstrators)
• Easier to show external references (Externals)
• Pretty much what is meant by feed-forward (ie
  showing the limits for energy input into the
  system)

33
What drives learning

• Not the assessment (itself)
• This (best) drives checking (controller in the
  system). Its an indicator (control signal)
• Driving learning (ve feedback)
• Internal (student doing the activity)
• External (comments from others, peers, tutor,
  manuals)
• Inhibitors to learning (-ve feedback)
• No remarks (feedback)
• Exams with no remarks (feedback) before the end
  of the module

34
students (tend to)revert to (school) type

• expectations of small classes with
• teachers they know well and who
• can give pretty immeiate remarks and even
• return work for repairs and re-submission
• may not be using good cognitive techniques which
  often rely on or use
• braindumping techqnuies fro assessment

35
Assessment for Learning(Schools are using this!)

• making exams less dominant
• and why should they be anyway?
• what do marks and remarks mean anyway?
• having care of customer for students,
  especially in year 1 (and Semester 1)
• Use class tests (MCQ, Certainty-based)
• Formal exams before the end of the semester (and
  proper remarks with the marks to the student -not
  just the External)

36
Care for customer

• Look after Level 1
• Make the assessment explicit
• Criteria Referenced
• Match marks to criteria (explicitly)
• Give students practice in assessment (types)

37
If we believe in these principles of deep
learningthen (in general)

• Why do we set essay exams?
• and seem to rely on closed exams?
• and not used open book or seen?
• Why do we set exams at the end of modules?
• Why do we not give feedback on exams
• Why do we believe MCQs are ok better?
• Why there are several cetls involved with active
  learning etc but few with Assessment?
• and you can probably think of more

38
Ways forward (rather than conclusions)

• Develop a broad skills base
• Include employability skills
• Use Problem-based learning
• Aim for the 6Cs
• Use the 3Cs
• And charting (etc)

Confidence Critical thinking Creativity Collabor
ation Commonality Curtiosity
Control Challenge Commitment
39
some references and websites

• Irons, A. 2008. Enhancing learning through
  formative Assessment and Feedback. Routledge.
• Rossett, A. First Things First (A handbook for
  performance analysis) Pfeiffer 2009.
• Siemens, G. Knowing Knowledge
• Sternberg, R. J. (1997). Thinking styles.
  Cambridge University Press.
• Stewart, G, Testing Times 2008 Routledge,
• Stobart, G. (2008). Testing times the uses and
  abuses of assessment. London Routledge.
• Sweller, J. (1994). Cognitive Load Theory,
  learning difficulty, and instructional design.
  Learning and Instruction 4 295312.

40
Things we can do, 2timing

• lead knowledge, skills, experience from start to
  finish
• clearly

41
practice and Kahneman and Tversky
42
to add

• Maslow needs?
• Informal learning
• Learning styles?
• Thinking styles
• Peer learning
• preflights and JiTT