Adjusting students' attitudes so that they benefit from feedback

1
Adjusting students' attitudes so that they
benefit from feedback

• Quintin Cutts, Peter Saffrey
• Dept of Computing Science, Glasgow Uni
• Steve Draper
• Dept of Psychology, Glasgow Uni
• Emily Durrant
• Centre for Confidence and Well-being, Glasgow
• Contact quintin_at_dcs.gla.ac.uk

2
Carol Dweck - Mindset

• Two opposing attitudes to one's ability to learn
• Fixed mindset
• Ability is a fixed commodity
• Performance level demonstrates ability level
• All feedback interpreted as a measure of ability
• Failures hit motivation and self beliefs hard
• Growth mindset
• Ability can be grown through effort
• Feedback interpreted as information to assist
  growth
• Failure is an opportunity to gain feedback to
  learn
• Experiments show that Growth M/S can be cultivated

3
Messages about learning make a difference

• Mueller and Dweck 98
• "You must be smart at those problems"
• "You must have worked hard at those problems"
• Simpson, Open University 07
• Multiple short tutoring phone calls
• Strongly emphasised growth mindset and personal
  motivation in every call
• Dar-Nimrod and Heine 06
• Exposure to experience-based learning theory
  affected female performance in maths
• Aronson, Fried Good 01
• Writing exercise telling others about the growth
  mindset

4
Mindsets Programming

• Suggestion
• Learning to program at university can easily
  generate a fixed M/S
• Very high number of failure messages
• School university programming widely different
• Very wide ability range in Yr 1 classes
• Paradoxically
• Experienced programmers must have a growth m/s
• but a fixed mindset towards learners
  "programmers born not made"
• (Heslin, Wandewalle Latham Managers mindset
  affects quality of coaching)

5
3 Implementations of Dweck

• Explicit Mindset Training - (M)
• 4 x 10-minute teaching sessions, led by tutors
• Blackwell, Trzesniewski Dweck (2007)
• Crib sheet / Wiki - (W)
• originated from McCartney et al (2007) "35 ways
  that students get unstuck"
• Feedback 'inducement' - (F)
• Message on feedback sheet encouraging engagement
• Remember, learning to program can take a
  surprising amount of time effort students may
  get there at different rates, but almost all
  students who put in the time effort get there
  eventually. Making good use of the feedback on
  this sheet is an essential part of this process.

6
Experiment

• Context
• Level 1, 2 semester (24 wk) course
• 2 lectures per week 2hr lab/tutorial - 12
  tutorial groups
• Wk 6, Wk 13 class tests and Wk 12 lab exam
• Interventions spread across groups
• 2 Control, 2M, 2W, 2F, 1MW, 1MF, 1WF, 1WMF
• Tutor training sessions and delivery
• 1 for Wiki used from Wk 2
• 4 for Mindsets delivered in Wks 2-5
• no training for F sheet given in Wks 3, 5, 7, 9
• Measures
• Wk 1 and Wk 7 measures of mindset, self-efficacy,
  hope, /- affect
• Class tests and lab exam
• Self-reported measure of previous experience

7
Results

• Week 6 first Class Test (CT1)
• W, F main effects against CT1, p lt 0.05
• Week 13 second Class Test (CT2)
• F main effect against CT2, p lt 0.005
• Could this be due to imperfect random assignment
  of conditions?
• PREV0,1,2 against PREV3
• p lt 0.0007, effect size 0.78, mean diff 1.5
  coarse grades
• Uneven distribution of PREV3 among groups
• Test effect of F against PREV0,1,2 and PREV3
  separately
• adjusted effect size 0.36, mean diff 6, or 0.6
  of a coarse grade
• Time of day not significant against F
• but independently significant, effect size 0.5,
  or 0.9 of a coarse grade

8
What to make of it

• M and W imperfectly applied
• so cannot draw any strong conclusions from
  results
• F definitely delivering a result Why?

9
Interpretation by student of feedback

• Dweck suggests one very clear interpretation
• "This is saying something about my ability"
• but experience regularly shows strategic
  interpretation
• "It's saying something about whether I've put
  enough effort in"
• We provide f/b hoping to enhance deep learning
• "It's advising me how to improve my subject
  skills/knowledge"
• PDP training, and evident student behaviour,
  suggests
• "It's showing me how my study skills could be
  improved"
• Other interpretations are
• "The marker's an idiot, the feedback is wrong"
• "This has nothing to do with me, it's random"

10
Action follows interpretation

• Ability
• fixed mindset avoidance, drop-out etc
• Amount of effort
• if high enough, move on to something else, given
  my overfull life
• Improve subject skills/knowledge
• read feedback closely, incorporate into my
  understanding, via a range of practices
• Improve study skills
• reflect on activity, identify and carry out
  changes
• Mark/feedback is wrong
• initiate the complaints procedure
• Random
• Do nothing

11
(No Transcript)
12
Sheet is directive and informative

• right at the point of receipt of the feedback
• We state the interpretation we think is most
  important
• Learning to program takes time. You'll get there
  with sufficient effort. It's worth making use of
  this feedback. and we almost hide the 'mark'
• We give appropriate actions
• "Look at items" ltthe circled itemsgt
• "F/b points mostly worth reading, even if not
  identified for you"
• and the feedback is plentiful
• far more than any tutor could hand write
• Whole sentences, real explanations, with
  references
• Feedback sheets become a potentially major
  resource
• Not cheap to develop, but speeds up tutor marking
  time

13
Web questionnaire 35 responses

• 70 found the sheets (v.) helpful
• 50 motivated to work harder
• 55 find sheets helped later
• Comments
• Although the program may have worked, the sheet
  provided comments on how you could have improved
  the program
• gave me the confidence and then the joy to
  continue to work hard
• I was getting problems outlined - I started
  working harder on next programs, so I could sort
  these problems before submission. I have started
  meeting deadlines much more efficiently.
• Plenty of good common sense pointers are even now
  continuing to be helpful. Especially on
  structure.
• Feedback in this course is better than that in
  other courses. .

14
Conclusions

• Students amenable to subtle messages
• and in their absence, apply their own
  interpretations/actions
• We need to promote our attitude towards learning
• in this case, on the values we place on feedback
• and on the actions we expect them to take
• We're now applying the rubric across the class
• and will look for any effects in summer diet
• We'll be exploring the other two interventions
  further
• direct teaching about mindsets clearly valuable
  in other contexts
• giving students strategies for solving problems
  is a key part of the growth mindset