Postgraduates who Teach

1
Postgraduates who Teach
Assessment Workshop
Dr Siobhan Hugh-Jones Institute of Psychological
Sciences,University of Leeds November 2008
2
Aim of Session

• to provide some time and space to consider
  issues relevant to assessment practice.
• to provide some guidance on best practice.
• to address concerns / issues relevant to you.
•  

3
Why do we assess students?
4
Why Assess (Race, 2001)

• To classify or grade students to register their
  level of achievement / understanding.
• To enable student progression knowing students
  current levels of achievement so that they (and
  we) can know if they are ready to progress. 
• To guide improvement both the process of
  completing assessment (either summative or
  formative) and the grade and feedback received
  should enable development and further learning.
• To facilitate students choice of options can be
  judged on current abilities.

5
Why Assess? (Race, 2001)

• To diagnose faults and enable students to rectify
  mistakes Effective assessment lets students know
  where their problems lie, and provides them with
  an essential tool to put things right.
• To give us feedback on our teaching generally
  significant gaps in student knowledge often
  indicates faults in teaching. Excellent
  achievement by a high proportion of students is
  often due to high quality facilitation of student
  learning.
• To motivate students Assessment methods can be
  designed to maximise student motivation, and
  prompt their efforts towards important
  achievements.
• To provide statistics for the course, or for the
  institution
• To enable grading and final degree
  classification

6
You as an assessor

• Which of the purposes of assessment do you feel
  most competent / least competent in fulfilling?

7
Your students

• What do you think students say is the purpose(s)
  of assessment?
• To judge my level of understanding
• To tell me if Im right or wrong
• To see if Ive done what you told me to do

8
Assessment Good Practice

• Need to consider
• What has been asked of the student in the
  assessment?
• Includes any guidance / tips they have been given
• What teaching have they received on the topic,
  and what are the relevant learning objectives?
• Is this assessment a measure of the learning
  objectives?
• What are the expected achievements for the level
  the student is at?
• What are the assessment criteria?

9
Example

• Assignment Title
• Write a 200 word abstract for a paper you want to
  submit to the British Journal of Social
  Psychology on the topic of sexism.
• Your assessment of it would vary depending on how
  much help students were given, how much guidance
  about the expected way to do it, the extent to
  which it is a measure of module learning
  objectives (or is it formative?).
• Bearing all of this in mind, would then assess
  the piece according to your published marking
  criteria.

10
You being assessed

• What are your recollections of being assessed as
  a student?

11
Novice Assessors findings from our MARK project

• Our research indicates that novice assessors
  tend to be quite heavily influenced by their own
  experiences of being assessed, and by the quality
  of work they themselves produced.
• More experienced assessors (3 yrs), who have
  been exposed to a broader range of work, and
  different ways in which students can produce good
  work, tend to have more open, flexible views of
  each grade of work.
• Novices / non-subject experts tend to be more
  sensitive to isolated features of student work,
  e.g. use of certain references, unique points.
  May implicate more fragile mental model of the
  quality of the work.
• Novices tend to rely more on the process of
  giving feedback to formulate their marks, whereas
  more experienced assessors tend to award marks
  first, then give feedback.

12
Novice Assessors findings from our MARK project

• Implications
• To appreciate that becoming a good assessor is
  necessarily a developmental process that requires
  exposure to very different types and levels of
  work, and to second-marking processes that enable
  benchmarking
• To be attentive to your own assessment
  experiences and how they may be affecting your
  assessment practice
• To be cautious that your view of a piece of work
  is not affected disproportionately by isolated
  features of the work

13
Using Marking Criteria

• usefulness of criteria often depends on how
  detailed they are
• try to judge the piece of work according to each
  of the prescribed features
• argument, use of evidence, structure, coherency,
  answering the question, critical thinking,
  discussion
• are these equally weighted in your criteria? If
  so, need to give equal consideration to all.
• marking criteria are often hard to apply across
  different forms of assessment.

14
More General Advice

• All of us need to go through 3-4 scripts to
  benchmark standards.
• May return to these earlier ones to raise /
  lower.
• Watch out for prejudice.
• There will be all sorts of things that you dont
  like handwriting style, pen colour, layout of
  scripts, font, the student etc. Make sure you are
  not influenced by these.
• Avoid the halo effect.
• after a brilliant one, next will look poor in
  comparison.
• Take advantage of the second marker system
• Dont assume longer is better
• Concise writing is a skill, as is knowing what to
  leave out

15
Tricky ones

• Borderlines and high Firsts
• look carefully at criteria
• ask what more the student would have to do to get
  the next highest mark
• compare to others in your batch
• highlight for second marking

16
Giving Feedback

• Why do we give feedback ?
• Characteristics of good feedback
• Characteristics of poor feedback

17
The not so good.

• Good but wrong!
• I know what you mean but do you?
• Waffle!
• No!
• Have you been to any of my lectures?
• An excellent answer only let down by its complete
  failure to answer the question!
• Have you ever been to the library?
• Your answer moved me - to tears!

18
Any good?

• Intro lousy, results part OK. No conclusion. Give
  some refs. next time. 55
• Good first attempt. Into too long. Try to
  summarise findings in the concluding paragraph
  and do provide more refs.
• Your style of writing is appalling. Have you not
  read the practical handbook? You have no
  conclusion, a vague rambling introduction and
  insufficient references. This is a University
  Department not a kindergarten. 55.
• Clarity of handwriting is no substitute for
  clarity of argument. You have neither. Need I say
  more? 55.
• David. Ive made some detailed comments below.
  The numbers refer to the relevant parts of the
  practical. It had the makings of a good practical
  but you need to state the hypothesis clearly in
  your intro. Remember the discussion is as
  important as the intro and do provide references.
  These comments are important to bear in mind
  before writing other practicals. 55.

19
Feedback has been shown to be most effective when
it is

• timely
• perceived as relevant
• meaningful
• encouraging
• and offers suggestions for improvement that are
  within a students grasp.

20
So it follows that.

• We should return assignments promptly.
• Avoid long woolly and short incisive comments.
• Aim for a few succinct supportive statements that
  encourage development feedback sandwich.
• Choose a few points that will produce the
  greatest improvements and hold back on the rest.
• For the next piece of work, it would be good to
  see you focus on use of more recent literature
  and to attempt a more confident conclusion. Do
  maintain your clear style of writing and your
  focus on the specifics of the questions.

21
And..

• We should identify what is good as well so the
  student knows what does not require extra
  attention
• Simply noting errors is useless unless the
  student knows how to address them
• A balance of praise and criticism is going to be
  much more effective than an exercise in
  destruction.
• Our MARK findings students more eager to get
  direction for improvement than just praise /
  encouragement.

22
Helping progression

• If a student is at the borderline for an overall
  practical mark or for a sub-section than it is
  particularly helpful for them if you identify
  what they could have done to hit the next degree
  class mark
• e.g. this results section was very good most
  areas and almost 1st class to improve your
  marks for this section you needed to talk more
  extensively about the descriptives as there were
  some interesting trends evident from the table of
  means this would also have given you some extra
  material to tie into the post hoc tests and also
  talk about in the discussion.

23
Sample

• What is meant by personality disorder and for
  psychologists have a good understanding of it?
• First Level 1 essay

24

• This essay will consider what is meant by
  personality disorder and will look at the
  evidence for them. Personality disorders are
  diagnosed according to the diagnostic criteria in
  the DSM-IV-TR (2004) and are very previlent in
  the west. Psychologists are still conducting
  considerable research to understand the nature of
  personality disorders, as well as their causes
  (Harris, 2003), but as yet there is no definitive
  conclusion as to why they occur. The most common
  disorders are schizophrenia, bi-polar disorder
  and narcissictic personalities.
• Schizophrenia has a prevalence rate of
  1.3 in western populations (Keys, 2006) and is
  the best understood of all personality disorders.