Assessing Contributions to Group Assignments

1
Assessing Contributions to Group Assignments

• Lucy Johnston and Lynden Miles
• University of Canterbury, New Zealand

Amy Maiden, Rachael Ramsbottom, Abbi Raine,
Samuel Pugh and Adam Pudvine
2
Introduction

• Our group looked at the subject of
• self and peer assessment
• Group learning with Individual reports
• The marks for the report were moderated by self
  and peer assessment

3
Group Benefits

• Other peoples points of view
• Develop teamwork skills
• Communication
• Leadership
• Planning and organization
• All are useful in the job world
• What University is all about.Supposedly

4
Group Problems

• Grading contribution
• Awarded for contribution
• Good grade less work
• A Very Happy Student
• Input Vs Output

5
Why use peer assessment?

• Monitoring group contributions
• This is where Peer Assessment is useful.
• Group members
• Peer Assessment grade
• Addition or
• Moderator
• To report grade

6
Aims of the study

• One aim looked at the
• Effects Peer Assessment has on
• Group Learning with individual reports
• Individual reports dont always expose free
  riders
• Take, Take, Take!
• A second aim was
• Self assessment
• Inclusion or Omission

7
Aims of the study

• Some researchers argue for inclusion and some
  against inclusion.
• The study included self evaluation but considered
  consequences of omitting the self evaluation.
• (Goldfinch, 1994) argued
• Nice Guys finish last!
• Self assessment allows balance.
• (Krause Popovich, 1996) Proves this is
  incorrect.
• The study we looked at aims to shed more light on
  this matter.

8
Method

• Participants
• 61 undergraduate students
• 48 females and 13 males
• 12 groups were self-selected by the students
• 3 groups (n 2) 4 groups (n 10) 5 groups (n
  3)
• 7 groups comprised of males and females.
• 5 groups comprised of females only.

9
Method

• Aim
• To give the students experience in designing and
  conducting a piece of social psychological
  research.
• 3 topic areas
• Investigating the relationship between empathy
  and personal behaviour.
• Testing the Ideal Standards Model (Fletcher et
  al., 1999 Simpson et al., 2001) in mate
  selection.
• Investigating the nature of impression formation.

10
Method

• Contributions Questionnaire
• Each student rated the contribution of each
  member of their work group, including themselves
  on task functions and group management.
• Generation of research ideas
• Contribution to background literature searches
  and reviews
• Contribution to research design and methodology
• Contribution to data collection
• Contribution to data analysis and interpretation
• Contribution to feedback presentations
• Contribution to project management

11
Method

• A score between 3 and -1 was given for each of
  these tasks.
• 3 major contribution, 2 some contribution, 1
  minor contribution, 0 no contribution, -1 a
  hindrance to the group.
• The average rating out of 21 was calculated for
  each group member and the average rating for each
  group was calculated.
• More than an average contribution exceeds the
  1.0 ratio increase in mark
• Less than average contribution less than 1.0
  ratio decease in mark.

12
Results Contributions Ratings

• A repeated measures ANOVA (rating of self others
  rating of self and rating of others in a group)
  showed a significant effect F(2,120)3.49,Plt0.05
  .
• Ratings of the self were significantly higher
  (Fisher LSH, Plt0.05) than others rating of the
  self and rating of others in-group, the latter
  two not differing from one and another (Means
  18.47 versus 17.09 and 17.00).
• Individuals believed they contributed more.

13
Results Contributions Ratings

• No difference between own contribution and the
  mean group contribution. Suggesting little
  difference in the scores of the group members
  within a given workgroup.
• Reflected in mean and median values for the
  contribution index being close to 1.0 (mean
  1.002 and median 1.008)
• Few significant effects were shown on impact of
  the sex of the student, size of group and type of
  group (mixed sex or all female)

14
Results Contributions Ratings

• No significant effects of sex of student.
• There was an effect of the size of the group on
  ratings of self, participants rating own
  contribution lower if in a larger group.
• No difference between the mean scores of those in
  mixed sex work groups and female only work
  groups.
• Pearson correlation coefficients were calculated
  between ratings of the individual by themselves
  and by other group members.
• No significant correlations in the perceptions of
  an individual by themselves and by other group
  members.

15
Results Contributions Ratings

• No correlation between the ratings of ones own
  contribution and other group members r(61)
  0.155, ns, suggesting a differentiation of own
  contribution from that of other group members.
• Individuals rate their contribution higher than
  that of other members of their work group.
• Suggests individuals are distorted in the their
  view of their own contribution.

16
Results Contributions Ratings

• Kendalls measure of concordance was calculated
  in each work group, a measure of agreement in the
  ranking of items across judges and ranges (0, no
  agreement 1, perfect agreement).
• No effect of sex of the group on Kendalls
  coefficient of concordance. There was however,
  an effect of group size within higher concordance
  for smaller groups.

17
Results Contribution ratings and Performance

• The marks awarded for the written assignment,
  before and after the adjustment by the
  contribution index are shown in Table 2.

18
Results Contribution ratings and Performance

• No effect of group type or group size on
  assignment marks. There was however a
  significant effect of sex on the assignment
  marks.
• Female students received higher marks before
  adjustment and was enhanced after adjustment for
  group contribution.

19
Results Contribution ratings and Performance

• There was no significant difference between the
  assignment marks before and after adjustment by
  the contribution index.
• Consistent with a mean contribution index close
  to 1.0 there was very little mean change in the
  marks awarded.
• However, there was a wide range of changes to
  marks from a decrease of 22.12 to an increase of
  20.18.

20
Results Contribution ratings and Performance

• Table 3 shows the correlations between the
  contribution ratings and assignment marks.

21
Results Contributions Ratings

• A significant relationship between others rating
  individuals marks and between the contribution
  index and marks was found.
• Lejk and Wyvill, (2001) suggested self-assessment
  varies as a function of ability level.
• High ability individual under rates self
• Low ability individual over rates self
• Investigated using a quartile spilt on the
  unadjusted marks to categorise the individuals on
  their level of ability. A discrepancy score
  wtaken for each individual between
  self-assessment and peer assessment.
• Positive scores higher self- then peer ratings
  and negative scores the opposite.

22
Results - Strategies

• Surprisingly there was no instance of group
  collaboration.
• Only 16 (26) students gave the same rating for
  all members including themselves.
• A further 2 (3) rated themselves differently to
  the others who they rated as the same.
• This follows Kendalls concordance measure,
  indicating that the students took the task
  seriously.

23
Results - Strategies

• Self bias could be the explanation for the lack
  of group strategies developing.
• Confidentiality allowed students to ignore group
  decided strategies if they chose to do so.
• A member may mark themselves highly and give the
  other group members low scores in order to
  increase the rating gap, indicating their higher
  contribution.
• However, the same pattern may be adopted by a
  more under contributing member to mask their
  inadequacy.

24
Results - Strategies

• The three top and three bottom scoring students
  scores were looked at in more detail.
• It is evident that students preferred to inflate
  their own contribution rather than devalue
  another members.
• Interestingly two of the groups included both a
  top scoring student and a bottom scoring student,
  this could be a result of one member of the group
  compensating for anothers inadequacy.

25
Results Alternative Approaches

• Removing the self-assessments and using
  contribution index to allocate additional marks.
• There was no correlation between self and peer
  assessment, also no correlation between self
  assessment and actual marks awarded.
• So does self-assessment hinder validity of marks
  more than just peer-assessment?
• Revised contribution index mean
  peer-assessment/mean group contribution (omitting
  influence of self-assessment).

26
Alternative Calculations of the Impact of the
Contribution Index
27
Results Alternative Approaches

• In the revised version there was a very strong
  correlation between the two contribution index
  measures r(61) 0.945, Plt0.05
• In both versions of the contribution index the
  correlations between each of the indices and the
  assignment scores are very similar.
• The impact on the assignment scores using the two
  measures of contribution index correlated highly,
  as did the two final assignment scores.

28
Results Alternative Approaches

• In the assessment ratings and actual marks were
  integrated, another approach however would be to
  determine a percentage of the final grade by each
  aspect.
• For example, written assignment 15,
  contributions rating 5. But how many marks to
  allocate for contribution out of five for an
  average student?
• If 65 was the mean for marked assignments prior
  to adjustment an average score would be 3.25/5
  for an average student in the new measure.
  (impact of these different measures is seen in
  table 4)

29
Discussion

• Discussed the nature and consequences of
  employing a self- and peer assessment as a method
  to mark a group-based project.
• Contribution index showed a 0.7 mean increase.
• Although this is not a substantial increase the
  researchers feel it still fulfilled its aim of
  engaging the students to work effectively in
  groups.
• This makes life hard for free riders.
• Usually a contribution index of close to 1.0 will
  show these aims achieved.

30
Discussion

• A high agreement was noted between different
  students individual rankings, showing that the
  students found it possible to rate there fellow
  students.
• This then helps to ensure that free riders are
  penalised.
• Those who contributed were rewarded.
• Results obtained were different to that of
  previous research as in this case students rated
  personal contribution higher that what the peers
  did.

31
Discussion

• Possible explanation as to why discrepancies
  occurred between studies is due to the extra
  piece of written work issued to each student.
• It was thought that those who scored low on the
  assignment did not contribute much to the group
  work.
• This would then mean that those who contributed
  effectively to the group would be scoring double.

32
Discussion

• Self assessment very controversial.
• Researchers against self-assessment (Lejk
  Wyvill, 2001).
• Researchers pro self-assessment (Goldfinch,
  1994).
• A peer- and self-assessment was used as well as a
  sole self-assessment to test the measures
  validity.
• Self- and peer-assessments were found to have
  little relationship.
• Only peer-assessments correlated significantly
  with the marks awarded.
• Self-ratings worth is still unclear.
• It does promote self-reflection and the need for
  critical thinking.

33
Conclusion

• Peer-assessment promotes independent, reflective,
  and critical learners (Somervell, 1993).
• Helps to enhance motivation for participation
  amongst students (Michaelsen, 1992).
• Encourages students to take responsibility for
  our own learning (Rafiq Fullerton, 1996).
• Students get feedback on effectiveness within a
  group.
• Project supports the positive contribution of
  peer-assessment within group-based work.

34
Group Evaluation

• It is felt as a group that tutor evaluation is
  more important than peer evaluation.
• Peer evaluation still has its uses of engaging
  students to work more productively as a group.
• This is shown in this years discussion board
  where the majority of students agree that the
  tutor evaluation is more important.
• Possible reasons for this due to a lack of trust
  within the group as they were randomly selected.