Online Student Peer Reviews

1
Online Student Peer Reviews

• William J. Wolfe
• Professor of Computer Science
• California State University Channel Islands
• william.wolfe_at_csuci.edu

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Online Peer Reviews Outline

• Pros and cons
• Implementation Considerations
• Student Websites
• Course Website
• Peer Review Process
• Results

9
Peer Reviews Concerns

• Students dont know the subject.
• Students are not skilled evaluators.
• Students might send inappropriate messages.
• Students will not do that much work.
• Students will copy (cheat)!
• Keeping track of the reviews is very difficult.
• Student privacy.

10
Peer Reviews Advantages

• Students learn from each other.
• Students get lots of feedback.
• Students develop skills as evaluators.
• Students learn to appreciate evaluation criteria.
• Students see how they compare to their peers.
• Students see the class from teachers
  perspective.
• Students get to know one another.
• Teacher plays role of supervisor
• (A much better use of the teachers
  skills/knowledge).

11
Implementation Details

• What type of assignment?
• How many reviews does each student do?
• How many reviews does each student get?
• Who reviews whom?
• Does the reviewer have to be qualified?
• Will students grade fairly and accurately?
• Anonymous reviews?
• Grades based on peer reviews?
• Grade the peer reviews?
• Opportunity to revise based on reviews?
• Peer review of the peer reviews?

12
Peer Review System
13
Course Website

• Instructor sets up a course website
• Web pages
• Database
• Scripts
• Keeps track of all the activity/data

14
Peer Review Process

• Student
• Logs onto the course website
• Unique password for each student.
• Sees list of URLs
• List of links to the student web sites
• Picks one from the list
• Accesses a student web site
• Finds the assignment
• Reviews the assignment.
• Submits an anonymous review
• score (1 10)
• comment
• Goes back to the list of URLs and picks another.

15
System Flow
16
List of URLs
17
Student Website

• Students use their own website.
• Students must have basic web skills.
• Students must have access to a web service.
• Students cannot (easily) hide their identity.

18

  COMP 449   Human Computer Interaction

  John Doe

Weekly Assignments
COMP 449 Assignment 1
COMP 449 Assignment 2
COMP 449 Assignment 3
COMP 449 Assignment 4
COMP 449 Assignment 5
COMP 449 Assignment 6
COMP 449 Assignment 7
COMP 449 Assignment 8
COMP 449 Assignment 9
COMP 449 Assignment 10
COMP 449 Assignment 11
COMP 449 Assignment 12
COMP 449 Assignment 13
COMP 449 Assignment 14


19
Score Comment
20
Grading Criteria (Rubric)
21
Peer Reviews Received
22
Sample Peer Review
 Looks pretty good
23
Perfunctory Reviews

• perfunctory \pur-FUNGK-tuh-ree\ --adjective
  Done merely to carry out a duty performed
  mechanically done in a careless and superficial
  manner characterized by indifference

24
Detailed Peer Review
You should have requirements that detail the
concepts in section 4.2. Although you had some
very good points (i.e. the database should look
up student's degree requirements view should
list courses, etc...) almost all your
requirements can be more detailed. Go through
section 4.2 (each of the sections) and think of
what the program would need to do to effective
run. Some good examples of what requirements are
necessary are on others' websites, however I'll
give some to you now1.Is there a timeline
requirement?2.Is there a requirement on how
much(or how little) this will cost?3.Is there
security requirements?4.Is there user view
requirements?These(and many other questions) are
what you should answer in your requirements
definition document. Good luck on Assignment 3.
25
Lets try it out

• http//faculty.csuci.edu/william.wolfe/ucd/online/
  

26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Summary

• Peer Review Process
• Stimulated class activity.
• Some passionate participation.
• The audience effect brought up all
  performance levels.
• Very accurate evaluations (as a whole).
• Immediate access to examples of good and poor
  work.
• Addressed late and incomplete work.
• Requires web skills.

38
References
1. Online Student Peer Reviews, Proceedings of
ACM SIGITE Annual Conference, Salt Lake City
Utah, Oct. 28-30, 2004. 2. Student Peer Reviews
in an Upper-Division Mathematics Class, exchanges
THE ONLINE JOURNAL OF TEACHING AND LEARNING IN
THE CSU, (From the Classroom), September,
2003. 3. Course Web Site http//compsci.csuci.ed
u/wwolfe/ucd/online Password GUEST 4.
William.Wolfe_at_csuci.edu
39
Acknowledgements
Carol Holder (Director of Faculty Development
CSUCI) Paul Rivera (Economics, CSUCI) Harley
Baker (Psychology, CSUCI) Bob Bleicher
(Education, CSUCI) Ivona Grzegorzcyk
(Mathematics, CSUCI) Todd Gibson (Colorado
Institute of Technology) Michael Cook (Forstmann
Leff).
40
(No Transcript)
41
Peer Reviews How?

• Student Web Pages
• Students post homework solutions on their own
  web page.
• Course Web Site
• Set up course web site to manage all the peer
  review activity. Keep track of
• Links to student web pages,
• Peer Reviews
• Scores,
• Comments.
• Anonymous reviews.

42
The Course Web Site
43
Student Web Pages
44
Average Peer Review Score
45
Scoring Comparison
46
Number of Reviews
Software Engineering (CSC 4508) 34
students Theory 1 Assignment 1,122
reviews. 15 Assignments 16,830
reviews. Fact 1 Assignment 300 400
reviews. 15 Assignments 5,212 reviews.
47
Software Engineering (CSC4508) Fall 2002
48
Software Engineering (CSC 4508) Fall 2002
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
Software Engineering (CSC 4508) Fall 2002
54
(No Transcript)
55
Real Analysis (Math 351) Spring 2003