Julia Glencer - PowerPoint PPT Presentation

Title: Julia Glencer


1
The Fruits of Hope Student Evaluations

• Julia Glencer
• Erin Karsman
• Jan Levine
• Tara Willke
• Special thanks to Research Assistant
• Katie ChengerY

2
Hope and Student Evaluations

• Martin Rand article
• Jan Levines comments (pp. 37-38, fns. 150-154
  deleted)
• Hopeful teaching is a give-and-take process
  between teachers and students.
• If the curriculum employed in the writing
  program engenders hope among the students, the
  program is viewed more positively by students.
• Conversely, if teachers and the program kill
  hope in their students, the students evaluations
  of the faculty, and the students work product,
  are weaker, leading in turn to faculty cynicism,
  disaffection, and disinterest such a downward
  spiral for all participants is a tragedy.

3
Scholarship on Student Evaluations

• Richard Abel, Evaluating Evaluations How Should
  Law Schools Judge Teaching?, 40 J. Leg. Educ. 407
  (1990).
• Arthur Best, Student Evaluations of Law Teaching
  Work Well Strongly Agree, Agree, Neutral,
  Disagree, Strongly Disagree, 38 Sw. U. L. Rev. 1
  (2008).
• Judith D. Fischer, The Use and Effects of Student
  Ratings in Legal Writing Courses A Plea for
  Holistic Evaluation of Teaching, 10 Leg. Writing
  111 (2004).
• Melissa Marlow-Shafer, Student Evaluation of
  Teacher Performance and the Legal Writing
  Pathology Diagnosis Confirmed, 5 N.Y.C. L. Rev.
  115 (2002).
• Deborah J. Merritt, Bias, The Brain, and Student
  Evaluations of Teaching, 82 St. Johns L. Rev.
  235 (2008).
• David D. Walter, Student Evaluations A Tool for
  Advancing Law Teacher Professionalism and Respect
  for Students, 6 Leg. Writing 177 (2000).

4
Legal Writing Pathology?

• Mention the topic of student evaluations during
  the coffee break at a national or regional legal
  writing conference and you will likely hear
  something along these lines Students dont like
  writing and they take it out on us in
  evaluations or students rate legal writing
  lower than doctrinal courses at our school.
• Marlow-Shafer, 5 N.Y.C. L. Rev. at 115.

5
Top Five Evaluative Hostility Factors

• Hiding the Ball
• Critiquing
• Grading
• Lack of Respect
• Evaluation Form Not LRW Program-Specific

6
Factor 1 Hiding the Ball

• Frequent comments from legal writing students
• about clarity . . . pinpoint this topic as a
  pervasive
• problem area.
• Professor hides the ball.
• Questions were answered too evasively.
• It seemed unclear what was an A paper. . . .
• The explanation . . . for the assignments were
  very poor.
• Walter, 6 Leg. Writing at 178.

7
Duquesnes Curricular Response

• Factor 1 Hiding the Ball

• Response Bounce the Ball Off Their Heads!

• Hidden expectations.

• Assignment-centered curriculum (not textbook).
• Integrated research, writing, and analysis.
• Show samples of the actual document in class.

8
Factor 2 Critiquing

• Students often pinpoint clarity in the critiques
  as a problem area
• Feedback was depressing, knowing that no matter
  how good you did, it was going to get butchered.
  It did show my mistakes though.
• Walter, 6 Leg. Writing at 178 (emphasis
  added).
• The comments were hard to decipher as to how to
  make improvements.
• Id. at 193 (emphasis added).
• We can also demonstrate empathy by taking great
  care not to destroy the students self esteem,
  either through direct comments or through
  cynicism in general. Legal writing teachers give
  considerable negative, but constructive,
  criticism. We must include positive comments,
  too.
• Id. at 215 (emphasis added).

9
Duquesnes Curricular Response

• Factor 2 Critiquing

• Response Coaching

• Lack of effective guidance.
• Too harsh.
• Too few.
• Lack of respect.
• Lack of hope and belief in the students.

• Act as coaches.
• Small class sizes and personalized feedback.
• Curriculum utilizes draft-critique-conference-revi
  sion.
• Electronic comments.
• Provide document 24 hours in advance of
  conference. Meet with students in conferences as
  frequently as possible.
• Hold high expectations for students.

10
Factor 3 Grading

• In terms of timing and its impact on legal
  writing faculty, Barbara Fines states that
  research confirms the suspicion of legal writing
  instructors that their early and frequent
  evaluation of students impacts their teaching
  evaluations more directly and negatively in
  comparison to their colleagues who do not
  distribute grades until after student evaluations
  are completed.
• Marlow-Shafer, 5 N.Y.C. L. Rev. at 124.
• Students who have received fairly low grades
  on earlier assignments, grades below what they
  were used to getting in undergraduate school, and
  often, in their minds, disproportionately low
  compared to the amount of work they perceive they
  did in preparing the documents may tend to resent
  the course or the teacher or both. . . . The
  teachers of courses that offer grades during the
  semesterbefore students complete their
  evaluation formsare at risk of lower evaluations
  from students disappointed with their grades.
• Walter, 6 Leg. Writing at 188-89 (internal
  quotations omitted).

11
Duquesnes Curricular Response

• Factor 3 Grading

• Response Delayed Grading

• Receive grades on assignments early in the
  semester, often lower than what the student
  received in college.
• Programs often utilize blind-grading.
• Lack of program-wide uniformity.

• Grade only the final assignment in each semester.
• Only the final grade (fall spring average) is
  reported on transcripts.
• Students know what is expected of them.
• Conform to programmatic normative grade
  distribution and compare examples of student work
  product at various grade points.

12
Factor 4 Lack of Respect

• The issue of status in the legal academy may
  play a role in lower student evaluations for
  writing faculty.
• Marlow-Shafer, 5 N.Y.C. L. Rev. at 132.
• The behaviors that most influence . . .
  evaluations are rooted in physiology, culture,
  personality, and habit. Those behaviors are
  difficult for any faculty member to alter and
  they often reflect characteristics like race,
  gender, nationality, or socioeconomic class.
• Merritt, 82 St. Johns L. Rev. at 254.

13
Duquesnes Curricular Response

• Factor 4 Lack of Respect

• Response Respect

• Importance of skills to practicing law.
• Enthusiasm.
• Small class size.
• Orientation Week.
• We treat them with respect.

• Externalities
• Status
• Race
• Gender

14
Factor 5 Not Program-Specific Evaluation Form

• In the context of controversy about student
  evaluations, it is somewhat surprising that many
  schools fail to use the process to accomplish
  what would likely be the most readily accepted
  function, the collection of observations that do
  not involve judgment but might provide worthwhile
  information about basic aspects of teaching such
  as being punctual, providing a syllabus, or
  offering clear statements of student
  obligations.
• Best, 38 Sw. U. L. Rev. at 30.
• Scholars also agree that teachers who receive
  midterm comments from students tend to receive
  better course-end ratings. An additional
  suggestion by several researchers is to use only
  questions aimed at specific behaviors, like
  whether the professor arrives on time. . . .
  Notably, 77 of the respondents to this study who
  said student ratings helped them improve their
  teaching identified feedback about specific
  behaviors, rather than general comments, as
  helpful.
• Fischer, 10 Leg. Writing at 157-158.

15
Duquesnes Curricular Response

• Factor 5 Not Program-Specific Evaluation Form

• Response Well-designed Evaluation Form

• LRW-specific.
• Reflect shared ethos and what we think is
  important.
• No focus on classroom performance.
• Holistic nature of the course.
• Are both formative and summative, and they make
  clear the shared expectations and goals for all
  involved most questions are about specific
  behaviors of the teacher.

• Not LRW-Specific.
• Evaluate classroom performance.
• Passive-learning. Overuse of quantitative
  scoring.

16
Duquesnes Curricular Response

• Factor 5 Not Program-Specific Evaluation Form

• Response Well-designed Evaluation Form

• Distributed twice per year.
• Are in addition to, and do not substitute for,
  the required course evaluations that are
  distributed by the University at the end of the
  year.
• The spring semester year-ending evaluation is
  timed to follow the peak experience of the oral
  argument before alumni judges, revealing to
  students how far theyve come and how much
  theyve accomplished.

• Not LRW-Specific.
• Evaluate classroom performance.
• Passive-learning. Overuse of quantitative
  scoring.

17
The Fruits of Hope Our Results

• Cumulative Course Evaluation tallies and
  percentages for all 12 sections/professors

18

Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My professor had high standards for my work and the course was demanding. 161/203 32/203 10/203
My professor had high standards for my work and the course was demanding. 79 16 5
19
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My professor made clear what was expected of me on the appellate brief assignment. 140/202 45/202 13/202 3/202 1/202
My professor made clear what was expected of me on the appellate brief assignment. 69 22 6 1 0
20
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My professors comments on my drafts were clear and understandable. 126/192 49/192 11/192 4/192 2/192
My professors comments on my drafts were clear and understandable. 66 26 6 2 1
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My professor provided sufficiently detailed written feedback on my drafts. 134/192 43/192 12/192 2/192 1/192
My professor provided sufficiently detailed written feedback on my drafts. 70 22 6 1 1
21
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My professors written critique of my draft was provided to me no later than the day before our scheduled conference. 132/192 18/192 7/192 3/192 4/192
My professors written critique of my draft was provided to me no later than the day before our scheduled conference. 83 9 4 2 2
22
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
My scheduled conference with my professor on the draft of the argument helped me improve my brief. 154/192 25/192 13/192
My scheduled conference with my professor on the draft of the argument helped me improve my brief. 80 13 7
23
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
The year of LRW helped me improve my writing skills. 159/200 28/200 10/200 2/200 1/200
The year of LRW helped me improve my writing skills. 80 14 5 1 1
24
Cumulative Course Evaluation tallies and
percentages for all 12 sections/professors
Criterion Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree Rankings 5 is Strongly Agree and 1 is Strongly Disagree
Criterion -5- -4- -3- -2- -1-
The year of LRW helped me improve my legal research skills. 141/200 42/200 14/200 2/200 1/200
The year of LRW helped me improve my legal research skills. 70 21 7 1 1