Assessment Learning

1
Assessment Learning

• What evidence will you and your peers accept?
• Diane Ebert-May, Ph.D.
• Lyman Briggs School
• Botany and Plant Pathology
• Michigan State University

2
toward
toward
necessitates
necessitates
by
by
by
collaborate with
supported by
develop
achieve
drives
e.g.
e.g.
e.g.
e.g.
Reasoning
3
Cooperative Groups

• 4 students per group
• Person A, B, C, D in each group
• First - read question/think about answer
  individually
• Discuss A with B
• C with D
• Form group consensus

4
Q1 - Engagement

• Imagine I invited you to visit and review my
  course Introductory Biology to help me decide if
  I was practicing active learning. What
  criteria and dimensions of teaching would you
  look for in my class?
• Oral report from groups

5
Biological Literacy for All

• Utilize process of scientific inquiry to think
  creatively and formulate questions about
  real-world problems
• Effectively communicate an understanding of and
  links among biological principles and concepts to
  peers and others
• Gain confidence in ability to write about,
  criticize and analyze concepts in biology

6
Biological Literacy for All (2)

• Reason logically and critically to evaluate
  information
• Develop positive attitudes about the relevance of
  biology to their lives
• Demonstrate positive interdependence and
  individual accountability within cooperative
  groups

7
NSF- Undergraduate Course and Curriculum
Development Grant

• Is it possible to implement strategies for
  active, inquiry-based learning and cooperative
  group interaction in large courses?
• Do these strategies promote more effective
  learning by more students?
• 600 -700 students in class meetings
• 450 students in laboratory

8
Compared Two Approaches to Teaching

• Traditional
• Experimental

9
Who Asks Questions

• Traditional Lab manual/faculty
• Experimental Students

10
Inquiry Lab/Class Meeting

• Traditional Confirmatory
• Experimental Inquiry

11
Functional Cooperative Groups

• Traditional Not Often
• Experimental Often

12
Writing/Reading to Learn

• Traditional Not Often
• Experimental Often

13
TA Professional Development

• Traditional Superficial
• Experimental In-depth, on-going

14
Assessment

• Traditional Tests
• Experimental Multiple Forms
• Aligned with student outcomes

15
Learning Cycle Model in Class Meetings

• Cooperative Groups 4 Students
• Engage 5 - 10 minutes
• Explore 10 - 15 minutes
• Explain 10 - 20 minutes
• Elaborate 10 - 15 minutes
• Evaluate 10 minutes

16
Experimental Design

• Covariates
• Pre-Test / Factor Scores
• Control Lectures Experimental / Control
  Lectures Experimental Lecture / LabControl /
  Experimental Labs Experimental Labs
• Fall 1994 Spring 1995 Fall 1995
• Self-Efficacy Self-Efficacy Self-Efficacy
• NABT NABT NABT
• Process Skills I, II Process Skills I, II
  Process Skills I, II
• TOSRA TOSRA TOSRA
• Independent Variables
• Intervention --- TA --- Gender --- Ethnicity ---
  Age
• (Lab, Lecture)

17
Q2 - Exploration

• What evidence will you (and your peers) accept
  that indicates your students have learned and
  achieved the outcomes of your course?
• Written group report

18
Self-Efficacy Subscales

• Fac 1 Writing and critiquing analytical skills
  (??.92)
• Fac 2 Generalizability to other science courses
  (??.87)
• Fac 3 Biological literacy (??.88)

19
Self-Efficacy, Fall 94
4.4
4.2
4.0
Factor Scores
3.8
PosFac 1
3.6
PosFac 2
3.4
PosFac 3
EXP
CON
Experimental Condition
p lt .01 (n 338)
20
NABT Process, Fall 1994
14.2
14.0
13.8
13.6
13.4
13.2
13.0
Experimental
Control
Lecture
p lt .01 (n 356)
21
NABT Content

• No difference in scores between traditional and
  control groups

22
Model with 700 students

• Two Sections _at_ 350 students each
• Taught back to back on schedule
• T, Th - 75 minute periods
• Team taught - 2 faculty as team partners in both
  sections

23
Number of Absences
24
(No Transcript)
25
Assessment and Curriculum Design

• Two sides of the Same Coin

26
Design Course Goals

• Course - goals
• Modules - objectives
• Activities - objectives

27
Develop Performance Expectations

• Descriptions of Student Performance
• Providing evidence convincing to you, your
  colleagues, and significant others that students
  have achieved a course goal or module objective
• Descriptions of the conditions under which the
  evidence is collected

28
Assessment

• Data collection with a purpose

29
What data are collected?

• Measures of students knowledge and abilities
• Answers to items on short answer tests
• Essays
• Position Papers
• Oral Presentations
• Poster Presentations

30
Assessments Linked to Goals

• Goals articulated with outcomes that are
  measurable or observable (actions)

31
Functions of Assessments (Hodson 1992)

• summative - description of students level of
  attainment
• formative - diagnostic feedback to students and
  instructor
• evaluative - provide instructors feedback about
  the effectiveness of the curriculum experiences
• educative - engage students in interesting,
  challenging, significant experiences to develop
  further insight and understanding

32
Development of Rubric

• Assessment tasks linked to outcomes
• Define performance standard for biologically
  literate student.

33
Categories

• Name of category
• Criteria within categories -- discriminate among
  qualities you determine important
• Reach consensus - students involved

34
(No Transcript)
35
Engagement problem

• E.O. Wilson states that insects and other
  land-dwelling arthropods are so important that if
  all were to disappear, humanity probably could
  not last more than a few months. Based upon what
  you know today, do you believe that this
  statement is true? Write an essay and explain
  your reasoning as logically and thoroughly as you
  can. Do not use any outside resources for your
  essay.

36
What is the evidence we accept that students have
learned?

• What do we accept as a reasoned argument?
• What are the criteria we use for evaluation of
  the argument?

37
Argument Structure
38
Why use argument structure in science?

• Consistent with nature of science
• Justifies conclusions to the learning community
• Develops problem solving and critical thinking
  skills
• Develops art of questioning

39
What about content?

• Argument and inquiry drive understanding of
  content
• Content drives argument and inquiry

40
What did we find?

• Students seldom disagreed with the statement
• Some misconceptions about content appeared
• Students rarely used warrants to support evidence

41
What did we do?
42
Revised Question

• The keynote speaker at a scientific meeting made
  the statement that insects and other
  land-dwelling arthropods are so important that if
  all were to disappear, humanity probably could
  not last more that a few months. This generated
  considerable discussion, both pro and con at the
  meeting. What do you think about this statement?
  Construct an argument to explain your views and
  to illustrate your reasoning.

43
Question

• When we introduce innovation into our classes,
  how are student evaluations affected?

44
Self-Evaluation

• Students who internalize valued achievement
  targets so thoroughly as to be able to
  confidently and completely evaluate their own and
  each others work, almost automatically become
  better performers in their own right.
• McMillan and Forsyth (1991)

45
(No Transcript)
46
Goal 2 - Effectively describe biological concepts
to peers

• So much group work made me realize that
  understanding a concept and being able to
  communicate a concept are different things.
• ...I find myself explaining things I was unable
  to explain before.

47
(No Transcript)
48
Goal 3 - Confidence in ability to write about,
criticize, analyze concepts in biology

• I always was good at biology (or at least got
  good grades in biology), but never really
  understood any CONNECTIONS. This class made it
  clear to me that everything is connected to each
  other. We were forced to write down how we
  UNDERSTOOD concepts, not simply to memorize parts
  and functions.

49
(No Transcript)
50
4 - Use process of scientific inquiry to think
creatively and formulate questions...

• ...we did our position papers on two real world
  issues that we deal with every day. One paper we
  did was on the environment and the other was on
  breast cancer...This class really makes you think
  about how important scientists are in the world
  today. They try to formulated and answer
  questions that will help us survive in the
  future...

51
(No Transcript)
52
7 - Positive interdependence individual
accountability within cooperative group

• Working in groups requires so much
  responsibility. This means coming to class every
  day, having your input for all of the questions
  and quizzes and most of all showing up on your
  own time to complete homework assignments,
  etc.... Working as a team requires dedication and
  cooperation This is how everyone will succeed.

53
Change in Self-Efficacy
Percent
Factor 1
Factor 2
Factor 3
54
So what?

• Pre-service teacher education
• Scientifically literate population
• Research in the disciplines about learning
• Recognizing, Evaluating, and Rewarding Excellent
  Undergraduate Teaching - NRC Study Committee on
  Higher Education