Two Levels of Course Assessment to Identify Student Learning

1
Two Levels of Course Assessment to Identify
Student Learning

• Carol Lerch, Ph.D
• lerch_at_dwc.edu
• S 52
• Friday 1145 1235

2
Background

• Assess new format College Algebra
• Introduced fall 2000
• Previous course failure rate 40
• Looking at students in one specific major

• Assess new format College Algebra
• Introduced fall 2000
• Previous course failure rate 40
• Looking at students in one specific major
• Course description
• Purpose of this course is to provide students
  with a proficiency in algebraic skills necessary
  for success in future coursework

• Assess new format College Algebra
• Introduced fall 2000
• Previous course failure rate 40

3
Collect Data

• Specifics Fall 2001 Summer 2007
• Pre-2000 60 pass (anecdotal)
• Fall average 85
• Spring average 64
• Summer 80
• Includes Withdraws

4
Analysis

• Improvement over previous College Algebra course
• Fall pass rates acceptable
• Continuing to improve

• Improvement over previous College Algebra course
• Fall pass rates acceptable
• Continuing to improve
• Spring pass rates much lower
• Needs analysis new plan

• Improvement over previous College Algebra course
• Fall pass rates acceptable
• Continuing to improve
• Spring pass rates much lower
• Needs analysis new plan
• Summer needs more data
• Online first time summer 07

• Improvement over previous College Algebra course

5
Changes to Course

• Added text related course management system
• online homework
• Interactive textbook
• Instant help
• Video, animation
• Added section JITTs
• Just-in-time-tests

• Added text related course management system
• online homework
• Interactive textbook
• Instant help
• Video, animation

6
Purpose of AssessmentPlan

• Pass rates
• Pass rates for particular course
• Course sequence
• Semesters to complete math sequence
• Thanks to Jack Rotman

• Pass rates
• Pass rates for particular course
• Course sequence
• Semesters to complete math sequence
• Thanks to Jack Rotman
• Concept assessment
• Learning goals/objectives

• Pass rates
• Pass rates for particular course

7
Semesters to Complete

• Semesters to pass through the math sequence,
  passing MA152 for each semester since fall 03
• 82 of those who started in MA098 passed MA152 in
  4 semesters (4 courses)
• 93 of those who started in MA122 passed MA152 in
  3 semesters (3 courses)

8
Levels of Assessment

• Gathering information about students achievement
  (Marzano, 2001, p. 115)

• Level I Assessment
• General Course information
• Pass rates
• Success in subsequent math courses
• Semesters to complete sequence

9
Levels of Assessment

• Level II Assessment
• Content specific
• Concepts

• Learning Map
• Summative assessment
• Overall course content
• Final exam

• Item Analysis
• Formative assessment
• Individual concepts tested

10
Cycle of Assessment
11
Plan

• Online homework required as of spring 2006
• Introduced as option for fall 2005
• Daily quiz
• 2-3 questions on concepts in each section
• Read text
• Come to class prepared
• Carrot replace test grade with quiz average

12
Collect Data

• Fall pass rate improved overall
• 90
• Spring stayed same
• 63

13
Analysis

• Fall pass rates acceptable
• Online homework helped
• Anecdotal evidence
• Students like all options available with online
  homework
• Have more opportunities to learn
• Spring rates need improvement

14
Plan
Plan

• Assesses student learning
• Concept assessment
• Learning goals/objectives
• Develop Learning Map
• Identify major concepts
• Identify teaching objectives
• Identify student learning objectives
• Means of assessment
• Set success standard

15
Learning Map
16
Learning Map

• 1st step
• Major topics of course
• Linear equations, quadratic equations,
  inequalities, absolute value, exponential log
  equations
• Modified as further developed
• 2nd step
• What do I want them to learn
• Teaching objectives

• 1st step
• Major topics of course
• Linear equations, quadratic equations,
  inequalities, absolute value, exponential log
  equations
• Modified as further developed

17
Learning Map

• Knowledge level
• Cognitive Process
• Action verbs

18
Levels of Knowledge

• Factual
• Basic elements of the discipline
• Facts specific to topic
• Parts of linear or quadratic equation
• Methods to solve quadratic equations

• Factual
• Basic elements of the discipline
• Facts specific to topic
• Parts of linear or quadratic equation
• Methods to solve quadratic equations
• Procedural
• Processes of mathematics
• Algorithms, techniques, methods
• Complete the square
• Quadratic formula

19
Levels of Knowledge

• Conceptual
• Interrelationships
• Classifications
• Principles, generalizations
• Pythagorean Theorem
• Properties

• Conceptual
• Interrelationships
• Classifications
• Principles, generalizations
• Pythagorean Theorem
• Properties
• Metacognitive
• Self knowledge
• Students need to reflect to learn

20
Teaching vs. Learning Objectives

• Student learning experience
• What they will do

• Teaching objective
• What I will do
• What I want the students to know

• Student learning objective
• Students will be able to

21
Assessment on Map

• Each objective has generic question
• Indicates how student is assessed

22
Learning Map

• Gave to students as part of syllabus
• Type of questions included
• Truth in advertizing
• Matched final exam to map
• Redesigned map
• Concepts tested but not part of map
• Redesigned final exam
• Some concepts not tested but important
• Some concepts tested multiple times

23
Level II Assessment

• Assess concepts learned
• Allow overall picture of student progress
• Shows strengths
• Identifies weaknesses
• Areas for further development

24
Collect Data

• Collected chapter test data
• Right/wrong only
• Collected final exam item results
• 4th semester of data collection
• Stats Percent correct

25
Collect Data

• Identified concept tested by semester
• Shows what is not tested (over-tested)

Results of chapter tests for Fall 06, Spring 07,
Summer 07
26
Analysis - Tests

• More Student Learning Objectives
• Needed to add very specific objectives
• Dropped some that I thought I wanted but dont
  test for

• More Student Learning Objectives
• Needed to add very specific objectives
• Dropped some that I thought I wanted but dont
  test for
• Success measure 65 or better correct

• More Student Learning Objectives
• Needed to add very specific objectives
• Dropped some that I thought I wanted but dont
  test for
• Success measure 65 or better correct
• Results below need attention

• More Student Learning Objectives
• Needed to add very specific objectives
• Dropped some that I thought I wanted but dont
  test for
• Success measure 65 or better correct
• Results below need attention
• 0 - 10 indicates problem with question as well
  as topic

27
Analysis Final exam

• Final used by all sections
• Data just my sections
• Shows problems with online course on specific
  concepts

28
Asides

• Only list what is being assessed
• If cant assess, dont put as objective

• Have a thick skin
• Data show changes made in course
• Provides longitudinal look at course
• Allows comparison of student results by semester
  or year
• Data show where more emphasis needed in teaching

29
Before Beginning Assessment

• Purpose of assessment
• What do you want to know?
• Why do assessment?
• Action leading from assessment
• What do you do with results?

30
5 Basic Questions

• Which objectives to assess?
• Student learning
• What should they know?
• How to assess?
• Multiple layers
• What are results?
• Collect/collate data
• What is success?
• What to do about results?
• Next steps

31
References

• Angelo, T. A. (1991). Introduction and overview
  From classroom assessment to classroom research.
  Classroom research Early lessons from success.
  T. A. Angelo and K. P. Cross. San Francisco,
  Jossey-Bass 7-15.
• Brockbank, A. and I. McGill (1998). Facilitating
  reflective learning in higher education.
  Philadelphia, Society for Research into Higher
  Education Open University Press.
• Carson, T. (1990). "What kind of knowing is
  critical action research?" Theory Into Practice
  XXIX(3) 167-173.
• Cochran-Smith, M. and S. L. Lytle (1993). Inside
  outside Teacher research and knowledge. New
  York, Teachers College.
• Elliott, J. (1991). Action research for
  educational change. Philadelphia, PA, Open
  University Press.
• Leibowitz, M. (1997). Instruction for process
  learning. Supporting the spirit of learning When
  process is content. A. Costa and R. M. Liebmann.
  Thousand Oaks, CA, Corwin Press, Inc. 47-54.
• Marzano, R. J. (2001). Designing a new taxonomy
  of educational objectives. Thousand Oaks, CA,
  Corwin Press.
• Mezirow, J. (1990). Fostering critical reflection
  in adulthood A guide to transformative and
  emancipatory learning. San Francisco,
  Jossey-Bass.
• Vygotsky, L. S. (1978). Mind in society The
  development of higher psychological processes.
  Cambridge, MA, Harvard University.

32
Participation

• Gather into groups
• Pick one course you teach
• List the major concepts
• Very few topics
• Pick one or two to use
• Identify Teaching Objective

• Identify Student Objective
• Identify knowledge levels
• Can be more then one
• Identify possible means of assessment
• Identify success measures

33
Share
34
Two Levels of Course Assessment to Identify
Student Learning

• Carol Lerch, Ph.D
• lerch_at_dwc.edu
• S 52
• Friday 1145 1235