By Teaching We Learn: Teacher Education Assessment

1
By Teaching We LearnTeacher Education Assessment
Idaho State University College of Education
2
By Teaching We LearnTeacher Education Assessment
Standards-based assessment system
Idaho State University College of Education
3
By Teaching We LearnTeacher Education Assessment
Connecting candidate performance to PK-12 student
learning
Standards-based assessment system
Idaho State University College of Education
4
Our Approach to Using TWS Is . . .

• One element of our assessment system.
• Model for professional development.
• Sample of impact of teacher performance on
  student learning.
• Context-embedded.
• Collaborative.
• Emerging.

5

• Learner Studies
• Development and Individual Differences Field
  Experience
• Families, Communities, Culture Field Experience
• Pedagogical Studies
• Inquiring, Thinking, Knowing
• Motivation and Management
• Instructional Planning, Delivery, and Assessment
  Pre-Internship
• Instructional Technology
• Language and Literacy
• Integrative Studies
• Adaptations for Diversity
• Student Teaching Internship

Professional Education Core
Standards-based. Integration of course work and
clinical experiences. Taken by all candidates in
all programs.
6
All candidates in all programs complete two
teacher work samples . . .

• One TWS is completed as guided practice during
  the pre-internship.
• The second TWS is completed independently during
  the student teaching internship.
• Elements of the TWS are addressed during the
  course of the program.

7
Integration of TWS Elements . . .
Program Component
TWS Element
Development and Individual Differences Field
Experience
Contextual Factors
Families, Community, Culture Field Experience
Contextual Factors
Inquiring, Thinking, Knowing
Learning Goals Instructional Decision-Making Refle
ction
8
Integration of TWS Elements . . .
Program Component
TWS Element
Design for Instruction Instructional
Decision-Making
Motivation and Management
Instructional Planning, Delivery, and
Assessment Pre-Internship
All elements Guided Practice TWS
Instructional Technology Pre-Internship
All elements Guided Practice TWS
9
Integration of TWS Elements . . .
Program Component
TWS Element
Design for Instruction Instructional
Decision-Making
Motivation and Management
Instructional Planning, Delivery, and
Assessment Pre-Internship
All elements Guided Practice TWS
Instructional Technology Pre-Internship
All elements Guided Practice TWS
10
Idaho State UniversityCollege of Education
Accreditation Website Assessment System
Performance Reports
http//isu.edu
11
Establishing Credibility Evidence for Teacher
Work Samples

• Training for Scoring
• Benchmarking
• Reliability
• Validity
• Impacts on PK-12 Student
  Learning

12
Panel of Expert Raters
N 40 faculty/educational personnel from 10
states. Teacher Work Sample raters were from

• Missouri 3
• Oklahoma 2
• Pennsylvania 3
• Tennessee 5
• Virginia 7

• California 4
• Iowa 5
• Kansas 2
• Kentucky 6
• Michigan - 3

13
Panel of Expert Raters
N 40 faculty/educational personnel from 10
states. Teacher Work Sample raters were from

• Missouri 3
• Oklahoma 2
• Pennsylvania 3
• Tennessee 5
• Virginia 7

• California 4
• Iowa 5
• Kansas 2
• Kentucky 6
• Michigan - 3

14
Benchmarked Performances

• Proto-typical performances at each level of the
  developmental scoring rubric.
• 1 Beginning
• 2 Developing
• 3 Proficient
• 4 Expert
• Benchmarks are used for training of scorers and
  mentoring of candidates

15
TWS Holistic Score Distribution Fall 2002

• Beginning 18
• Developing 48
• Proficient 29
• Expert 5
• n 135

16
Inter-Rater Reliability

• Extent to which scoring decisions are consistent
• across expert judges.
• Dependability Coefficients
• (Shavelson Webb, 1991)
• 6 raters .91
• 3 raters .83
• 1 rater .63

17
Validity
Expert judgments of . . .

• Content Representativeness (Crocker, 1997)
• -- Realism
  -- Criticality
  -- Necessity
  -- Frequency
• Alignment with INTASC Standards
  

18
Validity Realism
98.1 of the raters strongly agreed with the
statement The performance tasks required by
the TWS represent actual lessons delivered to
students in PK-12 school classrooms.
19
Validity Criticality
97.5 of the expert judges rated the tasks
required by the TWS as critical or important
to teaching.
20
Validity Frequency
All of the tasks required by the TWS were rated
by the expert judges as high frequency activities
for teachers 90 of more of the raters said
weekly or daily for all of the tasks.
21
Validity Necessity
Overall, does the TWS assess knowledge and skills
that are necessary for beginning
teachers? Absolutely 70 Yes 30 No 0
22
Alignment with INTASC Principles
INTASC Principle
Directly
Implicitly
Not at All
Subject matter
5
45
50
Human development and learning
50
5
45
Adapting instruction
2
8
95
Instructional strategies
20
2
78
Classroom motivation and management
7
50
43
Communication skills
30
58
12
Instructional planning skills
2
3
95
0
7
93
Assessment of student learning
7
40
53
Professional commitment/responsibility
55
38
7
Partnerships
23
Impacts on Student Learning

• Quality of the sources of evidence (learning
  goals and assessments).
• Percent of students who met the learning goals.
• Percent of students who showed increased learning
  (improvement).

Evidence of student learning is situated within
the context of the quality of the learning goals
and assessments.
24
Impacts on Student Learning

• Quality of the sources of evidence (learning
  goals and assessments).
• Percent of students who met the learning goals.
• Percent of students who showed increased learning
  (improvement).

Evidence of student learning is situated within
the context of the quality of the learning goals
and assessments.