Los Angeles Unified School District

1

Evaluation of ELD Practicum Year 1 Report 2006-07

• Los Angeles Unified School District
• Program Evaluation and Research Branch

Gojko Vuckovic, Ph.D., Kathy Hayes, Ph.D.,
Jesus Salazar, Ph.D.
January 9, 2009
2
Introduction

• This report examines implementation of the ELD
  program in 3rd and 4th grade classrooms looking
  at
• Regular Into English! lessons
• Enhanced Into English! Task Based Language
  Teaching (TBLT) lessons

3
Sample

• 25 randomly selected schools
• School Characteristic Index (SCI)
• 100 randomly selected classrooms
• Two Grade 3 classrooms per school
• Two Grade 4 classrooms per school

4
Data collection

• Two classroom observations, once per semester
• If ELD was taught, two additional consecutive
  days were observed
• Open-ended teacher interviews and administrator
  surveys

5
Data gathering strategies used in observations

• Fieldnotes
• 30-minute Timeline of Classroom Interaction
  Structures
• Observation Summary Statement
• Observation Summary Scale
• Interview Protocols
• Administrator Survey

6
Data reduction and analysis

• Application of a coding scheme to field notes and
  teacher interviews
• The coded data were analyzed for descriptive
  statistics, frequencies, and crosstabs

7
The evaluation objective 1

• To document
• The capacity of the schools to support ELD
  Program implementation in 3rd and 4th grade
  classrooms serving English learners.

8
The evaluation objective 2

• To document
• The extent and nature of ELD Program
• implementation in 3rd and 4th grade classrooms
  serving English learners.

9
The evaluation objective 3

• To document
• The effectiveness of ELD instruction as
  measured by student progress on CELDT and ELA CST.

10
Capacity Professional development

• 83 of teachers completed the ELD Practicum.
• Of remaining 17, 10 had ELD 5 day Institute
  completed, 3 were trained by EL Coordinator, and
  4 had no ELD training.
• Teachers found demonstration lessons to be the
  most beneficial aspect of the professional
  development.

11
Capacity ELD materials

• 82 of teachers had all of the ELD materials.
• 18 of teachers were missing some materials such
  as ballparking forms, Language Logs, lesson
  pictures, picture cards, CDs, and student books.
• 74 of teachers used supplemental materials, such
  as realia, visuals, literature, and the internet.

12
Implementation criteria

• Number of lessons taught
• Number of objectives met
• Materials used
• ELD strategies used
• Students on task (engagement)
• Time spent on ELD

13
Specific time ranges dedicated to ELD instruction
Average minutes per day Classrooms
30 to 45 60
More than 45 19
Less than 30 16
No ELD taught 5

14
Time by program and level
ELD Lesson Type Time in minutes Time in minutes
ELD Lesson Type Implemented Not Implemented
Enhanced 39 23
Regular 35 33
Regular and Enhanced 37 n.a.
Special Cases 44 38
Average 39 26
15
3rd Grade implementation
Semester Implemented Not Implemented Totals
First 43 7 50
Second 37 13 50
16
4th Grade implementation
Semester Implemented Not Implemented Totals
First 37 13 50
Second 40 10 50
17
General factors contributing to no or very low
implementation Second semester (23 classes)
School site issue 17
Substitute teacher (2 also school site issue) 6
Professional development (2 also school site issue) 4
18
Specific factors contributing to no or very low
implementation Second semester
No ELD instruction of any kind was observed 2
No ELD Program materials were used 4
Teacher cited timing issues 4
Only one lesson taught over 3 days 5
Grade level or student population issues 2
Other 6
19
ELD implementation by ELD program observed -
Semester 1
ELD Program Implemented Implemented Not Implemented Not Implemented
ELD Program N N
Enhanced 37 86 6 14
Regular 21 81 5 19
Enhanced Regular 9 82 2 18
Special Cases 13 81 3 19
No ELD Program 0 0 4 100
Total 80 80 20 20
20
ELD implementation by ELD program observed -
Semester 2
ELD Program Implemented Implemented Not Implemented Not Implemented
ELD Program N N
Enhanced 48 94 3 6
Regular 8 57 6 43
Enhanced Regular 11 100 0 0
Special Cases 10 53 9 47
No ELD Program 0 0 5 100
Total 77 77 23 23
21
Definitions of Special cases

• Implemented
• Regular lessons taught with TBLT framework
• Not Implemented
• Regular lessons taught with TBLT framework
• Made up lessons taught with TBLT framework
• Teacher self-created lesson with no connection to
  the IE! books

22
Use of strategies Enhanced Into English!
ELD Strategies Semester 1 (43 classes) Semester 1 (43 classes) Semester 2 (51 classes) Semester 2 (51 classes)
ELD Strategies N N N N
Backward Buildup 0 0 1 2
Ballparking 12 28 8 16
Corrective Recasting 19 44 10 20
Pull Out and Talk/Write 38 88 45 88
RASP 6 14 2 4
Think-Pair-Share 41 95 48 94
Thinking Maps 42 98 50 98
Vocalized Reading 33 77 35 69
23
Use of strategies Regular Into English!
ELD Strategies Semester 1 (26 classes) Semester 1 (26 classes) Semester 2 (14 classes) Semester 2 (14 classes)
ELD Strategies N N
Audio 14 54 7 50
Chant 11 42 4 29
Choral Reading 15 58 6 43
Collaborative Reading 1 4 1 7
Demonstrating 14 54 12 86
Echo Reading 4 15 1 7
Games 2 8 1 7
Graphs/Charts 16 62 8 57
Prior Knowledge 11 42 7 50
Realia 9 35 2 14
Think-Pair-Share 17 65 7 50
TPR 6 23 2 14
Visual/Illustrations 23 88 13 93
24
Effectiveness of ELD instructionCELDT
Grade Implemented Implemented Not Implemented Not Implemented
Grade Gain N Gain N
3rd grade .03 361 -.19 112
4th grade .25 483 -.02 124
25
Effectiveness of ELD instructionELA CST
Grade Implemented Implemented Not Implemented Not Implemented
Grade Gain N Gain N
3rd grade .00 379 -.32 118
4th grade -.06 505 -.35 118
26
Effect sizes
Grade ELA CST Impl. vs Not Impl. CELDT Impl. vs Not Impl.
3rd grade .32 .22
4th grade .29 .27
27
10 classes in which ELD levels 4 5
significantly outperformed lover ELD levels
Differentiation Classroom Classroom Classroom Classroom Classroom Classroom Classroom Classroom Classroom Classroom
Differentiation 1 2 3 4 5 6 7 8 9 10
Suggested number 8 1 6 7 6 3 4 6 6 4
Completed number 3 1 1 0 0 0 1 0 0 1
28
10 classes - Differentiation practices not or
rarely used

• Questioning techniques based on ELD level
• Mapping
• Discussion prompts
• Activity options by level
• Different grouping for reading
• Generating different student responses by
  facilitating simple, elaborate and complex
  sentences (some observed)

29
10 classes Quality of teaching and learning
analysis

• Learning environment typically not student
  centered
• No extra oral opportunities when needed
• Vocabulary needed for better understanding not
  covered
• No assessment practices
• Rarely accessed prior knowledge
• No evidence of listening to students to inform
  teaching

30
15 high performing classrooms
Classroom and implementation level (I, NI) Grade ELD Levels Num. of ELs CST ELA Gains CELDT Gains
Classroom 1(I) 3 1,2,3,4,5 17 .66 .31
Classroom 2(I) 3 1,2,3,4,5 11 .37 .44
Classroom 3(I) 4 1,2,3,4,5 17 .38 .76
Classroom 4(I) 4 1,2,3,4,5 23 .56 .09
Classroom 5(I) 3 3,4,5 6 .42 .25
Classroom 6(I) 3 1,2,3,4,5 12 .65 .23
Classroom 7(I) 3 3,4,5 6 .50 .70
Classroom 8(I) 4 3,4,5 12 .61 .17
Classroom 9(I) 4 1,2,3,4,5 13 .60 .28
Classroom 10(NI) 4 1,2,3,4,5 25 .30 .41
Classroom 11(NI) 3 1,2,3 11 .53 .33
Classroom 12(I) 3 4,5 5 1.40 .26
Classroom 13(I) 4 4,5 16 .44 .71
Classroom 14(I) 4 4,5 13 .85 .27
Classroom 15(I) 3 4,5 4 .48 .39

31
15 classes - Quality of teaching and learning
analysis

• Lessons student centered
• Topics relevant to the students lives, needs and
  interests
• Pacing plan adapted to meet the needs of the
  students
• Extra oral opportunities provided
• Vocabulary in context taught
• Prior knowledge accessed
• Evidence of listening to the students

32
Recommendations

• Implementation of the ELD program had a positive
  effect on student performance
• LD superintendants and directors of school
  services should plan for the professional
  development of school administrators and monitor
  the capacity of schools to implement and support
  the ELD program.

33
Recommendations (continued)

• Principals and assistant principals in
  partnership with school site leadership bodies
  should develop detailed plans for the
  implementation of the ELD program, provide
  necessary support, and closely monitor program
  implementation in each classroom with English
  learners.
• They must pay regular visits to the classrooms,
  observe instructional practices, and provide
  directions.

34
Recommendations (continued)

• Principals and assistant principals should take
  responsibility for their own personal development
  and training in ELD programs and practices, as
  well as the PD of school staff and teachers.
• The Language Acquisition Branch should find ways
  to strengthen the professional development
  design, curriculum, and the way the program is
  implemented in order to address the needs of
  students at lower ELD levels.

35
Recommendations (continued)

• The Local Districts in partnership with the
  Language Acquisition Branch should continue
  providing PD opportunities for teachers and
  administrators.
• The Language Acquisition Branch should provide
  additional training in core differentiation
  practices.
• Principals, local district superintendents and
  directors of school services should take a lead
  in creating conditions to ensure that there are
  no more then two consecutive ELD levels per class.

36
Recommendations (continued)

• Additional steps should be made by the Division
  of Professional Development and Leadership and
  the Language Acquisition Branch to incorporate
  high quality teaching strategies across all core
  content professional developments in our
  District.

37

• www.lausd.net/lausd/offices/perb