Explaining the Superintendent Summit Data

1
Explaining theSuperintendent SummitData

• Superintendents Summit
• September, 2006
• Texas Institute for Measurement, Evaluation, and
  Statistics

2
Outline

• Data provided at the Summit
• Reading and interpreting the graphs and tables
• Navigating the CD (Instructions in Superintendent
  folder)

3
Data provided at the Summit

• Outcome data (G1-G3 SAT10/Aprenda
  ITBS/Logramos, English/Spanish TAKS)
• TPRI Tejas LEE data (K-3)
• All data is presented at three levels
• Campus level
• District level (Roll-up of all RF campuses in the
  district)
• State Level (Roll-up of all RF campuses in the
  state)

4
Reading and interpretingthe graphs and tables
5

• OUTCOME DATA
• Outcome data is presented in two ways
• Graphs presenting data across years separately by
  language for all students (OUTCOME ACROSS YEARS).
• Tables presenting data across years separately by
  language for all students, as well as by
  demographic categories (OUTCOME BY DEMOGRAPHICS).

6
Type of Data Presented
Assessments
7
Percent students proficient
8
Sample Sizes
Year Grade
9
Additional graphical aspects of the OUTCOME
ACROSS YEARS graphs that will appear only on
campus and district level graphs.
10
Sample size 0 No data for that time point
Blank
11
State Reference
All District level graphs of Outcome data across
years will include a RF State level reference for
comparison.
12
District Reference
All Campus level graphs of Outcome data across
years will include a RF District level
reference for comparison.
13

• Things to keep in mind when reading OUTCOME
  graphs and tables
• Data is presented separately for English and
  Spanish assessments.
• Data presented at the district or state level in
  grades 1 and 2 within language is aggregated
  across different assessments (SAT10/ITBS or
  Aprenda/Logramos).
• TAKS data for 2005-2006 includes Administrations
  1 through 3. Use care when comparing to other
  reports of data to ensure same administrations
  are being compared, etc.

14

• Things to keep in mind when reading OUTCOME
  graphs and tables
• Campuses/Districts will vary in the number of
  years of data presented.
• Cycle 1 2003-2004, 2004-2005, 2005-2006
• Cycle 2 2004-2005, 2005-2006
• Cycle 3 2005-2006
• Campuses/Districts data will vary based on the
  grades served and the language of testing
  completed.
• Cycle is used by TIMES to denote campuses
  entering RF in 2005-2006.

15

• Things to keep in mind when reading OUTCOME
  graphs and tables
• DISTRICT and STATE level graphs are for RF
  campuses only - not the district or state as a
  whole.
• All graphs are based on data submitted to TIMES.
  There are instances where data has not been
  submitted from a campus/district. Graphs and
  tables are only based on data received.

16

• Important Information about Proficiency
• Grades 1 and 2
• Proficiency is the percent of students who are
  at or above the 40th percentile as indicated by
  scale score to percentile score conversion
  guidelines for Spring norms.

17

• Important Information about Proficiency
• Grades 1 and 2
• Districts test at very different times (mid- Feb
  late April). Districts also vary in the norms
  they use (mid-year and spring).
• In order to make all scores comparable across
  districts, proficiency was defined as a score at
  or above the scale score equivalent to the 40th
  percentile using SPRING norms.

18

• Important Information about Proficiency
• Grade 3
• Proficiency is the percent of students who are
  at or above a scaled score of 2100 on TAKS.
• This cut point changed between 2003-2004 and
  2004-2005. To make data comparable across
  years, the 2004-2005 cut point was back applied
  to 2003-2004 data.

19
Comparing performance using theOUTCOME ACROSS
YEARS graphs.
20
In Grades 1 and 2, examine the performance level
of each Grade and Year in relation to National
norms. If 60 or more of the students are
performing above the 40th percentile, then the
student population is performing at or above the
national norm.
21
Spanish Grade 1 and 2 scores are
significantly higher. Norming and item
development sample. Most districts and state at
or above 80
22
Examine the performance trends over time.
Increasing or decreasing? This is a campus level
example. Important to keep in mind that each year
is a different group of students. Differences
between cohort composition will alter patterns.
Inclusion of new campuses or changes in sample
size may also alter patterns.
23
Sample changes, but not likely the reason unless
the new students were known to be much lower
performers. Did the campus transition students
earlier in 05-06 than in 04-05 and as a results
test more Spanish students in English?
24
Examine the performance level of each Grade and
Year in relation to the District RF Performance
Level (for campuses) or State RF Level (for
Districts). This is an example of a campus with
district references.
25
OUTCOME BY DEMOGRAPHICS
26
Type of Data Presented
27
What level and specific District or Campus
data is being presented for
28
Language Grade Level Year
29
Demographic Categories
30
n Sample Size
31
Percent in Group that are Proficient
32
Overall Outcome Across Years Graphs
33
Of the 28,151 students with first grade English
data in 2003-2004, 3,614 were designated as
African-American. Of the 3,614 designated as
African American, 48 were at or above
proficiency.
34
(No Transcript)
35
(No Transcript)
36

• TPRI/Tejas LEE DATA (K-3)
• TPRI/Tejas LEE data is presented in three ways
• End of Year (EOY) performance across years by
  DOMAIN (TPRI TEJAS EOY Across Years/Domain).
• 2005-2006 data only
• Beginning, Middle and End of Year (BOY, MOY
  EOY) performance in 2005-2006 by DOMAIN (TPRI
  TEJAS Year 3 2005-2006/Domain).
• Beginning, Middle and End of Year (BOY, MOY
  EOY) performance in 2005-2006 by TASK (TPRI TEJAS
  Year 3 2005-2006/TASK).

37
END OF YEAR (EOY)ACROSS YEARSBY DOMAIN
38
Type of Data Presented
Test Grade
Year Domain
39
Percent students Still Developing
40
Two Sample Sizes
TPRI

• Two sample sizes are presented for all TPRI
  graphs
• Total students administered Screen and Inventory
  Sections (may or may not have been administered
  PA and GK sections).
• This is used as the denominator for the CO
  Domain.
• Total students administered PA and GK Sections
  This is used as the denominator for the PA and
  GK Domains. (PA GK - K and 1 GK Only - G2
  G3)
• Subtract PA/GK from SC/CO Jump Screen to
  Comprehension.

41
One sample
Tejas
Because there is no screen on the Tejas LEE, only
one sample size is presented. These are used as
the denominators for all Domains.
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
EnglishPerformance comparable from end of
2004-2005 to end of 2005-2006 End of 2005-2006
Grade Domain with most needKindergarten PA
(56) and CO (41)First Grade PA (49) and GK
(26)Second Grade GK (72) and CO (24)Third
Grade GK (81) and CO (31)
51
SpanishPerformance comparable from end of
2004-2005 to end of 2005-2006End of
2005-2006Grade Domain with most
needKindergarten GK (32) and CO (34)First
Grade GK (26) and CO (26) Second Grade
GK (35) and CO (23)Third Grade GK (72)
and CO (47)
52
Additional graphical aspects of the TPRI/TJL
DOMAIN ACROSS YEARS graphs that will appear only
on campus and district level graphs.
53
Sample size is not equal to 0
No students Still Developing or Nivel de
Intervención
54
State Reference
All district level graphs of the TPRI/Tejas
Domain data across years will include a state
level reference for comparison.
55
District Reference
All Campus level graphs of the TPRI/Tejas Domain
data across years will include a District level
reference for comparison.
56
2005-2006 Year 3BOY, MOY, EOYBY
DOMAIN(Follow the same format as theEOY Across
Year by Domain Graphs)
57
Samples by Time Point
2005-2006 Time Point Domain
58
(No Transcript)
59
2005-2006 Year 3BOY, MOY, EOYBY TASK
60
Title and sample information follow the same
pattern as the Domain graphs.
61
Data Presented by Time Point B BOY, M MOY and
E EOY
62
Data is presented for each section of the
assessment separately
63
(No Transcript)
64
(No Transcript)
65
An asterisk in ANY column indicates that there
is no performance expectation for that task at
that time point.
66
(No Transcript)
67
All students must fall into one of these
categories. For each time point, the percentages
across these four columns 100
68
Mis-Adm Percent of students for whom insufficient
data was provided to determine List/Read of
stories.
69
Listening Percent of students who were
administered stories as Listening rather than
Reading.
70
Read FRU Percent of students who read a story but
were not then administered Listening
Comprehension.
71
Read INS/IND Percent of students who read a story
at an INS or IND level (Instructional or
Independent).
72
In the last three sets of columns (red box) data
is only presented for EOY and represents a break
out of EOY Read INS/IND (blue box) percents by
story number. The total of these last three
columns of data the percent of students at EOY
in the Read INS/IND column.
73
Read S1-S3 INS/IND Percent of students reading
at INS/IND level who read one of stories 1-3.
74
Read S4-S5 INS/IND Percent of students reading
at INS/IND level who read either story 4 or 5.
75
Story Undetermined Insufficient data to determine
the story read.
76
(No Transcript)
77
(No Transcript)
78
Read FRU and Read INS/IND broken out by whether
the student read a Grade 1 or Grade 2 story.
(Students can go back to Grade 1 story). All
bars within each time point still add up to 100
79
Last three columns same as Grade 1. Presents
data broken out only for those students in READ
INS/IND in a second grade story at EOY.
80
(No Transcript)
81
Students can not go back to Grade 2 story, so the
Grade 3 is the same as the Grade 1.
82
Tejas LEE GRAPHS follow the same format as the
TPRI by Task Graphs. The one major difference is
83
Linking Task to Domain Performance
84
Domain with most needPA (56) and CO (41)
85
Domains with most needPA (49) and GK (26)
86
(No Transcript)
87
Domains with most needGK (72) and CO (24)
88
(No Transcript)
89
Domains with most needGK (81) and CO (31)
90
(No Transcript)
91
KindergartenDetecting Initial SoundsDetecting
Final SoundsListening ComprehensionFirst
GradeDetecting Initial SoundsDetecting Final
SoundsSpelling Blends in Final
PositionsReading Accuracy
English Specific Areas of Need
Second GradeSpelling Orthographic
PatternsSpelling Past TenseSpelling
InflectionsThird GradeSpelling Orthographic
PatternsSpelling Past TenseSpelling
ContractionsSpelling Vowel TeamsReading
Accuracy