Performance Based Monitoring Analysis System PBMAS

1
Performance Based Monitoring Analysis System
(PBMAS)

• Presented byMartha Collins and Pam Albritton
• Region VIII ESC
• November 2005

2
Todays Agendano 3s

• What am I here?
• Focused Data Analysis (FDA) and Continuous
  Improvement Plan (CIP) applies to Stages of
  Intervention 1A, 1B, and 2
• Program Effectiveness Review applies to Stages of
  Intervention 1B and 2
• LEA public meeting applies only to Stage of
  Intervention 2

3
TEA Monitoring

• 1) using a data driven, performance based model
  to observe, evaluate, and report on the public
  education system across diverse areas for the
  purpose of assessing that student needs are being
  met
• 2) integrated with the agencys desk audit and
  intervention process and
• 3) research based framework of interventions
  that ensure compliance and enhance student
  success. (p. 3 of PBMAS manual)

is all about the kids
4
How will LEAs be monitored for special education?

• Graduated interventions. All levels will require
  data analysis focused on student performance and
  program effectiveness.
• Interventions will require improvement planning
  based on results of that data analysis.
• Onsite visits will only be used when other
  alternative interventions are not appropriate.

5
Oversight, Sanctions, Interventions
Special Education Monitoring System 2005-2006
NO
YES
ONGOING
Implementation OK?
Focused Data Analysis (FDA) and CIP
Level 1Intervention
Resubmit plan (choose outside support)
NO, 1st Time
Implement CIP Evidence of Change (timely review
and check points)
Evaluation, Findings and CIP submitted to TEA
TEA Desk Review of Self-Evaluation Results, Data,
and Continuous Improvement Plan
FDA, LEA Public Meeting andCIP
YES
Plan OK?
YES
Review OK?
Level 2Intervention
NO, 2nd time
NO
Information Collection and Review(TEA data
andLEA submission)
Oversight, Sanctions, Interventions
Level 3Intervention
FDA, LEA Public Meeting, Compliance Review and CIP
ONGOING
NO
TEA On-Site Review or Contracted On-Site Review
and Resubmit Plan
Plan OK?
Districts w/ substantial or imminent risk
YES
Special Program Compliance Review
Information Collection and Review(TEA data
andLEA submission)
ONGOING
Required level of review and submittal may vary
depending upon initial data review. Other
interventions and sanctions may be used on a
case-by-case basis. Community stakeholders
must be part of self-evaluation team at all
stages of intervention (both required and
recommended team members TBD). CIP
Continuous Improvement Plan
Targeted TEA On-Site Review and Submission of CIP
Other Random Data and Self-Eval check
TEA Program Monitoring and Interventions October
2005
6
Whats new this year besides the name?

• Some indicators have been renumbered
• Some indicators changed to RO
• Manual changes marked New!
• Revised SA process (p. 5)
• Indicator levels modified

7
New!

• Minimum size requirements can be met either in
  the current year or through the aggregation of
  numerators and denominators over multiple years.
  When minimum size requirements are met based on
  two years of data, the numerator and denominator
  for the current and prior year are aggregated,
  the indicator is calculated, and a performance
  level is assigned based on the standard for the
  indicator. If the minimum size requirement is not
  met, then the district is evaluated special
  analysis.
• (p. 12)

8
Minimum Size Exception

• If a district does not meet the minimum size
  requirement for an indicator, but the performance
  of the district is high enough to earn a
  performance level of 0 Met Standard, then the
  district receives a performance level of 0,
  regardless of the number of students in the
  relevant segment of the student population. (p.
  12)

9
How did we get invited to the party?

• Stage 1A
• No individual SPED PBMAS indicator 3 but seven
  or more individual SPED PBMAS indicators 2
• or
• One individual SPED PBMAS indicator 3

10
How did we get invited to the party?

• Stage 1B
• Two individual SPED PBMAS indicators 3
• and
• No more than three individual SPED PBMAS
  indicators 2

11
How did we get invited to the party?

• Stage 2
• Two individual SPED PBMAS indicators 3 and four
  or more individual SPED PBMAS indicators 2
• or
• Three individual SPED PBMAS indicators 3 and no
  more than three individual SPED PBMAS indicators
  2

12
What party?

• Stage 3
• Three individual SPED PBMAS indicators 3 and
  four or more individual SPED PBMAS indicators 2
• or
• Four or more individual SPED PBMAS indicators 3

13
Graduated Interventions

• Stages of Intervention
• 1AFDA (Focused Data Analysis) and CIP
• 1BFDA, Program Effectiveness Review, and CIP
• 2FDA, Program Effectiveness Review, LEA Public
  Meeting and CIP
• 3FDA, Program Effectiveness Review, LEA Public
  Meeting, Compliance Review and CIP
• State Supervision InterventionSpecial Program
  Compliance Review, CIP

14
What documents do we need for a Focused Data
Analysis?

• PBMAS manual http//www.tea.state.tx.us/perfrepor
  t/account/2005/manual/index.html
• Your LEAs 2005 Special Education PBMAS results
• Your LEAs previous CIP, if there is one
• PBMAS templates from the TEA webpage
  http//www.tea.state.tx.us/pmi/spedmon/2006/fda_pe
  r.html
• Data, data, data

15
Purpose of Focused Data Analysis
The purpose of the data analysis and
effectiveness review is to analyze probes and/or
data sets that may point out

• data trends,
• systemic program issues,
• and/or areas of noncompliance with program
  requirements
• Include results of the review in the CIP.
• Make sure the CIP reflects the data analysis
  findings.

16
Scope of Review Required

• http//www.tea.state.tx.us/pmi/spedmon/2006/
• resources/fdaguidance_SPED_06.doc
• FDA required on all indictors rated 2 or 3 for
  all stages of intervention
• Suggested but not required to do an FDA on NE
• Other indictors may be addressed if the LEA
  determines an indictors is in need of further
  review (page 1 of FDA document)
• 1B, 2, or 3 Intervention must complete a program
  effectiveness review

17
Strategies for Data Analysis

• Assemble a Core Analysis Team (CAT)
• Review PBMAS manual to understand the math
• Consider additional data sources to collect more
  information about the data element
• The CAT analyzes the data
• Examine relationships among data elements
• Gather both quantitative and qualitative data and
  review longitudinal trends
• Complete the templates.
• Integrate the findings on the templates into the
  CIP.

18
Step 1 Assemble Core Analysis Team (CAT)
From Focused Data Analysis (FDA) and FDA
with Program Effectiveness Review, TEA website
19
Step 2 Analyze Data

• Use the PBMAS Results and Focused Data Analysis
  Probes Guide, seeing the last page for a section
  applying to all data elements
• Complete a comprehensive data analysis to
  determine issues
• Consider the degree to which data vary across
  campuses and by grade level, area of disability,
  ethnicity/race, economic disadvantage, limited
  English proficiency, gender, and/or other factors
  relevant to the element.
• Inaccurate data repercussions

20
Data Sources and other Stuff to Study for Each
PBMAS Area
1 SPED TAKS Passing Rate 2 SPED Year After
Exit TAKS Passing Rate 3 SDAA II Gap Closure
(Grades 3-8) (RO) 4 SDAA II Gap Closure (Grades
3-8) (RO) 5 TAKS Only Participation Rate
(RO) 6 SDAA II Only Participation Rate
(RO) 7 Statewide Assessment Exemption
Rate 8 LRE (ages 3-5) (RO) 9 LRE (ages
3-11) 10 LRE (ages 12-21)
21
Data Sources and other Stuff to Study for Each
PBMAS Area
11 SPED Annual Dropout Rate 12 SPED RHSP/DAP
Graduation Rate (RO) 13 SPED Identification 14 S
PED AA Representation 15 SPED Hispanic
Representation 16 SPED LEP Representation
(RO) 17 SPED Discretionary DAEP
Placements 18 SPED Discretionary
Expulsions 19 SPED Discretionary Expulsions to
ISS
22
PBMAS Manual, Indicator 13
23
PBMAS Manual, Indicator 13
24
PBMAS Manual, Indicator 19
25
PBMAS Manual, Indicator 19
26
Sample LEA, pages 1-3
27
Special Education PBMAS Indicators

• IndicatorsLeft Side, 1-20
• Columns
• 2005 PBMAS STANDARD () OR STATE RATE
• 2005 DISTRICT RATE OR DIFFERENCE
• 2005 NUMERATOR (COUNT)
• 2005 DENOMINATOR (COUNT)
• 2004 NUMERATOR (COUNT)
• 2004 DENOMINATOR (COUNT)
• 2005 INDICATOR PERFORMANCE LEVEL

28
Alphabet soup?
ND No Data (p. 6) 0/0SA Met the standard for the
indicator 1/1SA Low to Low Moderate
concern 2/2SA Moderate concern 3/3SA High
concern (p. 7) SA Special Analysis by TEA staff
(pp. 12-15) RO Report Only for LEA info/planning
(p. 11) MSR Minimum Size Requirement (p.
12) NE Not Evaluated-could be the result of SA
29
Sample LEA
30
Sample ISD

• Indicator 19 SPED Discretionary Placement to ISS

31
Sample ISD questions

• Was the data systemic, or localized to one or
  more campuses?
• Did the data change from the year before and has
  already changed since the snapshot?
• Do we need to review this down to the student
  level?

32
Last Years CIP

• Did it make a difference or does it need to be
  redone?

33
What else could the LEA try?

• Explore alternative strategies for behavior
  management for sp ed students with multiple
  placements in ISS.
• Analyze teacher referrals for patterns.
• ISS instructor to provide monthly reports
  indicating the number of students that have been
  placed and the number of repeat placements.

34
What else could the LEA try?

• Provide additional training to administrators in
  developing and implementing more effective BIPs.
  
• Provide training in manifestation determination
  process.
• Provide additional training for staff in
  classroom management specifically for sp ed
  students.
• Assess the counseling needs for sp ed students
  assigned to ISS.

35
What else could the LEA try?

• Review/revise the criteria for all student ISS
  placement particularly those students with
  chronic behavior patterns and implementing
  administrators discipline plans.
• Establish a committee to review behavior
  following the 5th removal from the instructional
  setting consisting of a parent, student,
  teacher(s), administrator and others deemed
  appropriate for the individual student/situation.

36
Charts can help tell the story
37
Step 3 Download and complete the Focused Data
Analysis Templates
 
38
Step 4 FDA into CIP

• Integrating the Information on the Focused Data
  Analysis Template(s) into the Continuous
  Improvement Planning Process

39
What we learned last year about Random
Verification

• Keep written agendas, minutes, and sign-in sheets
  of all committee meetings associated with this
  process. TEA will likely ask to see the
  documentation.
• Keep documentation of everything in one place so
  you can fax it within 24 hours, if asked by TEA.
• Make a timeline of actions in the CIP, then put
  someone in charge of keeping track of it. TEA
  will likely ask for documentation of CIP
  implementation.

40
Questions?