Measuring the

1
Measuring the Intervention Footprint Issues of
Planning, Documentation, Follow-ThroughJim
Wrightwww.interventioncentral.org
2
Elbow Group Activity Defining Interventions
In your group, define the term intervention.
Come up with guidelines for judging when teacher
activities should be considered interventions.
3
RTI Interventions A Definition (Wright, 2007)

• Interventions are specific strategies adopted
  to help students to make progress toward academic
  or behavioral goals.

Source Wright, J. (2007). The RTI toolkit A
practical guide for schools. Port Chester, NY
National Professional Resources, Inc.
4
Matching Interventions to Appropriate Tier Levels

• Simple interventions can stand alone as Tier I
  (classroom) strategies.
• Simple interventions can also be used in higher
  Tiers as part of a larger intervention package.

5
Elements of an Effective Intervention Plan
(Grimes Kurns, 2003)

• Intervention design and implementation.
  Interventions are designed based on the preceding
  analysis, the defined problem, parent input, and
  professional judgments about the potential
  effectiveness of interventions. The interventions
  are described in an intervention plan that
  includes goals and strategies a progress
  monitoring plan a decision-making plan for
  summarizing and analyzing progress monitoring
  data and responsible parties. Interventions are
  implemented as developed and modified on the
  basis of objective data and with the agreement of
  the responsible parties.

Source Grimes, J. Kurns, S. (2003). An
intervention-based system for addressing NCLB and
IDEA expectations A multiple tiered model to
ensure every child learns. Retrieved on September
23, 2007, from http//www.nrcld.org/symposium2003/
grimes/grimes2.html
6
Key Steps to Academic Intervention Planning

• Check for student motivation
• Apply the Instructional Hierarchy
• Define student academic problems in specific
  terms
• Package interventions as teacher-friendly scripts
• Verify that the intensity of an intervention
  appropriately matches the current RTI Tier of the
  student
• Measure intervention follow-through
• Assemble an Intervention Bank to have
  research-based ideas at hand when needed

7
Writing Quality Problem Identification
Statements
8
Writing Quality Problem Identification
Statements

• A frequent problem at RTI Team meetings is that
  teacher referral concerns are written in vague
  terms. If the referral concern is not written in
  explicit, observable, measurable terms, it will
  be very difficult to write clear goals for
  improvement or select appropriate interventions.
• Use this test for evaluating the quality of a
  problem-identification (teacher-concern)
  statement Can a third party enter a classroom
  with the problem definition in hand and know when
  they see the behavior and when they dont?

9
Writing Quality Problem-Identification
Statements Template
10
Writing Quality Teacher Referral Concern
Statements Examples

• Needs Work The student is disruptive.
• Better During independent seatwork , the student
  is out of her seat frequently and talking with
  other students.
• Needs Work The student doesnt do his math.
• Better When math homework is assigned, the
  student turns in math homework only about 20
  percent of the time. Assignments turned in are
  often not fully completed.

11
Judging the Intensity of Interventions Tier I,
II, or III?
12
Dimensions of Interventions Treatment Strength

• By strength of treatment, we refer to the a
  priori likelihood that the treatment could have
  its intended outcome. Strong treatments contain
  large amounts in pure form of those ingredients
  leading to change. Assessments of strength are
  made independently of knowedge of outcome of
  treatment in any given case. p. 156

Source Yeaton, W. H. Sechrest, L. (1981).
Critical dimensions in the choice and maintenance
of successful treatments Strength, integrity,
and effectiveness. Journal of Consulting and
Clinical Psychology, 49, 156-167.
13
Why Attempt to Judge the Intensity of
Interventions?

• Judging the intensity of interventions in
  advance ensures that intervention plans match the
  RTI Tier in which they are being used.

14
Avoiding the Intervention Trap

• When planning Tier II (individualized)
  interventions, RTI Teams should take care to
  ensure that those plans are feasible and
  maintainable in general-education settings. If
  a Tier II intervention is so ambitious as to
  resemble a Special Education (Tier III) program,
  the team may find that the student responds well
  to the plan but would still lack information
  about whether the student requires more support
  than general education can offer. And the plan
  may not be maintainable!

15
Intervention Intensity Rating Form (pp. 109-111)
16
Sample Intervention Intensity Rating Form Items
2.On a per-pupil basis, the cost to purchase or effort needed to create intervention materials Intervention materials not needed or do not entail significant expense or effort Intervention materials required but can be obtained at a modest cost or with reasonable effort Intervention materials per pupil are costly or require substantial effort to create
4. Amount of preparation required for each session of the intervention Little or no preparation is needed Some preparation is needed (up to 15 minutes per session) Substantial preparation is needed (more than 15 minutes per session)
9. Potential of the intervention to distract other students or disrupt their learning Intervention can be implemented with little or no distraction of other students or disruption to their learning Intervention is likely to result in mild distraction of other students or disruption to their learning Intervention is likely to result in significant distraction of other students or disruption to their learning
17
Intervention Intensity Rating Form
Guidelines for Interpreting Results If 7 or more
of your ratings on this 10-item form fall under
any single Tier, it is likely that the
intervention has a level of intensity matching
that Tier as well. An intervention with 8 checks
under the Tier II column, for example, should be
considered a Tier II intervention. If you have
a mixed pattern of ratingswith no single column
containing 7 or more checkscount up the number
of checks in each column. The intervention should
be considered equivalent in intensity to the
highest column that contains 3 or more checks.
(Tier I is the lowest column. Tier III is the
highest.) An intervention with more than 3
checks under the Tier III column, for example,
would be considered a Tier III intervention.
18
Evaluating Intervention Follow-Through
(Treatment Integrity)
19
Treatment Integrity Activity In your elbow
groups, discuss the following question How does
your school measure the quality of intervention
follow-through in classrooms?
20
What Consultant Factors Can Increase Teacher
Intervention Follow-Through (DiGennaro et al.,
2007)

• Study contrasted two conditions of teacher
  support for behavioral concerns (with four
  special education teachers participating)
• Condition 1 Goal Setting and Student Performance
  Feedback Teachers set goals for student
  improvement and received daily written and
  graphed feedback about student performance.
• Condition 2 Teacher Performance Feedback and
  Direct Rehearsal With Meeting Cancellation
  Teachers received daily feedback about their own
  performance in implementing the intervention, as
  well as student performance feedback. If teachers
  did not implement the intervention with 100
  integrity, they met with the consultant to
  practice the missed steps. If they carried out
  the intervention with full integrity, they were
  able to skip the consultant meeting.

Source DiGennaro, F. D., Martens, B. K.,
Kleinmann, A. E. (2007). A comparison of
performance feedback procedures on teachers
treatment implementation integrity and students
inappropriate behavior in special education
classrooms. Journal of Applied Behavior Analysis,
40, 447-461.
21
What Consultant Factors Can Increase Teacher
Intervention Follow-Through (Cont.) (DiGennaro
et al., 2007)

• Study outcome
• Teachers had the highest rates of intervention
  integrity under Condition 2 Teacher Performance
  Feedback and Direct Rehearsal With Meeting
  Cancellation.
• However, two of four participating teachers rated
  elements of Teacher Performance Feedback and
  Direct Rehearsal With Meeting Cancellation
  condition as unacceptable.
• The study concluded that allowing teachers to
  practice a skill and then avoid meeting with a
  consultant once skill acquisition in the natural
  setting is observed appears to be an effective
  means to promote treatment integrity. p. 458

Source DiGennaro, F. D., Martens, B. K.,
Kleinmann, A. E. (2007). A comparison of
performance feedback procedures on teachers
treatment implementation integrity and students
inappropriate behavior in special education
classrooms. Journal of Applied Behavior Analysis,
40, 447-461.
22
Why Monitor Intervention Follow-Through?

• If the RTI Team does not monitor the quality of
  the intervention follow-through, it will not know
  how to explain a students failure to respond to
  intervention.
• Do qualities within the student explain the lack
  of academic or behavioral progress?
• Did problems with implementing the intervention
  prevent the student from making progress?

23
What Are Potential Barriers to Assessing
Intervention Follow-Through?

• Direct observation of interventions is the gold
  standard for evaluating the quality of their
  implementation. However
• Teachers being observed may feel that they are
  being evaluated for global job performance
• Non-administrative staff may be uncomfortable
  observing a fellow educator to evaluate
  intervention follow-through
• It can be difficult for staff to find time to
  observe and evaluate interventions as they are
  being carried out

24
Intervention Script Builder pp. 107-108
25
Teacher Intervention Evaluation Log p.112
26
END