DataBased DecisionMaking: ProblemSolving Approach

1
Data-Based Decision-Making Problem-Solving
Approach

• Lisa Kelly-Vance, Ph.D.
• ESU 6 2/11/09

2
Data-Based Decision-Making

• Main Topics for Today
• Teaming
• Problem solving process and applications
• Use of data to inform decisions

3
Rule 51 Student Assistance Team (SAT) or
Comparable Problem Solving Team

• For a school age student, a general education
  student assistance team or a comparable problem
  solving team shall be used prior to referral for
  multidisciplinary team evaluation.
• The SAT or comparable problem solving team shall
  utilize and document problem solving and
  intervention strategies to assist the teacher in
  the provision of general education.
• If the student assistance team or comparable
  problem solving team feels that all viable
  alternatives have been explored, a referral for
  multidisciplinary evaluation shall be completed.

4
Applications

• General and special education
• Individuals, groups, systems
• Tiers 1, 2, and 3
• Behavioral, social-emotional, academic, language
  problems (etc.)
• All ages/grades

5
Teams educational definition

• A group of educators who are served with the
  purpose of enhancing the educational opportunity
  for ALL students

6
Teams

• Problem-solving teams are required by law
• Teams should benefit ALL students
• Administrative support is critical
• Current SAT structure in many schools may need
  modification
• Most teams are overworked
• Most school teams collect minimal data which is
  required by law
• Teams dont know if they are effective
• Brainstorming is not an effective strategy
  replace with problem-solving approach

7
Recommendation

• Have two active teams
• Response to Intervention Team
• Problem-Solving Team
• Membership may overlap or be identical (smaller
  districts)

8
Response to Intervention Team

• Analyze benchmarks
• Determine cutoff scores for movement between
  tiers
• Determine tier placements
• Oversee intervention process
• Can determine movement across groups
  decision-making rules
• Problem-solving is used at the systems and small
  group levels

9
Problem-Solving Team

• Use problem-solving model
• Apply to individuals and small groups who are
  receiving more intense services
• This should include students in special education
• Can determine movement across groups
  decision-making rules

10
Overall Team Purpose

• Plan and evaluate individualize interventions
• Monitor student progress
• Adjust interventions base on student performance
  data
• Guide data-based decision making about student
  responsiveness to intervention and instructional
  decisions
• From RtI Consortium/NDE

11
Team Members Roles and Functions

• Need people with expertise in
• Data collection, monitoring and analysis
• Decision rules
• Problem solving process
• Scientifically based interventions
• Meeting logistics

12
Team Members

• Case Coordinator
• Parent/Guardian
• Administrator
• Teacher(s)
• Support staff and specialists as needed and as
  available
• Examples
• School psychologist
• Speech/language pathologist
• Reading specialist

13
Pre-meeting activities

• Referring teacher completes appropriate paperwork
  and submits referral
• Case manager is assigned to the referral
• Case manager works with referring teacher to
  collect preliminary information
• Problem identification
• Problem analysis
• Case manager researches scientifically supported
  interventions

14
Problem Solving Team Meeting

• Review Problem Identification and Problem
  Analysis information with team
• Determine goal
• Present interventions
• Select interventions and plan implementation
• Plan data collection
• Determine the decision-making plan
• Assign roles
• Set next meeting date

15
Review Meeting

• Review progress monitoring data
• Determine if changes are needed

16
Points of Emphasis

• This is a process the steps cannot all be done
  in one meeting
• May do this process in teams or in individual
  consultation
• Data-based decision making is critical component
• Assessment is involved along the way in various
  forms
• Having forms (Intervention Planning Worksheet)
  helps guide the process

17
How to Maximize Effectiveness

• Start by analyzing the core curriculum
• Make the appropriate modifications so that 80-85
  of the students are successful in the core
  curriculum
• May mean increasing/changing instruction,
  changing curriculum,
• Use a scientifically-supported core
• After the core is fully supported, can be more
  strategic with who gets help reduces need

18
How to Maximize Effectiveness

• If Tier 1 is not addressed, staff will be doing
  more small group and individual interventions
  than can be addressed by existing resources.
• If Tier 1 is addressed appropriately, staff can
  then be strategic and creative with resources
  and
• Have a plan for nonresponders
• Form groups with similar needs
• Use standard protocol approach with groups

19
Aligns with

• School Improvement
• Title I
• Special Education
• Reading First
• Etc.

20
Now for the Process..

• Problem Solving Approach allows teams to use a
  systematic approach to solving educational
  problems (i.e., Data-based Decision Making)

21
Steps in the Problem-Solving Process

• Problem Identification
• Behavioral Definition
• Problem Analysis
• Possible Reasons for the Behavior
• Data Collection - Baseline
• Goal Setting
• Plan Implementation
• Intervention (empirically-based)
• Data Collection Graph/Chart
• Plan Evaluation

22
Problem Identification

• Behavioral Definition
• Select ONE behavior to address
• Remember that the problem can be at the
  individual, group, or system level
• Behavior means that/is when describe behavior
• also includes
• Examples
• Nonexamples

23
Problem Identification

• Characteristics of well-defined behavior
• Concrete two different observers would agree
• Observable can see or hear it
• Specific describes one behavior
• Measurable can be counted

24
Problem Identification

• Dimensions of behavior (i.e., what about the
  behavior is problematic?)
• Frequency behavior happens too much or too
  little
• Duration behavior happens too long or too short
  of a period of time
• Accuracy behavior doesnt happen correctly

25
Problem Identification

• Dimensions of behavior (i.e., what about the
  behavior is problematic?)
• Latency behavior takes too long to begin after a
  prompt
• Topography behavior occurs but is inappropriate
  or inefficient
• Intensity behavior is too loud, forceful, or too
  soft, passive, etc.

26
Problem Identification

• Example 1
• Behavior Temper tantrums mean that Edward
  screams and runs around the room
• Examples screaming, throwing objects, hitting
  himself, kicking
• Nonexamples not following directions, stubborn
  behavior, silent refusals

27
Problem Identification

• Example 2
• Behavior Not following directions means that
  Frank does not do what the teacher requests.
• Examples not responding to individual
  directions, not following group directions, does
  another activity when directions are given
• Nonexamples not following routines of the
  school/classroom, compliance after a period of
  time, complies after several prompts/reminders

28
Problem Identification

• Assessment
• Interview
• Teachers
• Parents
• Student
• Other personnel
• Observe
• Where the behavior is occurring
• Different settings

29
Problem Identification

• Make sure that you have the following
• A reliable behavior
• A valid behavior
• There are often many changes at this early stage
  in the process.

30
Example Problem Identification

• Behavior of Concern Sally is performing below
  grade level in reading, particularly in the areas
  of letter naming, letter sounds, and phonemic
  awareness tasks.
• Examples receiving lower grades on
  reading/alphabet assignments than peers, taking
  longer to complete reading assignments and
  activities than peers, not being able to answer a
  question or giving an incorrect response, needing
  additional assistance from a teacher to complete
  a reading task
• Nonexamples missing work because of illness or
  some other absence, performing at the same level
  as peers on reading assignments, completing
  assignments on time and without assistance

31
Problem Analysis

• The hypothesis of why the behavior is occurring
• Helps develop the link between observed
  performance (behavior of concern) and treatment
• The because statement

32
Problem Analysis

• Process
• Identify all possible explanations for the
  behavior of concern
• Review the list and determine if you can impact
  all the reasons of the list if not, eliminate
  from list
• From the remaining possible reasons, select the
  most plausible one. This becomes your working
  hypothesis. Develop prediction statements
• Ask questions about the hypothesis. This becomes
  your assessment process.
• Test the hypothesis through an assessment process
• Your assessment may result in changing the
  possible reason.
• The last 2 steps are critical to an effective
  process.

33
Problem Analysis - ICEL

• Categories of Possible Reasons for Behavior
• Instruction (teacher) how we teach what is
  being taught
• Curriculum what is being taught (sequence,
  content)
• Environmental (classroom) physical conditions
  in students surroundings
• Learner (student)

34
Problem Analysis Cant do/Wont do

• Can test to distinguish between the two
• Use reinforcement to test

35
Problem Analysis - FBA

• Functional Analysis of the Behavior
• Conditions
• Antecedents what comes before behavior
• Consequences what follows behavior (reinforcer)

36
Problem Analysis

• Assessment testing the hypothesis
• Review
• Existing data
• Past teacher reports
• Cumulative file
• Interview
• Teachers
• Parents
• Student
• Other personnel

37
Problem Analysis

• Assessment cont.
• Observe very useful!
• Where the behavior is occurring
• Different settings
• Test
• Questionnaires/checklists
• Standardized tests

38
Problem Analysis - Example

• Mike cannot do math story problems because he
  does not have the necessary prerequisite skills
  of addition and subtraction
• Prediction Statements
• If retaught the skills of addition and
  subtraction, he will be able to do math story
  problems
• It not retaught the skills of addition and
  subtraction, he will not be able to do math story
  problems
• Question What is Mikes skill level in addition
  and subtraction?
• Assessments
• Review past report cards, group testing
• Interview talk with teacher present and past
• Observe in math class and look for antecedents
  and consequences of not doing math story problems
• Test Conduct math evaluation can use
  Curriculum Based Measurement

39
Example Possible Reasons for Behavior

• Student Characteristics low vision and/or
  hearing, difficulty staying on-task, skill
  deficit in reading, lack of motivation
• Classroom (Environment) no prior exposure to
  school or reading, distracting classroom
  environment, too many peers
• Instruction directions are too complex, type of
  instruction used, too much information presented
  at once
• Curriculum too complex
• Family/Resources lack of reading materials at
  home, parental absence, family behaviors

40
Example FBA

• Conditions
• Antecedents Sally is presented with a reading
  assignment, direction, or activity/task to
  complete
• Behavior Sally does not complete the
  assignment/task, answer the question, or follow
  directions
• Consequences Sally gets frustrated, is off-task,
  does not complete the assignment, and/or gets
  help from a teacher

41
Example Selected Hypothesis

• Sally is below grade level in reading because she
  has not had adequate exposure to school and
  academic tasks such as reading the alphabet or
  practicing letter names and sounds.
• Prediction statements
• If taught skills related to reading and
  pre-reading, she will be able to complete reading
  tasks and perform at comparable level to peers
• If not taught reading and pre-reading skills, she
  will not be able to do complete reading tasks and
  improve to reach the level of her peers.
• Questions
• What is Sallys skill level in reading?
• Will she respond to direct teaching?

42
Data Collection - Baseline

• Select a measurement strategy
• How will you measure the behavior?
• Who will measure the behavior?
• What will you need to measure the behavior?
• Where will you measure the behavior?
• Use the same data collection procedure when
  collecting baseline data and when progress
  monitoring.

43
Data Collection - Baseline

• Measurement Strategies
• Event
• a tally or frequency count of the target behavior
  as it occurs.
• Use when.
• (1) the length of time in each observation
  session remains constant
• E.g., each observation session lasts 20 minutes

44
Data Collection - Baseline

• Event (cont.) use when.
• (2) the number of opportunities to perform the
  target behavior remain constant
• E.g., the child always has ten trials to
  demonstrate the behavior

45
Data Collection - Baseline

• Convert frequency count data to a percentage when
• (1) the length of time in each observation
  session varies
• (2) the number of opportunities to perform the
  target behavior vary

46
Data Collection - Baseline

• Latency
• Measures the amount of time between presentation
  of a prompt (stimulus) and initiation of the
  target behavior

47
Data Collection - Baseline

• Duration
• The amount of time between the beginning of a
  behavior and its conclusion is recorded

48
Data Collection - Baseline

• Collect baseline information
• Use the measurement strategy selected
• Collect sufficient data to be stable and
  representative
• Stable requires a minimum of 3 data collection
  points
• Visually display the data on a graph

49
Example Data Collection

• Measurement Strategy The number of letters Sally
  reads correctly in 1 minute on an AIMSweb Letter
  Naming Fluency probe.
• Materials One AIMSweb Letter Naming Fluency
  probe for each week, pencil, stopwatch
• Setting Reading specialists classroom
• Person Responsible Reading specialist

50
Example Data Collection

• Baseline Data
• 09/08/08 0 letter names
• 09/17/08 1 letter name
• 09/30/08 5 letter names
• Comparison Data
• Typical peers in Kindergarten at Sallys school
  were able to name 10 letters in 1 minute. The
  current average rate of improvement for same age
  peers is 2.53 letter names correct per week.

51
Goal Setting

• Select a performance standard (used to judge
  baseline and progress)
• Types of standards
• Norm referenced
• Discrepancy from peers
• Rate of increase (ROI) or decrease
• Criterion referent
• Benchmarks
• Performance standards
• Expert opinion

52
Goal Setting

• Child the learner
• Criterion standards for evaluation, acceptable
  level of behavior, measures the effectiveness of
  the intervention strategies, and sets the
  standard for evaluation (accuracy, frequency,
  duration, latency)
• Conditions time frame, situation/setting,
  stimuli to elicit behavior, date by which goal is
  to be achieved and measurement material
• Behavior description of the task to be performed

53
Example Goal

• By January 5, 2009, Sally will read 29 letter
  names correctly from a Kindergarten AIMSweb
  Letter Naming Fluency probe.
• Her rate of improvement will be 1.75 letter names
  correct per week.

54
Plan Implementation

• Intervention Scientific Research-Based
• Front-loading
• Review the literature
• Review websites
• Review conference/workshop information
• Be sure that your intervention has initial
  support
• Back-loading
• comes from your data collection and
  decision-making

55
Plan Implementation

• The selected intervention.
• Must have empirical support
• Must address the behavior of concern
• Must be derived from the hypothesis
• Must match the students educational environment
• Must be collaboratively developed
• Must be acceptable

56
Plan Implementation

• Intervention must contain the following
• Instructional procedures/teaching strategy
• Materials
• Arrangements
• Time
• Motivational strategies
• Treatment Fidelity plan
• Intervention vs. Accommodation

57
Plan Implementation

• Interventions also must have the following
  qualities/criteria
• Delivered or supervised by skilled, certified
  personnel
• Those delivering the intervention must be
  adequately trained on the intervention
• Must last a minimum of 14 weeks and 56 sessions
  with at least one substantive change in
  information prior to being identified as
  inadequately responding to intervention
• Sessions are at least 30 minutes in length
• Changes are made based on sufficient amount of
  time and data (e.g., 2 7-week periods with weekly
  data collection and a minimum of 28 intervention
  sessions)

58
Plan Implementation

• Treatment Fidelity
• Was the intervention implemented as planned?
• Use of scripts
• Can observe or have a checklists to determine
  treatment integrity
• Importance of feedback
• Document any modification in intervention
• Modifications should only be made based on data

59
Example Plan Implementation

• Scientific support literature review reference
  list
• Intervention
• Person Responsible Reading Specialist
• Instructional Procedure/Teaching Strategy small
  group direct reading instruction with 2 other
  students focusing on letter names, letter sounds,
  reading books, practicing sight words, creating
  stories, and practicing with flashcards.
• Materials Needed alphabet books, flashcards with
  letter names, sounds, sight words, timer, white
  boards
• Arrangements/Location Reading Specialists
  classroom
• Amount of Time 30 minutes per day, every day of
  the week
• Motivational Strategies/Reinforcement Plan
  Verbal praise for correct answers and effort, and
  re-directions when she is not paying attention or
  staying on task

60
Example Plan Implementation

• Decision-Making Plan
• Sally will receive the intervention for 45 days
• after this time period, her progress will be
  evaluated, and the team will decide
• whether she should continue to receive the
  intervention
• whether she can receive reading instruction in
  her general education classroom
• or whether an evaluation for special education
  services should be initiated.

61
Example Plan Implementation

• Treatment Integrity Plan
• The Reading Specialist will follow the curriculum
  outlined by the team of reading specialists
• Regular meetings with the school principal to go
  over the plan and students progress
• Ensure all components are included and all steps
  are being followed
• Direct observation of intervention sessions

62
Plan Implementation

• Graph/Chart
• Use to monitor progress
• Effective means of showing student performance to
  team and to the student
• Use for analyzing performance trends

63
Plan Implementation

• Graphing procedures
• Write the goal on the chart
• Label the horizontal and vertical axes
• Plot the baseline data (all data points)
• Draw line to separate baseline data from progress
  monitoring data
• Place an X at the point representing the median
  baseline score
• Plot goal data point
• Draw goal line (also called aim line)
• Plot data points develop a trend line
• Mark changes in intervention on the graph
• Tip Use EXCEL

64
Example Graph
65
Plan Evaluation

• Should be determined after baseline data are
  collected and before the intervention begins
• A frequently overlooked aspect of data collection
  and progress monitoring
• This is the way we determine if the intervention
  needs to be modified (back-loading)

66
Plan Evaluation

• Regularly review data base the timeline on your
  plan
• Consecutive Data Point rule
• 4-5 consecutive data points above goal, raise
  goal
• 4-5 consecutive data points below goal, change
  intervention
• Need a minimum of 16-18 data points (Hintze,
  2007)

67
Plan Evaluation

• Before reviewing data, the graph should include
  (at a minimum)
• Baseline data
• At least 7/8 data points after intervention is
  implemented
• Aim line
• Trend line
• Phase changes
• From RtI Consortium/NDE

68
Plan Evaluation

• Progress Review
• Keep the child aware of his/her progress
• Conduct periodically
• How often depends on the problem
• Use your data
• Make changes if needed - mark these on the graph
• Include all involved individuals
• May lead to special education decision

69
Example Plan Evaluation

• Modifications/Decision Sally is making progress
  on Letter Naming Fluency, and this progress has
  been consistently above the goal line, however,
  she is still below her peers in this and other
  areas related to reading.
• Sally will continue to receive the small group
  reading instruction from the Reading Specialist,
  and her progress will be monitored weekly and
  reviewed after 45 days.
• Outcome data
• 12/03/08 19 letter sounds
• 12/03/08 24 letter names read correctly in 1
  minute

70
Eligibility in an RtI Model

• All components of RtI must be in place
• Data from Tiers 1 and 2 are essential to the
  decision
• Data from Tier 3 problem-solving must be reported
  multiple treatment phases
• Treatment fidelity is critical

71
Eligibility in an RtI Model

• Consider ALL of the following
• Discrepancy from peers
• Amount and type of resources needed for progress
  to occur (i.e., is it beyond the capabilities of
  general education?)
• Rate of progress

72
Helpful Websites

• National Student Center on Student Progress
  Monitoring
• www.studentprogress.org
• Interventioncentral.org
• Research Institute on Progress Monitoring
• www.progressmonitoring.net
• AIMSWEB www.aimsweb.com
• Behavioral Research and Teaching Institute
• http//brt.uoregon.edu
• DIBELS www.dibels.uoregon.edu
• Positive Behavior Supports www.pbis.org
• What Works Clearinghouse www.whatworksclearinghou
  se.org
• Florida Center for Reading Research www.fcrr.org
• National Association of State Directors of
  Special Education www.nasdse.org

73
References

• Brown-Chidsey, R., Steege, M.W. (2005).
  Response to Intervention Principles and
  strategies for effective practice. New York, NY
  The Guilford Press
• National Association of State Directors of
  Special Education, Inc. (2005). Response to
  Intervention Policy considerations and
  implementation. New York, NY The Guilford
  Press.
• Rathvon, N. (1999). Effective school
  interventions Strategies for enhancing academic
  achievement and social competence. New York The
  Guildford Press.
• Thomas, A., Grimes, J. (2008). Best Practices
  in School Psychology V. Bethesda, MD National
  Association of School Psychologists.