Involving Families

1

• Involving Families

2
Informing Families

• What are you doing to inform families about the
  data collection?
• Why it is occurring
• What it involves
• What it means for them and their child

3
www.the-eco-center.org

• Federal and State Activities
• State Activities
• http//www.fpg.unc.edu/eco/whatstates.cfmState_s
  ites

4
(No Transcript)
5
Preparing Families

• Helping families be active participants in the
  discussion
• What is working?
• What is not working?
• General principle Families need to know what to
  expect

6
What Do We Expect from Families

• Yes - That they will be able to provide rich
  information about their childs functioning
  across settings and situation
• Maybe but not necessarily That they will know
  whether their child is showing age appropriate
  behavior

7
Involving Families in a Conversation about Their
Child

• Avoid jargon
• Avoid questions that can be answered with a yes
  or no
• Does Anthony finger feed himself?
• Ask questions that allow parents to tell you what
  they have seen
• Tell me about how Anthony eats

8
Involving Families in a Conversation about Their
Child

• What is working?
• What is not working?

9
Involving Families in the Rating Discussion

• What of families are participating?
• How are you doing this?
• What is working?
• What is not working?

10
Families Right to COSF Information

• All families have a right to know what ratings
  have given to their child -- and to the records
  containing the information.

11
Quality Review of Discussion

• Activity
• Including Families in the Rating Discussion

12
Looking for Quality Data
I know it is in here somewhere
13

• Activity
• What factors work to improve the quality of your
  data?
• What factors work to lessen the quality of your
  data?
• How to address these factors?

14
Take Home Message

• If you conclude the data are not (yet) valid,
  they cannot be used for program effectiveness,
  program improvement or anything else.
• What do you if the data are not as good as they
  should be?
• Answer Continue to improve data collection
  through ongoing quality assurance

15
Many steps to ensuring quality data
16
Many steps to ensuring quality data
17
Promoting quality data

• Training and support before and during data
  collection
• Analysis of the data after data collection
• Data system and verification after data collection

18
Promoting Quality Data

• Through training and communication related to
• Assessment
• Understanding the COSF
  process
• Age expectations
• Data entry

19
Promoting Quality Data

• Through training materials, such as
• Video team and child examples
• Written child examples
• Quizzes for ensuring learning
• Refresher trainings
• Beware of Drift!!

20
Promoting Quality Data

• Through data systems and verification, such as
• Data system error checks
• Good data entry procedures

21
Many steps to ensuring quality data
22
Ongoing supervision

• Review of the process
• Is the process high quality?
• Are teams reaching the correct rating?
• Methods
• Observation
• Videos

23
Quality Review of COSF Team Discussion

• 1. Do all team members participate in the
  discussion?
• 2. Is parent input considered in the rating?
  Give examples.
• 3. Is the team documenting the rating discussion?
  Give examples.
• 4. Does the team discuss multiple assessment
  sources? What are they?

24
Quality Review of COSF Team Discussion

• 5. Does the team describe the childs
  functioning, rather than just test scores? Give
  examples.
• 6. Does the discussion include the childs full
  range of functioning, including skills and
  behaviors that are age appropriate, immediate
  foundational, and leading to immediate
  foundational? Give examples.

25
Quality review through process checks

• Provider surveys
• Self assessment of competence
• Knowledge checks
• Process descriptions (who participates?)
• Identification of barriers
• Kansas survey
• Alaskas survey

26
Questions from Alaskas Survey
27
Ongoing Supervision

• Feedback to teams is critical
• Refresher training
• Beware of
• Auto pilot
• Drift

28
Quality Review of COSF Team Discussion

• Activity
• Observe team video
• Evaluate quality

29
Many steps to ensuring quality data
30
Quality Review of Completed COSFs

• Is the COSF complete?
• Is there adequate evidence for the basis for the
  rating?
• Does the evidence match the appropriate outcome
  area?
• Is the evidence based on functional behaviors?

31
Quality Review of Completed COSFs

• Is there evidence that the childs functioning
  across settings and situations considered?
• Are the ratings consistent with the evidence?

32
Quality Review of COSF

• Activity
• Review completed COSF with errors

33
Promoting quality data through data analysis

• Examine the data for inconsistencies
• If/when you find something strange, look for
  other data that might help explain it.
• Is the variation caused by something other than
  bad data?

34
The validity of your data is questionable if

• The overall pattern in the data looks strange
• Compared to what you expect
• Compared to other data
• Compared to similar states/regions/school
  districts

35
COSF Ratings - Fall
36
Outcome 3 Appropriate Action
37
Outcome 3 Appropriate Action
38
Outcome 3 Appropriate Action
39
Outcome 3 Appropriate Action
40
OSEP Categories
41
Questions to ask

• Do the data make sense?
• Am I surprised? Do I believe the data? Believe
  some of the data? All of the data?
• If the data are reasonable (or when they become
  reasonable), what might they tell us?

42
Validity

• Validity refers to the use of the information
• Does evidence and theory support the
  interpretation of the data for the proposed use?
• Or
• Are you justified in reaching the inference you
  are reaching based on the data?
• Standards for Educational and Psychological
  Testing (1999) by American Educational Research
  Association, American Psychological Association,
  National Council on Measurement in Education

43
The validity of your data is questionable if

• ?

44

• .not all providers are not knowledgeable about
  in the COSF process
• not all providers are careful with the COSF
  process
• the data look strange
• etc.

45
Many steps to ensuring quality data