Toolkit

1
Toolkit 3 Using Data to Make a Difference

• Carey Dimmitt , Ph.D.
• Center for School Counseling Outcome Research
• Providence, RI
• February 4, 2005

2
Why Use Data?

• Data can convince people of the need to change.
• Data can uncover problems that might otherwise
  remain invisible.
• Data can confirm or discredit assumptions about
  students and school practices.
• Data can get to the root of problems, pinpoint
  areas where change is most needed and guide
  resource allocation.
• Data can help schools evaluate program
  effectiveness and keep the focus on student
  learning outcomes.

3
Why Use Data?

• Data can provide feedback to counselors, teachers
  and administrators about what is working, what
  needs to continue, and what can be cut.
• Data can prevent over-reliance on standardized
  test scores.
• Data can prevent one-size-fits-all and quick-fix
  solutions.
• Data can give schools the ability to respond
  accurately to accountability questions.
• Data can help build a culture of inquiry and
  continuous improvement.

4
Why Use Data?Jay Careys Postulates

• Focusing on data can free people from guilt and
  blame and allow them to have conversations that
  would not happen otherwise.
• Good people can have bad data.

5
Data-Driven School Counseling Programs

• Implement comprehensive programs based on
  national design and local need
• Use data to determine directions and to make
  decisions
• Measure results
• Share successes

6
Types of Data

• The ASCA National Model identifies three types
  of data
• Process Data
• Perception Data
• Results Data

7
Process Data

• Process Data What was done for whom?
• Who received services?
• Seventh graders? Students at risk of failing?
• What did they receive?
• Curriculum intervention? Small-group
  intervention?
• When did they receive it?
• All year? Twice? For 30 minutes?
• Where and How was it provided?
• In the classroom? After school?

8
Process Data

• Process data alone does not tell us whether or
  not the student is different (in behavior,
  attitude or knowledge) as a result of this
  activity. It tells what events occurred.
• Coupled with results data, process data can help
  identify what factors may have led to success in
  an intervention.

9
Perception Data

• Perception data measures how students are
  different as a result of an intervention.
• Did students gain competencies?
• Every 10th grade student completed an interest
  inventory.
• 85 of 6th graders identified the steps in the
  conflict resolution process.
• Did they gain knowledge?
• 87 of 9th graders demonstrated knowledge of
  graduation requirements.
• Were there changes in their attitudes or beliefs?
• 86 of students believe smoking cigarettes is
  unhealthy.

10
Perception Data

• Differences in student knowledge, competency
  and attitudes are measured through
• Pre-post tests
• What do students know/believe before and after
  the intervention?
• Completion of an activity
• Completion of a 4-year plan
• Surveys
• What do students say they believe or know?

11
Results Data

• Results data is the proof that the intervention
  has or has not influenced students behaviors.
  An intervention may occur (process data),
  students may know the information (perception
  data), but the final question is whether or not
  the students are able to utilize the knowledge,
  attitudes and skills to affect behavior (results
  data).

12
Results Data

• Results data can be complex because many factors
  impact behavior change. A decrease in
  disciplinary referrals for bullying may be the
  result of a violence prevention curriculum, but
  there are likely to be other things influencing
  that outcome as well. Conversely, finding no
  changes in results data does not mean that an
  intervention has necessarily been unsuccessful.

13
Results Data

• How do we measure what our results are?
• How do we show that what we do makes a difference?

14
Data Can Be Used To

• Evaluate interventions and programs
• Measure results and outcomes
• Assess cost-effectiveness
• Monitor student progress
• Change systems
• Plan programs

15
Data Can Be Used To

• Identify needs
• Describe problems
• Discover patterns
• Target interventions
• Identify best practices
• Make decisions

16
What Data Do We Use to Measure Results?

• Student Achievement Data
• Achievement-Related Psychosocial Data
• Career Development Data
• Student Diversity Data
• School Data

17
Student Achievement Data

• Norm-Referenced Standardized Tests
• Criterion-Referenced Standardized Tests
• Performance Tests
• Portfolios, Exhibitions, and other proof of
  proficiency
• GPA, course completion
• Drop-out rate

18
Achievement-RelatedPsychosocial Data

• Attendance rates
• Behavioral problems
• Discipline referrals
• Suspension rates
• Drug, Tobacco, and Alcohol use patterns
• Parent involvement
• Extracurricular activities
• Curriculum benchmarks

19
Career Development Data

• Post-Secondary Plans/College Placements
• Financial Aid Offers
• Vocational Placements
• Curriculum benchmarks

20
Student Diversity Data

• Race
• Gender
• Limited English Proficient
• English Language Learners
• Free or Reduced School Lunch
• Mobility
• Special Needs
• Achievement Quartile
• Grade

21
School Data

• Course Taking
• Gate courses 8th Grade Algebra, Algebra 2
• Honors and Advanced Placement Courses
• Special Programs
• Special Education
• Bilingual Education Services
• Academic Support Programs
• School Culture and Climate
• Policies
• Expectations
• Equity
• Safety
• Frequent Evaluations

22
School Data

• Curriculum
• Rigorous, Meaningful and Inclusive
• College Prep (with supports)
• Instruction
• Effective Instructional Practices
• Effective Use of Homework
• Student and Parent Connection and Engagement
• Teacher Knowledge and Expectations

23
The RI School Counseling Data Management System
(SCDMS)

• The amount of data that CAN be gathered and
  looked at is overwhelming. In order to be
  effective, each person has to choose the most
  important data points according to
• Your school
• Your district
• The state
• The field of School Counseling

24
The RI School Counseling Data Management System

• Toolkit 3 provides a starting point for
  gathering and thinking about data.
• First question What is crucial for us to know
  right now?
• In small groups, decide on a crucial question
  about SC that needs asking in your building.

25
The RI SCDMS Template

• One important part of the SCDMS is the data
  template. This allows each person to do a random
  sample measure of key student outcomes
• State test scores
• Attendance
• Discipline

26
(No Transcript)
27
(No Transcript)
28
Excel version of SCDMS
29
What Do We Mean by Random Sample?

• SC do not have the time to gather data on every
  student in their caseload! If you choose a
  random sample of your caseload, that means that
  you gather data on a specific number of randomly
  chosen students from each group.
• You can choose to gather data on every 5th
  student for example. If your caseload is 300
  students, thats 60 students.

30
What Do We Want to Know?

• At the state level, RISCA, RIDE and CSCOR
  identified a few priorities
• Question 1 How do counselor activities impact
  student outcomes?

31
How Do We Answer Our Question?

• We do a random sample measure of how you are
  spending your time, using the Daily Activity Log
  or the SC Time Distribution Log.
• We do a random sample measure of key student
  outcomes (state test scores, attendance,
  discipline), using the SCDMS template.
• We can answer Q1 How do counselor activities
  (by building) impact student outcomes?

32
What Do We Want to Know?

• Q2 How do counselor/student ratios impact
  student outcomes?

33
How Do We Answer Our Questions?

• We gather data on staffing levels and student
  enrollment.
• We use the random sample measure of key student
  outcomes (state test scores, attendance,
  discipline).
• We can answer Q2 How do student to counselor
  ratios impact student outcomes?

34
What Do We Want to Know?

• Q3 Does a more thoroughly implemented CSCP
  impact
• State test scores?
• Attendance?
• Discipline?

35
How Do We Answer Our Questions?

• We use the School Counseling Program
  Implementation Survey.
• We use the random sample measure of key student
  outcomes (state test scores, attendance,
  discipline).
• We can answer Q3 Does a more thoroughly
  implemented CSCP impact on State test scores,
  Attendance, and Discipline?

36
What Do We Want to Know?

• Q4 What are stakeholder perceptions of the CSCP?
• Students
• Teachers
• Administrators
• Parents

37
How Do We Answer Our Questions?

• We use the stakeholder surveys, for ES, MS and
  HS. Stakeholders can be teachers, students,
  administrators, and/or parents.
• We can answer Q4 What are stakeholder
  perceptions of the CSCP?

38
Summary

• Using process, perception and results data allows
  school counselors to find out how school
  counseling program components are impacting
  student learning.
• In todays educational climate, we need to
  constantly assess the effectiveness of programs
  and practices.
• Efforts that are not leading to higher academic
  outcomes for all students need to be
  re-evaluated.
• We can show that what we are doing makes a
  difference for all the students in our schools.

39
National Center for School Counseling Outcome
Research
http//www.umass.edu/schoolcounseling/

• Thank You