ASSESSMENT OF CAC SELFSTUDY REPORT

1
ASSESSMENT OF CAC SELF-STUDY REPORT

• Curtis Cook
• Pankaj Matur
• Marcello Visconti

2
Introduction

• Computer Science Program Accreditation
• Change to Outcomes-Based Approach
• Good news , Bad news
• Accreditation Process
• Request by Program
• Self-Study Report - key role
• On site Visitation by Team
• Recommendation by Team
• Describe Tool that Helps Identify Deficiencies in
  Self- Study Report

3
Outcomes-Based Approach

• Change from Checklist Approach
• Outcomes-Based Approach Requires
• Specify outcomes (skills, knowledge, behavior)
• Assessment mechanism to determine if outcomes
  being achieved
• This change created anxiety
• Good News - Freedom to define objectives
• Bad News - Must demonstrate objectives being met

4
Self-Study Report

• Plays important role in accreditation process
• Must address 7 Criteria and list of Standards for
  each criterion
• Objectives and Assessments - new
• Student Support
• Faculty
• Curriculum
• Laboratories and Computer Facilities
• Institutional Support and Financial Resources
• Institutional Facilities

5
Self-Study Report (Cont)

• Assess thoroughness and completeness
• For assessment need 2 things
• Ideal practices (from model organization or
  accepted best practices)
• Mechanism to do comparison
• Meta Model Framework provides a means for
  obtaining both
• Set of Practices
• Assessment questionnaire

6
Ideal Practices

• CAC
• No model organization
• Ideal product is defined by CAC Criteria and
  Standards
• Use Meta Model Framework to
• Develop model and set of practices from CAC
  Criteria and Standards
• Yields 1 or more practices per standard

7
Example

• Criteria and Standards
• I-5. The results of the programs periodic
  assessments must be used to help identify
  opportunities for program improvement.
• Practices
• f. Evaluate assessment data for program
  objectives, program outcomes and/or course
  learning objectives and generate recommended
  changes for program improvement.
• g. Process or mechanism to select set of
  recommended changes and implement these changes
  to program objectives, program outcomes and/or
  course learning objectives.

8
Questionnaire

• Developed from set of practices
• Around 100 questions
• 1 or more questions for each practice
• Completed by person or persons most directly
  involved in preparation of Self-Study Report
• URL given in paper

9
Example Questions

• Practice d. Data relative to the program
  objectives is collected and documented on a
  regular basis
• Questions
• 7. What methods are used to collect data that
  may be used to measure degree of student
  achievement for each program outcome (Circle all
  that apply)
• Senior exit survey Senior exit interview
• Alumni survey Employee survey
• Local written exam National written exam
• Oral exam Industrial advisory panel
• Capstone course(s) other
• Dont Know

10
Example Questions (cont)

• Practice
• d. Data relative to the program objectives is
  collected and documented on a regular basis
• Questions
• 8. How often is this data in Question 7
  collected?
• Every year Every 2 years Seldom Never Dont
  Know
• 9. How often are program objectives assessed?
• Every year Every 2 years Seldom Never Dont
  Know

11
Questionnaire Scoring

• Main purpose of assessment is to identify
  deficient or possibly deficient standards
• Scoring Questions -gt Practices -gt Standards
• Each question graded as
• Satisfied
• Not Satisfied
• Possibly Satisfied
• Cannot Decide (No response or Dont Know)

12
Scoring

• Each practice graded as
• Satisfied - all questions satisfied
• Not Satisfied - one or more questions not
  satisfied
• Cannot Decide - all questions cannot decide
• Possibly Satisfied - otherwise
• Each Standard graded using scores of practices
  associated with it.

13
Assessment Report

• Goal of assessment is to identify deficiencies in
  Self-Study Report
• Assessment Report lists Standards not satisfied,
  or possibly not satisfied, or cannot decide
• Note Can also get score for each Criterion

14
Validations

• 2 pilots
• Recently accredited programs
• Minor deficiencies in Program 1
• Oversized upper division classes
• Major deficiency in Program 2
• Standard I-5 about using results of programs
  periodic assessments

15
Conclusion

• Described tool that aids in finding deficiencies
  in Self-Study Report
• Goal is to have a on-line tool that can be
  invoked and responds with assessment report