ARCC Staff

1
ARCC Staff

• Willard Hom, Dean/Director
• Catharine Liddicoat, Specialist
• Alice van Ommeren, Specialist
• LeAnn Fong-Batkin, Coordinator

2
ARCC Background

• For background and previous ARCC reports, consult
  the ARCC website
• http//www.cccco.edu/SystemOffice/Divisions/
  TechResearchInfo/ResearchandPlanning/ARCC/tabid/29
  2/Default.aspx

3
Objectives

• Participants will
• Review 2009 ARCC content
• Review changes to the drafts (Oct. Jan.)
• Review peer grouping strategy
• Review self-assessment requirements
• Review ARCC timelines
• Learn about ARCC resources

4
ARCC Content

• Two levels of analysis
• Systemwide
• College Indicators
• SPAR, At least 30 units, and Persistence
• Vocational Course Completion
• ESL and Basic Skills Course Improvement
• Career Development and College Preparation
  Student Progress Achievement Rate
• Demographics

5
Changes to October 2008 ARCC Draft

• Dropped earliest cohort, added most recent cohort
• College demographics are now pulled from the Data
  Mart
• Table 1.6 in the 2008 report was called Enhanced
  Noncredit and is now CDCP
• We added six Schools of Continuing Education as
  required by SB 361

6
January 2009 Draft

• Includes systemwide indicators (incl. updated
  wage data) and peer grouping
• No data changes can be made at this point
• Use this draft to write self-assessment
• Self-assessments are required for schools of
  continuing education this year

7
Why did my data change?

• New queries for all data
• Curriculum Reporting for the Community Colleges
  (CRCC) course recoding effort
• Master Course File (MCF) clean-up
• Student identifier (SB00) cleanup
• New methodology for National Student
  Clearinghouse match

8
Why did my data change?

• DR code added to SX04 Enrollment-Grade (part of
  the CRCC project)
• DR Drop after first census date
• MIS memo of 10/12/06
• Implemented in Summer 2007
• Possibly affects denominators for Vocational and
  Basic Skills Course Completion Rates could
  affect rates

9
Peer Grouping Methodology

• Identify potential uncontrollable factors
• Determine data availability and quality
• Run correlations with outcomes
• Develop hierarchical regression models
• Create peer groups with cluster analysis

10
Peer Group Toolbox

• Just like previous years, the 2009 peer group
  toolbox will be available upon request to
  arcc_at_cccco.edu.
• Access limited to college researchers
• Includes
• Dataset - outcomes, predictors, peer group
• SPSS syntax and dendrograms

11
Interpreting Peer Groups

• Use caution when using rankings within peer
  groups-things change.
• Use the outcome average of the group as a
  comparison.
• Use peer groups as a rough guide for evaluation.
• Reminderpeer groups are based on uncontrollable
  factors, not outcomes.

12
Peer Grouping Year Three

• Your peer groups for each metric may or may not
  have changed.
• Your placement in a peer group may have changed.
• Peer Group number (A1, C3, D4) may have
  changedthis has no implication.

13
How can I explain any major change in my peer
group to my audiences?

• Note that the systems major data improvement
  efforts (CRCC, MCF, Student Identifier) may have
  changed data in many situations enough to change
  the peer groups.
• Even if the names of colleges in a peer group
  have changed, the average level of performance
  for that peer group may not have changed.

14
Why did some peer groups change?

• Major data revisions occurred because of several
  data improvement efforts (CRCC, MCF, Student
  Identifier)
• New exogenous variables as predictors resulted in
  new peer grouping factors, models and results.
• Updated outcomes led to changes in the regression
  models.

15
What use are peer groups for an indicator that
you cant predict very well?

• We recognize that the statistical model behind
  some of the indicator peer groups is relatively
  weak, meaning that these results should carry
  less weight in local deliberations.
• Because low predictive quality is a continuum
  rather than a clear yes/no situation, it can be
  a problem to omit a peer grouping result with low
  predictive quality.

16
2009 Regression Models
17
Why must there be 6 peer groups?

• Fewer than six groups will tend to produce peer
  groups that contain some poorly matched colleges.
• More than six groups will tend to create more
  one-college groups or two-college
  groups---making it very hard to evaluate these
  colleges at all.
• In general, peer groups of small size increase
  the chance that we will unfairly depict a college
  as the worst.

18
How do we know that these new peer groups are
valid?

• The clustering algorithm used for peer grouping
  minimizes within-cluster variation.
• The analysis used dendrograms from the cluster
  analysis behind each indicator peer grouping.
• Because the peer grouping only tries to produce
  groups with basic similarity, the members within
  a peer group can differ to some degree.

19
The Self-Assessment

• Strive for informative text (avoid redundancy).
• Show commitment to self-evaluation.
• Primary evaluation college year-to-year
  performance.
• Self-assessments are required for schools of
  continuing education this year

20
Timelines as of Feb. 2009

• College self-assessment is due on Feb.27, 2009 by
  500 p.m.
• Board of Trustee Interaction for the 2008 report
  is due by March 16, 2009.
• 2009 ARCC Report released by Mar. 31, 2009
• Board of Trustee Interaction for the 2009 report
  is due by March 16, 2010.

21
Resources

• Your ARCC draft
• Appendix A List of Peer Groups
• Appendix B General Methodology
• Appendix C Regression Details
• Appendix D Peer Grouping Method.
• ARCC website
• FAQs
• Self-assessment guidelines

22
Resources

• Data on Demand WebsiteNew!
• Demo https//misweb.cccco.edu/dataondemand/
• Replaces ARCC Data Retrieval Website
• Does not contain data from Oct. 2008 draft
• Data currently available
• SPAR Persistence (2007, 2008, and 2009)
• Vocational Course Completion (2009)
• Basic Skills Course Completion (2009)
• Basic Skills Improvement (Math, English, and ESL)
  (2009)
• Obtain password from your CISO

23
Conclusion

• Supplementary material available on the ARCC
  website
• Explore and use the ARCC website
• ARCC e-mail distribution list to join,
• e-mail arcc_at_cccco.edu
• ARCC Website http//www.cccco.edu/SystemOffice/Di
  visions/TechResearchInfo/ResearchandPlanning/ARCC/
  tabid/292/Default.aspx