The Chancellor's Office Research Agenda

1
The Chancellor's Office Research Agenda

• RP/CISOA Conference
• April 2009

2
Presenters

• Willard Hom, Dean/Director
• Alice van Ommeren, Research and Planning Staff
• LeAnn Fong-Batkin, Research and Planning Staff

3
Objectives

• To inform researchers and IT staff about specific
  future research efforts in the Chancellors
  Office
• To gather comments related to the research agenda
• Preview the research agenda prior to public
  release

4
Background

• Board of Governors adopted the System Strategic
  Plan in 2006. http//strategicplan.cccco.edu/
• Research agenda addresses goal D3 of the
  Strategic Plan, Analytical Capacity for Measuring
  Success.
• Prior work on organizing research topics was in
  the 1990s.

5
Goal of the Research Agenda

• Research agenda helps prioritize projects.
• Research agenda promotes coverage of topics that
  concern the wide array of system stakeholders.
• Research agenda increases transparency.

6
Research Agendas

• A research agenda structures a prioritization
  process so that an organization can
  systematically weigh factors in its consideration
  of research projects and activities.
• Research agendas are used in a wide variety of
  disciplines

7
Creation of the Research Agenda

• Two-day meeting in October 2008
• Used an external facilitator

8
Participants

• Community college representatives, including
  chief executive officers, chief information
  systems officers, and researchers
• Academic Senate
• RP Group
• Department of Finance
• Legislative Analysts Office
• Cal-PASS
• Community College League of California
• Foundation for California Community Colleges
• Chancellors Office Staff

9
Process

• Discussed value of creating a research agenda
• Identified external opportunities and challenges
• Identified internal strengths and weaknesses
• Discussed potential research projects
• Developed criteria to evaluate each project
• Prioritized the projects

10
Value of Our Research Agenda

• Main ideas guiding our discussion
• Focus Resources via Prioritization
• Manage Expectations
• Provide Leadership

11
External Opportunities

• Culture of evidence
• Economy
• Changing role of researchers
• Change in technology
• Increased interest in CCs from external parties
• New leadership and new relationships
• Changing demographics
• New research opportunities
• as reported by participants

12
External Challenges

• Restrictions on access to data
• Data quality, data coverage, and research methods
• Research resources (expertise)
• Dynamic environment
• as reported by participants

13
Internal Strengths

• System alignment with certain entities
• Collaborative approach to designing agenda
• Availability of data elements
• Ability to communicate and teach how to analyze
  and use the data
• as reported by participants

14
Internal Weaknesses

• Leadership and staff turnover
• Capacity
• State and local silos
• Funding pressures
• Linking research to instruction
• Too many areas to research
• No consequences/incentives for bad data
• as reported by participants

15
Potential Research Projects Considered Strategy
A

• Strategy A College Awareness and Access
• Financial aid study (fees, role of fin. aid,
  practices)
• Access study (Improve SEARS, GIS, time series
  study)
• Program evaluations (EOPS, Financial Aid, EAP)
• Distance education delivery model analysis
• Classification Study
• Other
• Concurrent enrollment
• Noncredit to Credit Transition
• University of Phoenix phenomena

16
Potential Research Projects Considered Strategy
B

• Strategy B Student Success and Readiness
• Transfer study (disaggregate transfer population,
  explore/identify transfers)
• Analysis of course factors (distance ed, learning
  communities, scheduling)
• Course placement recommendation collection (tie
  to CB 21 revision)

17
Potential Research Projects Considered Strategy
C

• Strategy C Partnerships for Economic and
  Workforce Development
• Data integration (matching supply with demand)
• Employment outcomes (longitudinal study)
• Curriculum development (improve response time)
• Partnership academies
• CTE programs (measure costs and levels of success)

18
Potential Research Projects Considered Strategy
D

• Strategy D System Effectiveness
• ARCCintegrate equity/national peering/benchmarkin
  g
• Student learning objectives (impact on student
  success)
• Swirl study
• Study of professional development (what is being
  done and how)

19
Potential Research Projects Considered Strategy
E

• Strategy E Resource Development
• Fee Policy (revenue vs. affordability affect on
  student access and success 50 law, distribution
  of funds, SB 361)
• Cost index
• Program analysis (break even costs for certain
  programs)
• Develop inventory of existing and proposed
  studies

20
Potential Research Projects Considered Others

• Establish common guidelines for conducting
  studies
• Performance-based funding
• Expand research methods and reporting tools
• Super model for forecasting

21
Selection Criteria

• Is the project doable? Do we have the data?
• Topic addresses multiple goals in the Strategic
  Plan
• Will the studys output move the system forward?
• Will the study impact
• Political leadership
• Students
• Economy/workforce
• Can we use prior studies to leverage this study?
• How much new information will the study provide?

22
Prioritization Process

• We categorized the projects into
• Quick wins (high impact, short-term timeframe,
  0-12 months)
• Stars (high impact, long-term timeframe (12-24
  months)
• Building Blocks (low impact, short term
  timeframe, 0-12 months)
• Back Burners (Low impact, long-term timeframe,
  12-24 months)
• Then, each participant voted for their choice of
  projects

23
Voting Process

• Used the facilitated decision making method,
  also known as the 10-4 method.
• Each participant received 10 dots.
• The participant placed 4 dots on the project that
  has the highest priority for the individual the
  remaining 6 dots are placed elsewhere.

24
Definition of Research Projects

• Participants divided into three groups to discuss
  the following for the 9 projects that received
  the most votes
• Scope
• Objectives
• Benefits
• Action Steps

25
And the winners are

• Project 1 Course Section Factors
• Objectives
• Conduct program evaluations
• Create infrastructure
• Operationalize definitions and magnitude
• Project 2 Course Placement Recommendations
• Objectives
• Collect course placement recommendations and test
  scores

26
And the winners are

• Project 3 Inventory of Existing Studies
• Objective Create warehouse of existing studies,
  including program evaluations, financial aid,
  internal and external studies
• Project 4 GIS Data Analysis
• Objective
• Enrollment management
• Program and service planning
• Bond planning analysis

27
And the winners are

• Project 5 Employment Outcomes
• Objectives
• Classification of programs
• Expansion of outcome data (longer tracking)
• Project 6 Evaluate CTE Programs
• Objectives
• Start with evaluation of Nursing programs
• Establish methodology for determining cost and
  performance indicators

28
And the winners are

• Project 7 Integrate equity data into ARCC
• Objective
• Make colleges more aware of equity issues
• Project 8 Expansion of Student Attributes
  (SEARS Survey)
• Objective
• Conduct research using student attributes as
  related to success

29
And the winners are

• Project 9 Fees, Financial Aid, and
  Affordability
• Objective
• Evaluate other states fees, revenue, financial
  aid, and participation
• Information will inform state policy and budget
  discussions
• Optimize student access and success

30
Caveats and Concerns

• Agenda is system level research
• Need literature reviews
• Did not identify projects to build research
  capacity
• Need to support classroom level research
• Projects require technical assistance
• Message and marketing of studies is important
• Rigor and utility analysis in design

31
Retreat Evaluation

• What went well
• Representation of a wide variety of CC and
  government organizations
• Process and facilitation
• Organization of event
• Changes for next time
• Need small college representation
• Need more rigorous guidelines and input about
  research units priorities
• Need a mechanism to connect with external
  research groups

32
Applications

• Allows us to make decisions about how the
  Chancellors Office will allocate its scarce
  resources
• Achieve additional efficiency in research-related
  activities
• Help external stakeholders identify projects to
  pursue
• Clarification of unmet research needs

33
Limitations of the Agenda

• Narrow scope
• Studies can be done with limited costs
• Omits studies that require
• Collection of new data
• Use of field experiments
• Large-scale studies
• Extensive literature review not conducted may
  have missed existing studies
• Does not state how research capacity can be
  expanded

34
Future Action

• We will attempt to do studies as time/resources
  permit
• We will work with external researchers to see if
  they can partner with us to perform the study
• We will periodically revisit the agenda
• Specific features of research agenda need
  refinement

35
Questions?

• Contact Willard Hom at [phone] or
  whom_at_cccco.edu