State Higher Education Assessment Policies: Findings from Case Studies

1
State Higher Education Assessment Policies
Findings from Case Studies

• Thomas E. Perorazio
• John J.K. Cole

The National Center for Postsecondary Improvement
Project 5.1 The University of Michigan Associatio
n for Institutional Research 42nd Annual
Forum Toronto, Ontario, Canada
2
Session Goals

• Discuss Policy Process for Assessment
• Relate Important State Experiences
• Share Lessons Learned from Cases Relevant to
  Institutions

3
The National Center for Postsecondary Improvement
Main Project Web Site http//www.stanford.edu/gro
up/ncpi/
University of Michigan - Project
Five http//www.umich.edu/ncpi/
4
NCPI Project Area 5.1
State Government and Regional Accreditation
Association Policies of Assessment for Student
Learning

• Examine State Regional Policies
• What Policies Are in Existence?
• Impact on Institutional Assessment Practices
• Impact on Teaching Learning
• Utilize Policy Process Framework
• Analyze Relationships Among Levels

5
Overview of Project 5.1 Research

• Benchmarking Assessment
• Documented Existing Policies
• SHEAQ Survey
• Of SHEEO Administrators
• Literature Review
• Case Studies

6
Case Study Research
Purposes

• Examine State Policies in Detail
• Explore Critical Issues with State Officials
• Trace Evolution and Development
• Learn about Policys Impact on Institutions
• Infer Lessons about the Policy Process

7
States Selected for Study
(with Accreditation Region)

• New York (MSACS)
• South Carolina (SACS)
• Washington (NWASC)
• Missouri (NCA/HLC)
• Florida (SACS)

8
Policy Context

• Historical, Political, Social, Economic Factors
• Existing Policies for Accountability, Efficiency,
  etc.
• Political Climate in the State for Higher
  Education
• Previous Political Action RE Assessment
• Governance Structure for Higher Education
• Relations Communications between Government,
  SHEEO, Institutions
• Budgetary Financial Issues for Higher Education

9
Policy Process Framework
Five Stages

• 1.      Problem Formation
• Recognizing the Need for a State-level Assessment
  Policy
• 2.      Policy Formulation
• Development of Proposed Courses of Action
• 3.      Policy Adoption
• Development of Support for a Specific Proposal
• 4.      Policy Implementation
• Application of the Policy to the Problem
• 5.      Policy Evaluation
• Attempt to Determine the Policys Effectiveness

10
(No Transcript)
11
Problem Formation Policy Origination

• SHEEO Seeks to Focus Goals of Institutions on
  State Priorities
• Statewide Planning
• Task Forces
• Priorities of Quality, Effectiveness, Prestige,
  Efficiency
• Gubernatorial/Legislative Interest in Performance
• Desires Information on Results/Success
• Task Force Studies
• Data Generation Collection Measures
• Concern about Public Perceptions of
  Prestige/Quality

12
Problem Formation (2)

• Institutional Actions
• Initiatives to Engage in Assessment
• Procedures to Improve Program Quality
• Programs to Enhance Learning
• Public Opinion Critical for Political Will
• Change in Political Power

13
Policy Formulation
Options for Consideration

• Quality Assurance
• Institutional Data Generation/Collection
• Performance Indicators/Assessment Reports
• Make Information Publicly Available
• Increase Information to Policymakers
• Accountability
• State Planning Coordination
• Institutions Meet State Goals/Targets
• Centralized Approach to Data Analysis
• Results Tied to Budgetary Decisions

14
Policy Formulation (2)

• Institutional Improvement
• Management Effectiveness
• Link Measures to Accreditation Standards
• Institutional Self-Evaluation
• Differentiated by Sector
• Bring Improvement Through Quality
• Meet Both State and Institutional Goals

15
Policy Adoption
Four General Methods

• Legislative Action
• Mandate
• Authorize SHEEO Monitor
• Work with Institutions for Revision
• SHEEO Authority
• Originator of Policy
• Monitor, Collector, Distributor of Info
• Mediator between State, Institutions, Public

16
Policy Adoption (2)

• Task Forces/Blue Ribbon Committees
• Authority SHEEO or Legislature
• Business Leaders, Institutional Presidents
• Conduct Study/ Make Recommendations
• State/System Planning
• Process Produces Actionable Objectives
• Assessment In Service of Plan Goals

17
Policy Implementation--Mechanisms

• Reporting Institutional Statistics
• New York, Washington, Florida
• Performance Funding/Reporting
• South Carolina
• Missouri -- FFR
• Florida
• System Goals Institutional Improvement
• Missouri, Washington
• Accountability
• Florida, South Carolina, Missouri

18
Policy Implementation (2)

• Decentralized
• State Sets Broad Parameters for Performance
• Institutions Develop Effectiveness Plans
• Measurement Defined by Institutions
• Institutions Report Results up to State
• State Makes Decisions on Aggregated Data
• New York Washington

19
Policy Implementation (3)

• Centrally-Guided
• Prescribed State Institutional Goals
• Performance Standards Less Variable
• Central Data Collection Analysis
• Findings Utilized in Budget Decisions
• South Carolina Missouri

20
Policy Implementation (4)

• Combination Approach
• State Expectations Performance Guidelines
• Institutional Variability for Compliance
• Institutional Activity for Internal Improvement
• Florida

21
Data Collection
Emphasis on Data Data Systems

• Centralized Databases v. Institutional Data
• Levels of Aggregation
• Instruments/Testing
• Commonality Associated with Centralization
• Institutional Reporting
• To SHEEO, Legislature, Public
• Data is Reported Up the System

22
Data Usage for Decision Making

• Rewards-- For Meeting Targets
• MO SC
• Incentives-- To Achieve State Goals
• FL
• Public Knowledge--Consumer Information
• WA NY

23
Links to Teaching/Learning Improvements

• Making Assessment Institution-centered
• Public Accountability
• Institutions Share Data on Learning
• Revisiting Indicators Regularly
• Close Information Loop
• Focused Goals at Different Inst. Levels

24
Outcomes of Policy

• Institutional Resistance
• Disparate Effects
• Negative Improvement
• Excessive/Burdensome Requirements
• Indicators Not Useful for Management
• Institutional Cooperation
• Focus on Improvement
• Trust between SHEEO State
• Partnership to Develop System

25
Policy Evaluation

• Revisiting Adjustment
• MO SC
• Implementation
• NY WA
• Implementation Evaluation
• FL

26
Lessons Learned

• Assessment Must Be Incorporated Into
  Institutional Management
• Successful Policies Developed in Consultation
  with Institutions
• Culture of Institutions Can be Changed If The
  Process Contributes to Mgt

27
Lessons Learned (2)

• Institutions Must Also Be Willing to Form Working
  Relationships with State Officials
• Stakeholders at All Levels Must Be Engaged with
  Assessment
• Involving Too Many Stakeholders in Development
  Bogs Process Down

28
Lessons Learned (3)

• Sustained Commitment of Leadership Required
• Political Will for Success Required
• Policy Process Can Be As, If Not More Important
  Than Its Results

29
Successful Policies

• Have a Clear Focused Purpose
• Differentiate by Sector/Mission
• Emphasize Institutional Improvement
• Embrace Simpler, Rather than Complex,
  Indicator/Reporting Mechanisms
• Incorporate Priorities of Multiple Stakeholders
• Provide Useful Data for Decision Makers

30
Important Considerations

• Context for Assessment Shapes Process
• Policies Result In Improved Data Systems
• Process Forces Articulation of Principles
• New Policies May Not Replace Old Ones
• Policy Must Be Useful to Instituions