A Faculty Perspective on Accountability

1
A Faculty Perspective on Accountability

• Presentation to the Meeting of Faculty
  LeadersIndiana Commission for Higher
  EducationIndianapolis, March 22, 2004
• John W. Curtis, Director of ResearchAmerican
  Association of University ProfessorsWashington,
  DC

2
Accountability to whom?

• Internal
• Peer review
• Shared governance
• External
• Accreditation (Institutional, disciplinary)
• State government
• Federal government
• US News, AAUP

3
Accountability for what?

• Quality (shift from resources?process?outcomes)
• Productivity (Efficiency)
• Tension between the two goals
• Third aspect Access

4
Accountability and Support

• State and federal governments are calling for
  more accountability at the same time that funding
  for higher education is decreasing

5
External Accountability Federal

• US Dept. of Education strategic plan 2002-07
• reducing disparities in college participation and
  completion between different groups
• strengthening institutional accountability
• primarily based on graduation rates
• GAO report (2003) the measure used by
  Education may not fully reflect an institutions
  performance because institutional goals and
  missions are not captured in the measure.

6
External Accountability Federal

• Current re-authorization of the Higher Education
  Act (HEA)
• penalizing excessive tuition cost increases
• mandating transfer of credit
• separating institutional eligibility for
  financial aid from accreditation
• Student Right-to-Know Act (1997?) campus crime,
  graduation rates

7
External Accountability Accreditation

• Regional accreditation is moving toward an
  exclusive focus on outcomes
• Weakening of standards
• Changing the process (not comprehensive, less
  on-site, more continuous reporting)
• Disciplinary accreditors may be upholding higher
  standards
• AAUP national committee reviewing standards and
  communicating with accreditors

8
External Accountability State

• GAO Report (2003) on College Completion
• 18 states publicize performance measures for
  higher education (usually public only)
• 9 states tie funding to performance measures
  (contingent funding, incentives)
• Frequently linked to broader state government
  initiatives

9
Case Study Oregon

• Oregon University System (OUS)
• 7 universities (not community colleges)
• performance indicators since 1997 (background of
  state agency benchmarks since 1989)
• mandate from legislature through Dept. of
  Administrative Services through OUS

10
Case Study Oregon

• Performance Measures
• Campus measures (12 common 2 individual)
• Goals Access, Quality, Employability of
  graduates, Institutional cost-effectiveness
• Indicators enrollment, persistence, graduation,
  faculty compensation, R D, philanthropy,
  graduate success
• Recognition of institutional mission in
  identifying measures

11
Case Study Oregon

• Agency Measures
• Summarized at system level
• 12 campus measures (2 individuals drop out!)
• 13 additional measures
• 25 total in 2003, down from 30 in 1997 (some
  dropped, some added)
• Linked to 90 Oregon Benchmarks (5 identified
  specifically, several more indirectly)

12
Case Study Oregon

• Reporting
• Elaborate Web site (campus, agency, FAQ,
  handbook)
• Annual reports to System Board
• Biennial report to legislature
• Limited link to funding
• Incentives 1 of total for 1999-2001 proposed 5
  of 2002-03 coming from existing allocation

13
Oregon Observations

• Enormous effort involving time and resources
  developing indicators, creating plans, collecting
  data, producing reports
• Target setting sustaining growth (low)
  turnaround high targets (stretch or
  accelerated)
• Use of peer groups for each institution

14
Oregon Observations

• Intricate web of interrelated efforts
• program review, strategic planning, budgeting,
  performance indicators, agency reports,
  accreditation
• department, college/division, campus, system,
  state, accreditor, federal
• Increased demands for accountability at a time of
  reduced financial support.
• Goals and objectives is it possible to improve
  all the time on every indicator?

15
Oregon Observations

• Where is the teaching and learning? Where are
  scholarship and research?
• Indicator of professional licensure was dropped.
• What is the purpose?
• 2003 governors advisory committee of business
  leaders reviewed the overall state program. One
  finding The purpose of Oregons performance
  measurement system should be clearer. Is the
  intent to focus on outputs or outcomes,
  effectiveness or efficiency, cost savings or
  continuous improvement?

16
Case Study Oregon

• Results in the 2003 report to the legislature
• Report on 11 indicators, trends over 5 years
• positive trends for all but two of the key
  indicators.
• competitive faculty compensation (4-13 below
  peers)
• Student/faculty ratio increased 10.7

17
Case Study Oregon

• Funding declines, combined with growing
  enrollments, have contributed to an increase in
  student-faculty ratios, particularly over the
  last three years. Taken together, higher faculty
  workload and non-competitive compensation are
  early warning signs that both access and quality
  are at risk.

18
Case Study Virginia

• State Council for Higher Education (16 public
  colleges and universities, 23 community colleges
  1)
• Reports of Institutional Effectiveness (ROIE)
• Proposed 1999, legislation 2000, first published
  2001
• 5 sections Institutional mission college
  profile system-wide measures institution-specifi
  c measures core competencies (learning outcomes)
• Recognition of institutional mission comparisons
  to be made to the institutions historic
  performance and to peers.

19
Case Study Virginia

• Reports of Institutional Effectiveness (ROIE)
• System-wide measures (14) retention, graduation
  size distribution of UG class sections, percent
  of lower-division enrollments taught by FT
  faculty living alumni who donate, classroom
  and laboratory space utilization
• Institution-specific measures an opportunity to
  sell the institution to the legislature and the
  general public.

20
Case Study Virginia

• Assessment of learning outcomes
• State mandate since 1987 reflected in SACS
  accreditation process
• Institution had flexibility
• Consequences administrative personnel,
  institutional prestige (limited), no funding
• In 2002 ROIE, boiled down to two indicators
  written communications and technology/information
  literacy.

21
Virginia Observations

• Process reflects weak central administration
  institutions can appeal directly to legislature
• Despite long history of outcomes assessment,
  learning only minimally incorporated into
  performance measures
• Fiscal crisis no link to funding (an unfunded
  mandate)
• ROIE seem to have disappeared after 2002

22
Case study Tennessee

• Tennessee Higher Education Commission (9
  universities, 2 institutes, 14 community
  colleges, 26 technology centers)
• Performance-based funding dates back to 1978
• 1984 Legislative benchmarks for higher
  education
• 1989 Tennessee Challenge 2000
• 1993 Reporting included independent (private)
  institutions
• Explicit links to Southern Regional Education
  Board (SREB) standards and SACS accreditation
  processes

23
Case study Tennessee

• Tennessee Performance funding
• Five-year cycles up to 5.45 incentives above
  annual formula appropriation
• Utilizes an elaborate point system to score
  institutions on performance standards annual
  reporting
• Much more prescriptive than other states
• Testing emphasis 30-35 of evaluation based on
  standardized tests another 10 on standard
  satisfaction surveys

24
States Overall Observations

• Inconsistency of measures and reporting over time
• Boil it down (quantifiable, summarized, data
  available)
• Frequently not reflective of teaching and
  learning research and scholarship institutional
  role in community culture
• Relatively short lifespan
• Processes and reports not well integrated
• Unfunded mandates requiring a lot of time and
  effort

25
Accountability A Faculty Perspective

• To faculty colleagues
• Trust us, we know what were doing is not
  sufficient.
• We must acknowledge the need to communicate more
  effectively our contributions to the larger
  community.

26
Accountability A Faculty Perspective

• To institutional leaders
• Defend your institutional mission
• Do not allow your products to be reduced to
  easily measurable standardized units
• Involve your faculty meaningfully in planning to
  meet accountability challenges.

27
Accountability A Faculty Perspective

• To policymakers
• Remember the broader purposes of higher
  education, and the many populations that the
  institutions serve
• Support the investment in higher education as a
  public good, and not just in terms of narrow
  cost/benefit calculations.
• Recognize that conflicting goals for higher
  education create a bad policy environment.

28
Implementing Accountability Measures

• Assessing student learning
• Most assessments are designed as a diagnostic to
  indicate areas for improvement. They do not
  indicate how to bring about the improvementthat
  is the role of faculty.

29
Implementing Accountability Measures

• Assessing student learning
• Assessment should not be punitive for students,
  individual faculty, programs, or institutions.
  This will lead to standardization, a lack of
  innovation, and a lowering of standards.
• A standardized test should not be the sole (or
  primary) basis for determining attainment of a
  general qualification (i.e., a degree). This
  contrasts with professional licensure, which
  refers to a specific narrowly-defined skill.

30
Implementing Accountability Measures

• Faculty should have primary responsibility for
  establishing criteria related to teaching and
  learning
• Measures imposed from above and evaluated on the
  basis of centrally-gathered data will not lead to
  real improvement
• Measures must reflect diverse institutional
  missions
• Leave room for qualitative descriptions of
  outcomes and process
• Evaluation process should be formative
  (encouraging ongoing improvement) and not
  punitive (seeking to blame)

31
Trends in higher education

• There is a general trend toward
• Bottom-line measures of output
• Corporate management
• A view of education as a commodity, an investment
  purchased in expectation of a private benefit

32
Trends in higher education

• and away from
• General education for informed citizenship
• Shared governance of educational institutions
• Colleges and universities existing to enhance the
  common good
• To the extent that accountability measures have
  the effect of accelerating these trends, they are
  undermining the role of higher education in our
  democratic society.

33
Thank you

• John W. Curtis, Director of ResearchAmerican
  Association of University ProfessorsWashington,
  DC
• [phone]jcurtis_at_aaup.org