Assessing Library Performance:

1
Assessing Library Performance

• New Measures, Methods, and Models

24th IATUL Conference 2-5 June 2003 Ankara,
Turkey Julia C. Blixrud, ARL Assistant Executive
Director, External Relations
2
Measures that Matter

• Input Output Outcome Impact
• Consistent with organizational mission, goals and
  objectives
• Integration with program review
• Balance customer, stakeholder, and employee
  interests and needs
• Establish accountability
• Collection and use of reliable and valid data

3
Forces and Challenges

• Increasing demand for libraries to demonstrate
  outcomes/impacts in areas of importance to
  institution
• Increasing pressure to maximize use of resources
  through benchmarking resulting in
• Cost savings
• Reallocation

4
ARL New Measures Projects

• Demonstration project for service effectiveness
  measures (LibQUAL)
• Project to define usage measures for electronic
  information resources (includes institutional
  outcomes)
• Identification of measures that demonstrate a
  librarys contribution to student learning
  outcomes
• Investigation of role libraries play in support
  of the research process
• Identification of cost-drivers and development of
  cost-benefit studies (ILL/DocDel)

5
User Surveys

• Obtain direct responses to a series of questions
  from the community surveyed
• Identify user issues, concerns and needs
• Measure library performance from the user
  perspective, including satisfaction
• Acquire quantifiable data that can be
  statistically analyzed and generalizeable for the
  larger population
• Improve or change services
• Increase library visibility and marketing
• Contribute to broader institutional
  assessment/accreditation

6
LibQUAL Project Goals

• Establishment of a library service quality
  assessment program at ARL
• Development of web-based tools for assessing
  library service quality
• Development of mechanisms and protocols for
  evaluating libraries
• Identification of best practices in providing
  library service

7
Survey Instrument

• 3 scales
• Minimum
• Desired
• Perceived

8
Dimensions of Library Service Quality
9
(No Transcript)
10
(No Transcript)
11
Percentages of Acquisitions Dollars Devoted to
Electronic Resources
12
ARL E-Metrics Project

• Three phases
• Initial Phase (May-October 2000) What do we
  know? Inventory of current practices at ARL
  libraries as to statistics, measures, processes,
  and activities that pertain to networked
  resources and services.
• Second Phase (November 2000-June 2001) What can
  we collect? Identified and field tested an
  initial draft set of statistics and measures
• Final phase (July 2001-December 2001) What
  difference does this make? Build linkages to
  educational outcomes/impact, research, technical
  infrastructure

13
Recommended Statistics Measures

• Patron Accessible Electronic Resources (R1-3)
• Use of Networked Resources Services (U1-5)
• Expenditures for Networked Resources Related
  Infrastructure (C1-3)
• Library Digitization Activities (D1-3)
• Performance Measures (P1-3)

14
Characteristics of Each Recommended Measure

• Definition
• Rationale
• Unit of Measure
• Data source
• Frequency
• Process
• Related Issues

15
Focus on Electronic Resources

• Project COUNTER
• ICOLC
• Measuring Impact (Franklin/Plum)

16
Cost-Effectiveness

• ILL/DD study
• Technical Services Cost Study
• Activity Based Costing
• Balanced Scorecard
• Baldrige Quality Award

17
Outcomes

• Institutional
• Learning
• Research

18
Learning Outcomes

• Develop strategy to involve library in campus
  assessment activities to demonstrate the value of
  the library to the learning community
• Move from content view (books, subject knowledge)
  to competency view (what students are able to do)
• Understand learning outcomes of academic degree
  programs
• Develop curriculum segments through which the
  library achieves outcomes
• Information Literacy Competency Standards for
  Higher Education approved by the Association of
  College and Research Libraries in January 2000

19
Standardized Assessment of Information Literacy
Skills

• Base Based on outcomes as defined by ACRL
  information literacy competency standards d on
  ACRL standards
• Contains items not specific to a particular
  institution or library
• Is easily administered
• Has been proven valid and reliable
• Assesses at institutional level
• Provides for both external and internal
  benchmarking

20
Research Outcomes(Surrogates)

• RD expenditures
• PhDs awarded/year
• Post-doctoral appointments
• Research awards per year
• Academy memberships
• Faculty quality rankings

21
New Measure Challenges

• Resources (i.e., time and money)
• Buy-in (commitment to a culture of assessment
• Access to individuals to evaluate
• Expertise to conduct evaluation
• Project management experience
• Appropriate benchmarks
• Conceptual clarity
• Measurement design requirements
• Instrument validity and reliability

22

• Julia C. Blixrud
• Assistant Executive Director, External Relations
• Association of Research Libraries
• 21 Dupont Circle, Ste 800
• Washington, DC 20036
• jblix_at_arl.org
• 202-296-2296 ext. 133
• 202-872-0884 (fax)
• 202-251-4678 (cell)