ARL New Measures

1
ARL New Measuresa View from Cornell

• Anne R. Kenney
• ARL Survey Coordinators and SPEC Liaisons
  Meeting
• ALA June 2007

2
Cornell and Assessment

• Participant in ARL Making Library Assessment Work
  project (August 2006)
• New Scope of Research and Assessment Unit (RAU)
  Fall 2006
• organizational effectiveness, use and usability,
  service innovation

3
Making Library Assessment Work Recommendations

• Examine organizational placement, resources,
  skill sets, and function of assessment
• Continue LibQUAL and develop customized survey(s)
• Gather more qualitative data from customers
• Put ARL statistics into perspective
• Develop a usability program
• Utilize measures of success, move from assessment
  to action
• Limit scope of data farm project

4
Research and Assessment Unit
Team Xin Li, Rich Entlich, Ellie Buckley, David
Banush, Linda Miller
5
RAU Charge 1 Report on Library Performance

• Prioritize efforts
• Assemble and interpret data
• Produce final reports

6
Charge 2 Manage Assessment Data
David Banush, Options for a CUL Data Mart (July
2007)
7
The unhappiness with data collection coupled with
a perceived need for even more data collection
may perhaps be traced to a very basic problem
an unclear or perhaps non-existent rationale for
gathering the data Several interview subjects
expressed significant doubts about data-driven
decision models generally. Some maintain that
data are not used by the CUL administration in
decision making and asked why more information
would have any added value. David Banush
8
Charge 3 Conduct Hi-Priority Projects

• intelligence unit
• Environmental scans
• Patterns and trends
• consultant
• Design assessment instrument
• Administer assessment projects
• Analyze and report findings with recommendations/
  observations

9
ARL New Measures The Cornell Perspective

• Reviewed ARL Board approved action items along
  with Yvonna Lincoln and Bruce Thompson reports
• Find the characteristics of the library of the
  future insightful
• Applaud ARLs shift from output-focused to
  outcome-based measures and collaboration
• Look to ARL for clear definitions and
  communication plan for the new measures

10
ARL New Measures The Cornell Perspective

• Assessed potential impact on CUL
• No immediate impact other than adjustments of
  data in some sub-categories
• If implemented, significant additional investment
  in data collection, requiring data outside
  librarys control
• Some data may not be practically attainable
• New measures may lead to incomplete picture,
  e.g., expenditure-focused index.
• New measures may help CUL embrace culture of
  assessment

11
New Measures Cost-Benefit Estimate
12
Next Steps

• Actively participate in ARL decision making
  process evaluate and help shape definitions.
• Discuss/determine in CUL executive committees
  what decisions to make for functional areas.
• Decide what success indicators are, what data
  captures the success levels, how to organize the
  data to reveal performance.
• Decide to what extent to invest resources for the
  new data collection (cost-benefit) and what
  activities to drop so resources can be
  re-directed.
• Connect the New Measurements initiative with ARL
  visiting program officers findings about CUL
• Be prepared to make tough choices, tying
  direction and resources to new measures.

13
Operationalize the to-dos

• Focus on local so to create an effective way to
  support ARL directions
• Examine the value of data for CUL decision making
  and prioritiesinternal and external data and
  measures
• Tie library measures more closely to university
  measures/goals/requirements
• Make cost-benefit choices
• Streamline existing data collection effort by 50
  or more within the next 12 months
• Identify gaps, estimate additional investment
  needed for new measures
• Pilot new approach with co-sponsored project

14
Make the so what Leap

• Resist the temptation of data collection based on
  it would be interesting, if we know
• Focus on outcome measures.
• Take a hard look at data collection with vague
  goals.
• Recognize value of good enough data.
• Run RAU like a business with customer-service
  focus.
• Have clear charge, authority, leadership, and
  goals
• Build staff skills
• Recognize that culture change and politics are
  hard.
• Dare to say, so what.
• Spend the money when your skills and timeline may
  compromise the result, e.g. persona.