Title: Preparing to Work Effectively with LibQUAL
1Preparing to Work Effectively withLibQUAL
Survey Results
Raynna Bowlby Martha Kyrillidou Association of
Research Libraries
ALA Annual Conference Washington, DC June 26, 2007
old.libqual.org
2Preparing to Work Effectively with LibQUAL
Survey Results
- Prepared
- properly expectant, organized, or equipped
ready - Ready
- completely prepared or in condition for
immediate action or use or progress - Dictionary.com Unabridged (v 1.1)
3Desired OutcomesAs a result of our work together
today, you will be prepared to
- Perform some simple analyses of the LibQUAL
survey results data - Today well be working mainly with the 22 core
LibQUAL questions and Comments - Organize your library colleagues and groups /
teams to work with LibQUAL results - Present the results to different stakeholders
- Utilize the data to target areas for improvement
- Begin a process of continuous assessment
- Thank you for helping us beta test this
workshop we welcome your feedback!
4After the LibQUAL Survey -- Work Plan
- Understanding the Data and Results
- Interpreting representativeness, zone of
tolerance, peer comparisons, longitudinal
analysis, analytics and SPSS - Developing the Organization
- Confirming survey objectives, clarifying roles
responsibilities, presenting results, engaging
others in utilizing data, discussing results
implications - Using LibQUAL Effectively
- Targeting incremental improvements, applying data
in decision-making, implementing a cycle of
continuous assessment and improvement
5After the LibQUAL Survey -- Work Plan
- A list of steps describing work that can be
accomplished after receiving the survey,
including quantitative and qualitative data
analysis and organizational development - Think about
- Which steps might your library undertake?
- Will the assignment be individual or a group /
team? - What KSAs might best support the accomplishment
of this work? - Who will be assigned?
- (Who will you consult with to make these
decisions)?
6Understanding the Data Results
7Results Notebooks
- Sections for Overall, Undergraduates, Graduates,
Faculty, Staff, Library Staff include - Demographic Summary
- Core Questions Summary
- Dimensions Summary
- Local Questions
- General Satisfaction Questions
- Information Literacy Outcomes Questions
- Library Use Summary
- Appendix describing changes in the dimensions and
the questions included in each dimension.
8Results
- Results Notebooks (PDF)
- http//old.libqual.org/Manage/Results/index.cfm
- Group Notebook (PDF)
- Unified Notebook (PDF)
- User Comments
- Excel Data File
- SPSS data files (if requested at registration)
will be e-mailed to the primary contact two to
three months after the session has closed.
9DO YOU UNDERSTAND YOUR RESULTS?
10Representativeness - Standard Disciplines An
Academic Library
Respondents Population
11Representativeness - Customized Disciplines An
Academic Library
Respondents Population
12Radar Chart Basics
13Key to Radar Charts
14Understanding a Radar Chart
15What Do the Colors Mean?
Green And Blue
16What Do the Colors Mean?
Little Yellow
17What Do the Colors Mean?
A lot of Yellow
18What Do the Colors Mean?
Red
19Dimension Summary
20Understanding Your Individual Results
Thermometer Charts
Key Term Zone of Tolerance
21Dimension Summary
Key Term Zone of Tolerance
22Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
23Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
24Peer Comparisons
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
25Longitudinal Analysis
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
26NORMS
- Identify your score
- Compare it to a relevant norms table
- Year
- Subgroup
- Dimension
- Norms are stable and are not calculated on an
annual basis anymore (see article)
27ANALYTICS
- Institution Explorer
- User group and Discipline analysis
- Representativeness graphs
- Radar chart
- Library Use chart
- Thermometer chart
- Cumulative percentile distribution
- Longitudinal
28SPSS
- What is SPSS?
- Statistical Package for the Social Sciences
- How do I know that my data are trustworthy?
- Reliability and Validity Analysis
- Are there differences between different groups?
- Compare mean scores for subgroups
29Quantitative Analysis Excel Data
- Excel data files available on the LibQUAL Web
site - http//old.libqual.org/Manage/Results/index.cfm
- Use customized radar chart template to create
custom analyses - http//old.libqual.org/Manage/Resources/SampleRada
rChart/index.cfm
30Qualitative Analysis User Comments
- About one-half of users include comments on their
surveys - User Comments available on the LibQUAL Web site
- Download comments in Excel or text file
- Skim the comments
- Conduct ATLAS.ti analysis
31Identifying What is Actionable
- Conclusions and recommendations are grounded in
survey data - Identifying What is Actionable Worksheets
- Priorities, Weaknesses, Strengths
32Developing the Organization
33Confirming Survey Objectives/Desired Outcomes
- Conducting LibQUAL is not an end in itself
- Articulate WHY the library wants feedback from
users be specific - Indicate potential targets of interest, for
example - Customer input for library Strategic Planning
- Assessment for institutional accreditation
- Intention to become more user-centered
- Focus on specific institutional or library
strategic initiatives such as space/facilities
issues, collection(s) changes, web re/design,
customer service, etc.
34Identifying Clarifying Roles Responsibilities
- Develop an understanding of the work that your
library may want to undertake after the LibQUAL
survey (see the Work Plan) - Decide what work your library will/will not do
- Determine if your library will assign the
responsibility to a single individual or will
assemble a group / team - Identify the knowledge, skills, abilities (KSAs)
needed to successfully perform the desired work - Assign the work to match the KSAs and/or acquire
the needed KSAs
35Knowledge, Skills, Abilities (KSAs)
- Commitment to focus on user-centered outcomes
to apply user-centered thinking - Dedication to achieving continuous process
improvement - Ability to think act system-wide, to see the
big picture - Effective communication skills
- Comfortable w/statistics data
- Critical thinking / analytical skills
36Knowledge, Skills, Abilities (KSAs)Useful to
have or to develop (individually or collectively)
- Statistical methods quantitative data analysis
(e.g., use of SPSS) - Qualitative data analysis (e.g., Atlas-TI)
- Automate data collection tasks (e.g., write
scripts) - Design charts graphs (e.g., Excel)
- Writing (e.g., Word)
- Presentation (e.g., Powerpoint)
37Implementing a LibQUAL Group or Team
- Consider the purpose / scope of the groups work
- Broad, all assessment vs. specific, LibQUAL
only - On-going vs. situational?
- Task-oriented, e.g. administer, analyze?
- Facilitative, e.g. communicate, present, assist
others in understanding/utilizing data/results? - Leadership-focused, e.g. synthesize, recommend,
direct others to implement changes?
38Components of the Chargefor a Group/Team
- Purpose and/or Scope
- Products / Deliverables
- Assumptions and Parameters
- the extent of the groups autonomy / authority
- approval process(es)
- w/whom and how often the group will communicate
w/colleagues - suggested approaches for methods of operation and
how the group will function - available resources (budget, people, etc.)
- Timeline
- Member KSAs
39Assessment Group Resources
- Coming soon ARL SPEC Kit on Assessment
- Stephanie Wright (U. Washington) and Lynda White
(U. Virginia) - Boston College http//www.bc.edu/libraries/about/
assessment/ - Ohio State University http//library.osu.edu/site
s/staff/assess/ - U. Colorado (Boulder) http//ucblibraries.colorad
o.edu/internal/assessment/index.htm - U. Connecticut http//www.lib.uconn.edu/about/adm
inistration/surveys/ - U. Georgia http//dataserv.libs.uga.edu/assessmen
t/index.html - U. Kansas http//www.informationservices.ku.edu/a
ssessment/charge.shtml - U. Maryland http//www.lib.umd.edu/PASD/MIS/larc/
- U. Mass http//www.library.umass.edu/assessment/i
ndex.html - U. Washington http//www.lib.washington.edu/asses
sment/ - Washington State U. http//www.wsulibs.wsu.edu/ge
neral/WG/AWG.html
40Presenting Results to Stakeholders
- Identify all of the stakeholders or constituents
who want and need to know about the survey
results - Consider the stake of each of the above what
specific aspect of LibQUAL will be of most
interest / concern - Determine how to communicate with each identified
stakeholder
41Brown University Library Stakeholder
Accountability Matrix http//www.arl.org/arldocs/s
tats/statsevents/laconf/2006/BowbyOMahony.ppt
42Assumption College LibQUAL Marketing
Planold.libqual.org/documents/admin/ThistleLibQU
ALIntroSeattle.ppt
43Communicate with your Customers(faculty,
students, other)
- Particularly those whom you asked to participate
in the survey - Announce incentive award winners
- Inform users of highlights of survey results
- More importantly, what the library intends to do
- You asked for it
44Engaging Others in Understanding Utilizing
Data Results
- Spread knowledge about LibQUAL
- Develop a culture of accountability for listening
to customers and acting on user feedback
45Developing a Culture of AssessmentToolkit
http//personal.anderson.ucla.edu/amos.lakos/CUtoo
lkit.html
46Spreading Knowledge about LibQUAL throughout
the Library Staff
- Post Report on library web
- Make staff aware of LQ tutorial
- http//old.libqual.org/Information/Tools/index.cfm
- All-staff presentation
- Disseminate Comments to depts, units, groups,
branches, librarians, etc. - Work w/department leaders, managers
- Identifying Library Departments with Contribution
or Impact on LibQUAL Questions - Other?
47Corroborating LibQUALResults w/Other Data
- Consider if your LibQUAL data confirms your
prior expectations and results from other
assessment and measurement, and, vice versa - For example
- Undergrads have increasing needs desires re
Library as Place and show increases in library
gate counts/attendance - Faculty grad students have access from home or
office as highest priority and library web logs
demonstrate increasing use from campus depts.
48Discussing Results Implications
- Go beyond informing
- Lead efforts to analyze, synthesize, and
interpret findings
49Brown University LibraryUsing the Mini-Survey
to Explain LQ to Stakeholders
50The Library SummitClemson University U. of
Texas at Austinhttp//old.libqual.org/documents/a
dmin/LibQUAL_Summit.pdf
51Why Hold a Library Summit?http//www.lib.utexas.e
du/vprovost/assessment/libsummit/pdf/info_pkt.pdf
52Identify 2 or 3 LibQUAL questions for which
you/your department has primary responsibility
Being User-CenteredGuiding Library Staff in
Understanding Using LibQUAL Results
Do these cluster in a specific dimension?
53Overcoming Resistance BarriersChanging
Attitudes Behaviors
- We know whats best
- They are wrong
- We dont have the resources to
- ...only customers judge quality all other
judgments are essentially irrelevant. - Zeithaml, Parasuraman, Berry.(1999). Delivering
Quality Service. NY The Free Press
54Individual ExerciseAfter the LibQUAL Survey
-- Work Plan
- Review the list of steps describing work that can
be accomplished after receiving the survey,
including quantitative and qualitative data
analysis and organizational development - Which steps might your library undertake?
- Will the assignment be individual or a group /
team? - What KSAs might best support the accomplishment
of this work? - Who will be assigned?
- (Who will you consult with to make these
decisions)?
55Using LibQUAL Effectively
56Targeting Incremental Improvements
- From all of the data, determine what can and
should be addressed - Prioritize some action items
- Align with mission, vision and goals of parent
organization - Address users top priorities, by user group
- Improve areas of strong user dissatisfaction
- Build on strengths, if they are truly user needs
and priorities - Identify work that can be de-emphasized and
resources that can be reallocated
57S.M.A.R.T. Goals
- S Specific
- the desired outcome or result is clearly defined
- M Measurable
- accomplishment can be charted and/or observed
- A Attainable
- achievable, goal is challenging but realistic
- R Relevant
- results-oriented, in line with institutional
goals and library vision - T Timely
- deadlines are set for accomplishment
58SMART Goals -- Examples
- 75 of materials acquired from other libraries
are received by users within 10 days of request
by 12/07 - 50 of new books are on shelf within 5 days after
library receipt by 10/07 - 60 courses utilizing WebCT include links to
library research materials by 3/08 - Lower the unit cost of each service desk
transaction by 10 from FY08 to FY08 - Versus general goals
- Improve ILL turn-around time
- Make new materials accessible to users more
quickly - Decrease staffing budget
59(No Transcript)
60Applying Data in Decision-Making
- Assessment and LibQUAL data and results are
easily accessible and present when, where, and
with whom decisions are made in your library - Goals, priorities, and resource allocation are
influenced by customer needs and wants
61Archiving Data for Access
62Archiving Data for Access
63Cycle of Planning and Assessment
www.mindtools.com
64Identifying Using Best Practices
- Seeking out and learning from colleague
institutions - Participate in LibQUAL as part of a consortial,
regional, or interest-group implementation - Identify peer institutions who may agree to do
LibQUAL during the same cycle - Implementing improvements based upon best
practices
65Integrating LibQUAL w/ Other Assessments
- Confirming your conclusions
- Obtaining user specifications for initiatives
- Monitoring progress
- Some options
- Suggestions boxes
- Focus groups
- Exit surveys
66Planning Next StepsContinuous Assessment
Improvement
67In development .
- Customized Consulting Services
- Longitudinal data analysis, across multi years
- Tailored peer comparisons, libraries you choose
- Assistance w/ content analysis for Comments
- Facilitation of LibQUAL staff retreat or summit
- Guidance in preparing communication plan
- Customized training in assessment skills
- Launch an assessment group or team
- Assistance in identifying key customer issues for
future action
68Workshop Evaluation
martha_at_arl.org raynna.bowlby_at_charter.net
69LibQUAL Resources
- LibQUAL Web site
- http//old.libqual.org
- Publications
- http//old.libqual.org/publications
- Events and Training
- http//old.libqual.org/events
- Gap Theory/Radar Graph Introduction
- http//old.libqual.org/Information/Tools/libqualp
resentation.cfm - LibQUAL Procedures Manual
- http//old.libqual.org/Publications/index.cfm
70LibQUAL Team
- MaShana Davis
- Technical Communications Liaison
- mashana_at_arl.org
- Yolanda Glass
- Administrative Assistant
- yolanda_at_arl.org
- Kristina Justh
- Customer Relations Coordinator
- kristina_at_arl.org
- Martha Kyrillidou
- Director, Statistics and Service Quality
Programs - martha_at_arl.org
- Khyati Nayak
- Senior Applications Developer
- khyati_at_arl.org