Preparing to Work Effectively with LibQUAL - PowerPoint PPT Presentation

1 / 70
About This Presentation
Title:

Preparing to Work Effectively with LibQUAL

Description:

completely prepared or in condition for immediate action ... Dictionary.com Unabridged (v 1.1) Desired Outcomes. As a result of our work together today, you ... – PowerPoint PPT presentation

Number of Views:66
Avg rating:3.0/5.0
Slides: 71
Provided by: mashan
Learn more at: https://www.libqual.org
Category:

less

Transcript and Presenter's Notes

Title: Preparing to Work Effectively with LibQUAL


1
Preparing to Work Effectively withLibQUAL
Survey Results
Raynna Bowlby Martha Kyrillidou Association of
Research Libraries
ALA Annual Conference Washington, DC June 26, 2007
old.libqual.org
2
Preparing to Work Effectively with LibQUAL
Survey Results
  • Prepared
  • properly expectant, organized, or equipped
    ready
  • Ready
  • completely prepared or in condition for
    immediate action or use or progress
  • Dictionary.com Unabridged (v 1.1)

3
Desired OutcomesAs a result of our work together
today, you will be prepared to
  • Perform some simple analyses of the LibQUAL
    survey results data
  • Today well be working mainly with the 22 core
    LibQUAL questions and Comments
  • Organize your library colleagues and groups /
    teams to work with LibQUAL results
  • Present the results to different stakeholders
  • Utilize the data to target areas for improvement
  • Begin a process of continuous assessment
  • Thank you for helping us beta test this
    workshop we welcome your feedback!

4
After the LibQUAL Survey -- Work Plan
  • Understanding the Data and Results
  • Interpreting representativeness, zone of
    tolerance, peer comparisons, longitudinal
    analysis, analytics and SPSS
  • Developing the Organization
  • Confirming survey objectives, clarifying roles
    responsibilities, presenting results, engaging
    others in utilizing data, discussing results
    implications
  • Using LibQUAL Effectively
  • Targeting incremental improvements, applying data
    in decision-making, implementing a cycle of
    continuous assessment and improvement

5
After the LibQUAL Survey -- Work Plan
  • A list of steps describing work that can be
    accomplished after receiving the survey,
    including quantitative and qualitative data
    analysis and organizational development
  • Think about
  • Which steps might your library undertake?
  • Will the assignment be individual or a group /
    team?
  • What KSAs might best support the accomplishment
    of this work?
  • Who will be assigned?
  • (Who will you consult with to make these
    decisions)?

6
Understanding the Data Results
7
Results Notebooks
  • Sections for Overall, Undergraduates, Graduates,
    Faculty, Staff, Library Staff include
  • Demographic Summary
  • Core Questions Summary
  • Dimensions Summary
  • Local Questions
  • General Satisfaction Questions
  • Information Literacy Outcomes Questions
  • Library Use Summary
  • Appendix describing changes in the dimensions and
    the questions included in each dimension.

8
Results
  • Results Notebooks (PDF)
  • http//old.libqual.org/Manage/Results/index.cfm
  • Group Notebook (PDF)
  • Unified Notebook (PDF)
  • User Comments
  • Excel Data File
  • SPSS data files (if requested at registration)
    will be e-mailed to the primary contact two to
    three months after the session has closed.

9
DO YOU UNDERSTAND YOUR RESULTS?
10
Representativeness - Standard Disciplines An
Academic Library
Respondents Population
11
Representativeness - Customized Disciplines An
Academic Library
Respondents Population
12
Radar Chart Basics
13
Key to Radar Charts
14
Understanding a Radar Chart
15
What Do the Colors Mean?
Green And Blue
16
What Do the Colors Mean?
Little Yellow
17
What Do the Colors Mean?
A lot of Yellow
18
What Do the Colors Mean?
Red
19
Dimension Summary
20
Understanding Your Individual Results
Thermometer Charts
Key Term Zone of Tolerance
21
Dimension Summary
Key Term Zone of Tolerance
22
Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
23
Peer Comparisons
Source Jim Self, University of Virginia,
Presented at Performance Measurement in Academic
Libraries Workshop, EBLIP4, Durham, North
Carolina, May 11, 2007
24
Peer Comparisons
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
25
Longitudinal Analysis
Source Fred Heath, LibQUAL Results Meeting,
ALA Annual Conference, Seattle, WA, January 22,
2007
26
NORMS
  • Identify your score
  • Compare it to a relevant norms table
  • Year
  • Subgroup
  • Dimension
  • Norms are stable and are not calculated on an
    annual basis anymore (see article)

27
ANALYTICS
  • Institution Explorer
  • User group and Discipline analysis
  • Representativeness graphs
  • Radar chart
  • Library Use chart
  • Thermometer chart
  • Cumulative percentile distribution
  • Longitudinal

28
SPSS
  • What is SPSS?
  • Statistical Package for the Social Sciences
  • How do I know that my data are trustworthy?
  • Reliability and Validity Analysis
  • Are there differences between different groups?
  • Compare mean scores for subgroups

29
Quantitative Analysis Excel Data
  • Excel data files available on the LibQUAL Web
    site
  • http//old.libqual.org/Manage/Results/index.cfm
  • Use customized radar chart template to create
    custom analyses
  • http//old.libqual.org/Manage/Resources/SampleRada
    rChart/index.cfm

30
Qualitative Analysis User Comments
  • About one-half of users include comments on their
    surveys
  • User Comments available on the LibQUAL Web site
  • Download comments in Excel or text file
  • Skim the comments
  • Conduct ATLAS.ti analysis

31
Identifying What is Actionable
  • Conclusions and recommendations are grounded in
    survey data
  • Identifying What is Actionable Worksheets
  • Priorities, Weaknesses, Strengths

32
Developing the Organization
33
Confirming Survey Objectives/Desired Outcomes
  • Conducting LibQUAL is not an end in itself
  • Articulate WHY the library wants feedback from
    users be specific
  • Indicate potential targets of interest, for
    example
  • Customer input for library Strategic Planning
  • Assessment for institutional accreditation
  • Intention to become more user-centered
  • Focus on specific institutional or library
    strategic initiatives such as space/facilities
    issues, collection(s) changes, web re/design,
    customer service, etc.

34
Identifying Clarifying Roles Responsibilities
  • Develop an understanding of the work that your
    library may want to undertake after the LibQUAL
    survey (see the Work Plan)
  • Decide what work your library will/will not do
  • Determine if your library will assign the
    responsibility to a single individual or will
    assemble a group / team
  • Identify the knowledge, skills, abilities (KSAs)
    needed to successfully perform the desired work
  • Assign the work to match the KSAs and/or acquire
    the needed KSAs

35
Knowledge, Skills, Abilities (KSAs)
  • Commitment to focus on user-centered outcomes
    to apply user-centered thinking
  • Dedication to achieving continuous process
    improvement
  • Ability to think act system-wide, to see the
    big picture
  • Effective communication skills
  • Comfortable w/statistics data
  • Critical thinking / analytical skills

36
Knowledge, Skills, Abilities (KSAs)Useful to
have or to develop (individually or collectively)
  • Statistical methods quantitative data analysis
    (e.g., use of SPSS)
  • Qualitative data analysis (e.g., Atlas-TI)
  • Automate data collection tasks (e.g., write
    scripts)
  • Design charts graphs (e.g., Excel)
  • Writing (e.g., Word)
  • Presentation (e.g., Powerpoint)

37
Implementing a LibQUAL Group or Team
  • Consider the purpose / scope of the groups work
  • Broad, all assessment vs. specific, LibQUAL
    only
  • On-going vs. situational?
  • Task-oriented, e.g. administer, analyze?
  • Facilitative, e.g. communicate, present, assist
    others in understanding/utilizing data/results?
  • Leadership-focused, e.g. synthesize, recommend,
    direct others to implement changes?

38
Components of the Chargefor a Group/Team
  • Purpose and/or Scope
  • Products / Deliverables
  • Assumptions and Parameters
  • the extent of the groups autonomy / authority
  • approval process(es)
  • w/whom and how often the group will communicate
    w/colleagues
  • suggested approaches for methods of operation and
    how the group will function
  • available resources (budget, people, etc.)
  • Timeline
  • Member KSAs

39
Assessment Group Resources
  • Coming soon ARL SPEC Kit on Assessment
  • Stephanie Wright (U. Washington) and Lynda White
    (U. Virginia)
  • Boston College http//www.bc.edu/libraries/about/
    assessment/
  • Ohio State University http//library.osu.edu/site
    s/staff/assess/
  • U. Colorado (Boulder) http//ucblibraries.colorad
    o.edu/internal/assessment/index.htm
  • U. Connecticut http//www.lib.uconn.edu/about/adm
    inistration/surveys/
  • U. Georgia http//dataserv.libs.uga.edu/assessmen
    t/index.html
  • U. Kansas http//www.informationservices.ku.edu/a
    ssessment/charge.shtml
  • U. Maryland http//www.lib.umd.edu/PASD/MIS/larc/
  • U. Mass http//www.library.umass.edu/assessment/i
    ndex.html
  • U. Washington http//www.lib.washington.edu/asses
    sment/
  • Washington State U. http//www.wsulibs.wsu.edu/ge
    neral/WG/AWG.html

40
Presenting Results to Stakeholders
  • Identify all of the stakeholders or constituents
    who want and need to know about the survey
    results
  • Consider the stake of each of the above what
    specific aspect of LibQUAL will be of most
    interest / concern
  • Determine how to communicate with each identified
    stakeholder

41
Brown University Library Stakeholder
Accountability Matrix http//www.arl.org/arldocs/s
tats/statsevents/laconf/2006/BowbyOMahony.ppt
42
Assumption College LibQUAL Marketing
Planold.libqual.org/documents/admin/ThistleLibQU
ALIntroSeattle.ppt
43
Communicate with your Customers(faculty,
students, other)
  • Particularly those whom you asked to participate
    in the survey
  • Announce incentive award winners
  • Inform users of highlights of survey results
  • More importantly, what the library intends to do
  • You asked for it

44
Engaging Others in Understanding Utilizing
Data Results
  • Spread knowledge about LibQUAL
  • Develop a culture of accountability for listening
    to customers and acting on user feedback

45
Developing a Culture of AssessmentToolkit
http//personal.anderson.ucla.edu/amos.lakos/CUtoo
lkit.html

46
Spreading Knowledge about LibQUAL throughout
the Library Staff
  • Post Report on library web
  • Make staff aware of LQ tutorial
  • http//old.libqual.org/Information/Tools/index.cfm
  • All-staff presentation
  • Disseminate Comments to depts, units, groups,
    branches, librarians, etc.
  • Work w/department leaders, managers
  • Identifying Library Departments with Contribution
    or Impact on LibQUAL Questions
  • Other?

47
Corroborating LibQUALResults w/Other Data
  • Consider if your LibQUAL data confirms your
    prior expectations and results from other
    assessment and measurement, and, vice versa
  • For example
  • Undergrads have increasing needs desires re
    Library as Place and show increases in library
    gate counts/attendance
  • Faculty grad students have access from home or
    office as highest priority and library web logs
    demonstrate increasing use from campus depts.

48
Discussing Results Implications
  • Go beyond informing
  • Lead efforts to analyze, synthesize, and
    interpret findings

49
Brown University LibraryUsing the Mini-Survey
to Explain LQ to Stakeholders
50
The Library SummitClemson University U. of
Texas at Austinhttp//old.libqual.org/documents/a
dmin/LibQUAL_Summit.pdf
51
Why Hold a Library Summit?http//www.lib.utexas.e
du/vprovost/assessment/libsummit/pdf/info_pkt.pdf
52
Identify 2 or 3 LibQUAL questions for which
you/your department has primary responsibility
Being User-CenteredGuiding Library Staff in
Understanding Using LibQUAL Results
Do these cluster in a specific dimension?
53
Overcoming Resistance BarriersChanging
Attitudes Behaviors
  • We know whats best
  • They are wrong
  • We dont have the resources to
  • ...only customers judge quality all other
    judgments are essentially irrelevant.
  • Zeithaml, Parasuraman, Berry.(1999). Delivering
    Quality Service. NY The Free Press

54
Individual ExerciseAfter the LibQUAL Survey
-- Work Plan
  • Review the list of steps describing work that can
    be accomplished after receiving the survey,
    including quantitative and qualitative data
    analysis and organizational development
  • Which steps might your library undertake?
  • Will the assignment be individual or a group /
    team?
  • What KSAs might best support the accomplishment
    of this work?
  • Who will be assigned?
  • (Who will you consult with to make these
    decisions)?

55
Using LibQUAL Effectively
56
Targeting Incremental Improvements
  • From all of the data, determine what can and
    should be addressed
  • Prioritize some action items
  • Align with mission, vision and goals of parent
    organization
  • Address users top priorities, by user group
  • Improve areas of strong user dissatisfaction
  • Build on strengths, if they are truly user needs
    and priorities
  • Identify work that can be de-emphasized and
    resources that can be reallocated

57
S.M.A.R.T. Goals
  • S Specific
  • the desired outcome or result is clearly defined
  • M Measurable
  • accomplishment can be charted and/or observed
  • A Attainable
  • achievable, goal is challenging but realistic
  • R Relevant
  • results-oriented, in line with institutional
    goals and library vision
  • T Timely
  • deadlines are set for accomplishment

58
SMART Goals -- Examples
  • 75 of materials acquired from other libraries
    are received by users within 10 days of request
    by 12/07
  • 50 of new books are on shelf within 5 days after
    library receipt by 10/07
  • 60 courses utilizing WebCT include links to
    library research materials by 3/08
  • Lower the unit cost of each service desk
    transaction by 10 from FY08 to FY08
  • Versus general goals
  • Improve ILL turn-around time
  • Make new materials accessible to users more
    quickly
  • Decrease staffing budget

59
(No Transcript)
60
Applying Data in Decision-Making
  • Assessment and LibQUAL data and results are
    easily accessible and present when, where, and
    with whom decisions are made in your library
  • Goals, priorities, and resource allocation are
    influenced by customer needs and wants

61
Archiving Data for Access
62
Archiving Data for Access
63
Cycle of Planning and Assessment
www.mindtools.com
64
Identifying Using Best Practices
  • Seeking out and learning from colleague
    institutions
  • Participate in LibQUAL as part of a consortial,
    regional, or interest-group implementation
  • Identify peer institutions who may agree to do
    LibQUAL during the same cycle
  • Implementing improvements based upon best
    practices

65
Integrating LibQUAL w/ Other Assessments
  • Confirming your conclusions
  • Obtaining user specifications for initiatives
  • Monitoring progress
  • Some options
  • Suggestions boxes
  • Focus groups
  • Exit surveys

66
Planning Next StepsContinuous Assessment
Improvement
67
In development .
  • Customized Consulting Services
  • Longitudinal data analysis, across multi years
  • Tailored peer comparisons, libraries you choose
  • Assistance w/ content analysis for Comments
  • Facilitation of LibQUAL staff retreat or summit
  • Guidance in preparing communication plan
  • Customized training in assessment skills
  • Launch an assessment group or team
  • Assistance in identifying key customer issues for
    future action

68
Workshop Evaluation
martha_at_arl.org raynna.bowlby_at_charter.net
69
LibQUAL Resources
  • LibQUAL Web site
  • http//old.libqual.org
  • Publications
  • http//old.libqual.org/publications
  • Events and Training
  • http//old.libqual.org/events
  • Gap Theory/Radar Graph Introduction
  • http//old.libqual.org/Information/Tools/libqualp
    resentation.cfm
  • LibQUAL Procedures Manual
  • http//old.libqual.org/Publications/index.cfm

70
LibQUAL Team
  • MaShana Davis
  • Technical Communications Liaison
  • mashana_at_arl.org
  • Yolanda Glass
  • Administrative Assistant
  • yolanda_at_arl.org
  • Kristina Justh
  • Customer Relations Coordinator
  • kristina_at_arl.org
  • Martha Kyrillidou
  • Director, Statistics and Service Quality
    Programs
  • martha_at_arl.org
  • Khyati Nayak
  • Senior Applications Developer
  • khyati_at_arl.org
Write a Comment
User Comments (0)
About PowerShow.com