Performance Management Presentation Provide Library Services

1
Performance Management PresentationProvide
Library Services

• Team Members
• Ted Crump, Rosalie Stroman, Jennifer Vrettos, and
  Susan Whitmore (Team Leader)
• ORS
• National Institutes of Health
• January 21, 2004

2
Table of Contents

• Main Presentation
• PM Template ....5
• Customer Perspective.....7
• Internal Business Process Perspective
  ..16
• Learning and Growth Perspective.
  .22
• Financial Perspective..25
• Conclusions and Recommendations..
  31

3
Table of Contents (cont.)

• Appendix
• Customer Perspective
• C1a Percent of current print journals in
  electronic format..38
• C1b Percentage of license agreements that cover
  all NIH....39
• C1c Percentage of document requests filled
  in-house...40
• C1d Number of items translated by government
  staff and contractors41
• C2a Number of web-based instruction
  modules..43
• C2b Training attendance by type
  44
• C2c Training evaluation scores..
  45
• C3a Customer Scorecard ratings for translation
  service and
• mediated database search service...
  ..48
• C3c Digital reference service evaluation
  ratings..54
• C4 Customer segmentation data...
  .56
• Internal Business Process Perspective
• IB1 Number and type of special projects
  contracted for....63
• IB2 Number of new IT systems implemented
  ..65
• IB3a Percent of document requests filled
  in-house in 3 business days
  ..69
• IB3b Percentage of documents translated by
  library staff

4
Table of Contents (cont.)

• Appendix
• Learning and Growth Perspective
• LG1a Percentage of staff members with IDPs
  completed by
• February 28 each fiscal year......
  .....74
• LG1b Percentage of staff completing 80 of
  activities listed in
• IDP.............................................
  ..................................................
  ........75
• LG1c Percentage of training modules completed
  on schedule76
• Financial Perspective
• F1 Change in unit cost of information.
  .79
• F2a Change in unit cost of custom
  services...82
• F2b No. of contracts/IAGs..
  82
• F3a Change in unit cost of providing copies of
  publications.82
• F3b No. of document requests
  filled.82
• F4a Change in unit cost of translating
  services..83
• F4b No. of documents translated.
  .83
• F5a Change in unit cost of providing primary
  library services..83
• F5b No. of units of information
  provided..83
• F6a Change in unit cost of providing
  self-service copiers....83

5
(No Transcript)
6
Relationship Among Performance Objectives
7
Customer Perspective
8
Customer Perspective
9
C1a Percent of current print journals in
electronic format

• Target Provide 100 of current journals in
  electronic format
• Eighty-one percent of the journals are now
  available in electronic format (as compared to
  75 in FY02 and 52 in FY01)

10
Customer Perspective (cont.)

11
C3a Customer Scorecard ratings for mediated
database searching service

• Mediated database searching service was evaluated
  from mid-October to mid-December of 2003 using
  paper or electronic survey forms
• Forty surveys were distributed and 14 were
  returned for a return rate of 35
• The surveys asked users of the service to rate
  customer service and search results on a 10 point
  scale with 1 being unsatisfactory and 10 being
  outstanding
• Customers were very satisfied with the service

12
C3a Customer Scorecard ratings for mediated
database searching service Customer Service
13
C3a Customer Scorecard ratings for mediated
database searching service Search Results
14
Customer Perspective (cont.)
15
C4 Customer segmentation data

• Customer segmentation data is presented in the
  Appendix for four services
• Provide copies of publications
• Provide self-service copiers
• Translate documents
• Provide custom research assistance
• As in past years, customers are spread across all
  ICs
• Heavier use of services by larger ICs and groups
  most directly engaged in patient care and
  biomedical research
• The Library serves not only NIH but also FDA/CBER
  on the Bethesda campus and provides document
  delivery to several groups including NIST and CDC.

16
Internal Business Perspective
17
Internal Business Perspective
18
IB2 Number of new IT systems implemented

• There were four areas of concentration in FY03
• HHS Library Consortium
• SFX or Full-Text Plus!
• Relais
• Content Management Software
• The major areas of concentration for FY04 are
• Continue with the development of the HHS
  Consortium, Full-Text Plus!, Relais, and CMS
• Implement MetaLib
• Enable wireless access in the Library

19
New IT systems implemented in FY03

• HHS Library Consortium In FY03 the Consortium
  jointly purchased the Nature journals, Science,
  and Science Express at a considerable savings to
  NIH
• Full-Text Plus! This software, which links
  various services and resources available from the
  Library, was implemented for the online journals,
  the Web of Science and Ovid databases, the new
  online document request form, and other resources
  in FY03
• Relais In FY03, phase II of the Relais system
  was implemented with the integration of the
  web-based request form and the Web of Science
  requests. Through Relais the processing of
  document requests has been considerably
  streamlined. A process that used to have eight
  steps now has only three with a turn around time
  of 1-3 business days
• Content Management Software has been implemented
  for the Librarys Intranet and public websites
  resulting in a diversification of responsibility
  for the sites. The redesigned public website will
  be unveiled on February 10, 2004

20
Internal Business Perspective (cont.)
21
IB3b Percentage of documents translated by
library staff within 3 business days

• Target Translate document requests in 3 business
  days
• Results In FY03 an average 55 of documents were
  translated in 3 business days. In FY04 the
  translation speed was measured for Library
  translators and contractors separately. In FY04
  Library translators translated 55 of documents
  in 3 days and contractors translated 26 of
  documents in 3 days.
• Efforts to improve the turn around time for
  translating documents are complicated by
• Translation requests are increasing and the
  current staff are already working as efficiently
  as possible
• The more translations that are contracted out,
  the more time the Library translation staff must
  spend checking contracted translations thereby
  reducing the time they have to translate

22

• Learning and Growth Perspective

23
Learning and Growth Perspective
24
LG1a Percentage of staff members with IDPs
completed by February 28 each fiscal year

• Staff are asked to develop Individual Development
  Plans (IDPs) in conjunction with their supervisor
  each year
• IDPs list areas for development and training for
  each staff member for the coming calendar year
• Target 33 of staff will have IDPs developed by
  February 28, 2003
• Results 39 of staff had IDPs by February 28,
  2003

25
Financial Perspective
26
Financial Perspective
27
F1 Change in Unit Cost of Information Number
of Units Used

• Use of information units has increased 35 in the
  past four years
• A unit of information may be a search, a document
  supplied, a customer trained, an article
  downloaded, or a visit to our web site
• The electronic resources are heavily used and
  their use increases each year

28
F1 Change in Unit Cost of Information Unit
Costs FY00FY03

• Target Maintain or reduce the cost of
  information provided to NIH staff
• The unit cost of information is calculated by
  dividing the cost of the information into the
  number of information units used
• The unit cost of information has remained fairly
  steady over the past 4 years
• E-resources are more expensive than print
  resources however, they are used more heavily
  thereby keeping the unit cost of information
  steady

29
F2-6a Change in Unit Cost

• The unit cost for provide copies of publications
  is lower in FY04. In FY03 276,000 copies of
  publications were supplied. As more documents are
  delivered as e-documents, costs are reduced.
• The unit cost for self-service copiers rose in
  FY04. In FY03 there were 24,000 visits to the
  self-service copiers. Use of copiers has
  decreased and hence the unit cost for the service
  has increased.

30
F2-6a Change in Unit Cost (cont.)

• The unit cost for translate documents has risen
  over the past 4 years. The costs are largely
  based on salaries and contracting costs. 755
  items were translated in FY04.
• The unit cost for provide primary library
  services has risen over the past 4 years. This
  service is growing. Costs are primarily based on
  salaries and purchase of print and e-resources.
  The cost of print and electronic resources rises
  10-20 each year, and the Library is increasing
  the amount spent on e-resources by 20 each year.
  There were 9.6 million units of information
  provided in FY04.
• The unit cost for provide custom research
  assistance has not changed since last year.
  Seventeen custom projects were undertaken in
  FY04.

31
Conclusions
32
Accomplishments for FY03

• Customers are very satisfied with the Librarys
  resources and services.
• The website has been redesigned in content
  management software, a new brand has been
  developed for the Library, and new print
  materials have been designed. These materials
  will be introduced to the NIH staff on February
  10, 2004.
• Full-Text Plus! software that will enable users
  to link easily from databases to journal articles
  to training and more has been implemented.
• Relais II has been implemented, which has
  streamlined the process used to fill document
  requests, and a web-based document request form
  is available to customers.
• The Unit Cost of Information remains steady as
  we add new electronic resources to the
  collection.

33
Initiatives for FY04

• We are in the process of developing a web-based
  survey, in conjunction with the NLM, to assess
  customer satisfaction with the informationist
  service.
• We will be conducting either the bi-annual user
  satisfaction telephone survey or a user needs
  assessment in FY04, and we will use the scorecard
  to gauge customer satisfaction with provide
  copies of publications and reference service
  provided at the information desk.
• The Parklawn Health Library will become a branch
  of the NIH Library.
• In FY04, MetaLib, software that will enable
  searching across many e-resources, will be
  implemented.
• The Web of Science software will be updated to
  the Web of Knowledge.
• Development of an in-house training curriculum to
  ensure all staff are knowledgeable about Library
  services and to retrain staff for new work will
  continue.
• To maintain the unit cost of information,
  resources and services will continue to be
  selected based on their usefulness to NIH staff
  and cost per use will be a major factor in the
  selection and retention of electronic resources.

34
Appendix

• Customer Perspective
• C1a Percent of current print journals in
  electronic format..38
• C1b Percentage of license agreements that cover
  all NIH....39
• C1c Percentage of document requests filled
  in-house...40
• C1d Number of items translated by government
  staff and contractors41
• C2a Number of web-based instruction
  modules..43
• C2b Training attendance by type
  44
• C2c Training evaluation scores..
  45
• C3a Customer Scorecard ratings for translation
  service and
• mediated database search service...
  ..48
• C3c Digital reference service evaluation
  ratings..54
• C4 Customer segmentation data...
  .56
• Internal Business Process Perspective
• IB1 Number and type of special projects
  contracted for....63
• IB2 Number of new IT systems implemented
  ..65
• IB3a Percent of document requests filled
  in-house in 3 business days
  ..69
• IB3b Percentage of documents translated by
  library staff
• within 3 business days......7
  0

35
Appendix (cont.)

• Learning and Growth Perspective
• LG1a Percentage of staff members with IDPs
  completed by
• February 28 each fiscal year......
  .....74
• LG1b Percentage of staff completing 80 of
  activities listed in
• IDP.............................................
  ..................................................
  ........75
• LG1c Percentage of training modules completed
  on schedule76
• Financial Perspective
• F1 Change in unit cost of information.
  .79
• F2a Change in unit cost of custom
  services...82
• F2b No. of contracts/IAGs..
  82
• F3a Change in unit cost of providing copies of
  publications.82
• F3b No. of document requests
  filled.82
• F4a Change in unit cost of translating
  services..83
• F4b No. of documents translated.
  .83
• F5a Change in unit cost of providing primary
  library services..83
• F5b No. of units of information
  provided..83
• F6a Change in unit cost of providing
  self-service copiers....83
• F6b No. of visits to self-service copy
  center...83

36
Customer Perspective
37
Customer Perspective
38
C1a Percent of current print journals in
electronic format

• Target Provide 100 of current journals in
  electronic format
• Eighty-one percent of the journals are now
  available in electronic format (as compared to
  75 in FY02 and 52 in FY01)

39
C1b Percentage of license agreements that cover
all NIH

• Target Negotiate all electronic resource license
  agreements for access by all of NIH
• Currently 62 of e-license agreements give all
  NIH staff access (as compared to 53 in FY02 and
  45 in FY01)

40
C1c Percentage of document requests filled
in-house

• Target Maintain percentage of document requests
  filled in-house or at NLM to keep costs down and
  turn around time fast
• Results Percentage filled in-house in FY03 was
  82 and in FY02 was 84
• The large number of requests that can be filled
  in-house reflects the careful tailoring of the
  collection to the needs of the NIH staff

41
C1d Number of items translated by government
staff and contractors

• The total number of items translated in FY03 was
  755 items
• The number of words translated has increased 44
  over the past 6 years
• To ensure items are translated in a timely manner
  increasingly, items are given to contractors to
  translate

42
Customer Perspective (cont.)
43
C2a Number of web-based instruction modules

• Target Develop web-based instruction modules for
  all training classes, if feasible
• We have found that realistically only about half
  of our classes can be made into web-based
  modules.
• Library staff currently are busy updating all
  modules with new NIH Library branding

44
C2b Training attendance by type

• Customers have clearly indicated their preference
  for web-based training which is available 24/7
• Nevertheless, attendance at classes remains
  steady
• Class attendance numbers are used to determine
  which classes to offer

45
C2c Training evaluation scores

• Classes are provided to teach NIH staff effective
  use of electronic resources and bibliographic
  management software
• In FY03, 75 of attendees (451/599) completed an
  evaluation form designed to assess satisfaction
  with the training classes
• Customers were asked to rate their satisfaction
  on a 5 point scale with 1 being poor and 5 being
  excellent
• The classroom was upgraded at the beginning of
  FY03 and the rating for the classroom rose from
  4.2 to 4.6

46
C2c Training evaluation scores
47
Customer Perspective (cont.)

• Following are the results of the survey of
  customers satisfaction with three services
• Translation Services
• Mediated database searching
• Digital reference service
• Evaluations were transaction based
• All services received high scores

48
C3a Customer Scorecard ratings for translation
service

• Evaluation surveys were distributed to customers
  who used the translation service in FY02 and FY03

Survey distribution FY02 FY03
No. of surveys distributed 88 390
No. of respondents 37 70
Response rate 42 18

• Customers were asked to rate satisfaction and
  importance of the service on a 10 point scale
  with 1 being unsatisfactory and 10 being
  outstanding
• Customers were highly satisfied with all aspects
  of the translation service. In addition,
  customers felt that timeliness was much more
  important than cost

49
C3a Customer Scorecard ratings for translation
service Satisfaction Ratings

• FY03 availability ratings are significantly
  higher (plt .05) than FY02 ratings

50
C3a Customer Scorecard ratings for translation
service Importance Ratings
51
C3a Customer Scorecard ratings for mediated
database searching service

• Mediated database searching service was evaluated
  from mid-October to mid-December of 2003 using
  paper or electronic survey forms
• Forty surveys were distributed and 14 were
  returned for a return rate of 35
• The surveys asked users of the service to rate
  customer service and search results on a 10 point
  scale with 1 being unsatisfactory and 10 being
  outstanding
• Customers were very satisfied with the service

52
C3a Customer Scorecard ratings for mediated
database searching service Customer Service
53
C3a Customer Scorecard ratings for mediated
database searching service Search Results
54
C3c Digital reference service evaluation ratings

• The Library has offered a digital reference
  service for the past two years
• Digital reference combines chat and the ability
  to push web pages to customers in response to a
  reference question
• Customers are given the option of filling out an
  evaluation form at the end of each transaction
• In FY02 37 of customers responded to the survey
  and in FY03 the response rate was 39
• Customers found the service easy to use and would
  use it again. Survey questions will be revised
  in FY04 to give a more rounded view of the
  customers satisfaction with the service.

55
C3c Digital reference service evaluation ratings
56
Customer Perspective (cont.)
57
C4 Customer segmentation data

• Customer segmentation data is presented for four
  services
• Provide copies of publications
• Provide self-service copiers
• Translate documents
• Provide custom research assistance
• As in past years, customers are spread across all
  ICs
• Heavier use of services by larger ICs and groups
  most directly engaged in patient care and
  biomedical research
• The Library serves not only NIH but also FDA/CBER
  on the Bethesda campus and provides document
  delivery to several groups including NIST and CDC.

58
C4 Customer Segmentation for Provide Copies of
Publications
59
C4 Customer Segmentation for Provide
Self-Service Copiers
60
C4 Customer Segmentation for Translate Documents
61
C4 Customer Segmentation for Provide Custom
Research Assistance
62
Internal Business Perspective
63
Internal Business Perspective
64
IB1 Number and type of special projects
contracted for
Custom Services FY02 FY03
Collection Related 6 7
Information Related
Bibliography, database, web page 7 5
Informationist assignments 3 5
Totals 16 17

• Target for FY04 is to hire more informationists
  and increase the number of informationist
  assignments to 8

65
Internal Business Perspective (cont.)
66
IB2 Number of new IT systems implemented

• There were four areas of concentration in FY03
• HHS Library Consortium
• SFX or Full-Text Plus!
• Relais
• Content Management Software
• The major areas of concentration for FY04 are
• Continue with the development of the HHS
  Consortium, Full-Text Plus!, Relais, and CMS
• Implement MetaLib
• Enable wireless access in the Library

67
New IT systems implemented in FY03

• HHS Library Consortium In FY03 the Consortium
  jointly purchased the Nature journals, Science,
  and Science Express at a considerable savings to
  NIH
• Full-Text Plus! This software, which links
  various services and resources available from the
  Library, was implemented for the online journals,
  the Web of Science and Ovid databases, the new
  online document request form, and other resources
  in FY03
• Relais In FY03, phase II of the Relais system
  was implemented with the integration of the
  web-based request form and the Web of Science
  requests. Through Relais the processing of
  document requests has been considerably
  streamlined. A process that used to have eight
  steps now has only three with a turn around time
  of 1-3 business days
• Content Management Software has been implemented
  for the Librarys Intranet and public websites
  resulting in a diversification of responsibility
  for the sites. The redesigned public website will
  be unveiled on February 10, 2004

68
Internal Business Perspective (cont.)
69
IB3a Percent of document requests filled
inhouse in 3 business days

• Target Fill 96 of requests in 3 business days
• Results Average fill rate for FY03 was 98 and
  for FY03 was 97

70
IB3b Percentage of documents translated by
library staff within 3 business days

• Target Translate document requests in 3 business
  days
• Results In FY03 an average 55 of documents were
  translated in 3 business days. In FY04 the
  translation speed was measured for Library
  translators and contractors separately. In FY04
  Library translators translated 55 of documents
  in 3 days and contractors translated 26 of
  documents in 3 days.
• Efforts to improve the turn around time for
  translating documents are complicated by
• Translation requests are increasing and the
  current staff are already working as efficiently
  as possible
• The more translations that are contracted out,
  the more time the Library translation staff must
  spend checking contracted translations thereby
  reducing the time they have to translate

71
IB3c Percent of tutorial requests responded to
within 1 business day
72
Learning and Growth Perspective
73
Learning and Growth Perspective
74
LG1a Percentage of staff members with IDPs
completed by February 28 each fiscal year

• Staff are asked to develop Individual Development
  Plans (IDPs) in conjunction with their supervisor
  each year
• IDPs list areas for development and training for
  each staff member for the coming calendar year
• Target 33 of staff will have IDPs developed by
  February 28, 2003
• Results 39 of staff had IDPs by February 28,
  2003

75
LG1b Percentage of staff completing 80 of
activities listed in IDP

• Target Eighty percent of staff will have
  completed 80 of IDP objectives by Dec. 30
• Result 88 of staff with plans in place in
  February completed 80 of their objectives

76
LG1c Percentage of training modules completed on
schedule

• Training Project Team established in FY03
• Projects for FY04 are
• Survey Library staff to assess the state of
  learning in the NIH Library
• Conduct needs assessment
• Develop training plan

77
Financial Perspective
78
Financial Perspective
79
F1 Change in Unit Cost of Information Number
of Units Used

• Use of information units has increased 35 in the
  past four years
• A unit of information may be a search, a document
  supplied, a customer trained, an article
  downloaded, or a visit to our web site
• The electronic resources are heavily used and
  their use increases each year

80
F1 Change in Unit Cost of Information Unit
Cost FY00FY03

• Target Maintain or reduce the cost of
  information provided to NIH staff
• The unit cost of information is calculated by
  dividing the cost of the information into the
  number of information units used
• The unit cost of information has remained fairly
  steady over the past 4 years
• E-resources are more expensive than print
  resources however, they are used more heavily
  thereby keeping the unit cost of information
  steady

81
F1 Calculations for the Unit Cost of
Information
82
F2-6a Change in Unit Cost

• The unit cost for provide copies of publications
  is lower in FY04. In FY03 276,000 copies of
  publications were supplied. As more documents are
  delivered as e-documents, costs are reduced.
• The unit cost for self-service copiers rose in
  FY04. In FY03 there were 24,000 visits to the
  self-service copiers. Use of copiers has
  decreased and hence the unit cost for the service
  has increased.

83
F2-6a Change in Unit Cost (cont.)

• The unit cost for translate documents has risen
  over the past 4 years. The costs are largely
  based on salaries and contracting costs. 755
  items were translated in FY04.
• The unit cost for provide primary library
  services has risen over the past 4 years. This
  service is growing. Costs are primarily based on
  salaries and purchase of print and e-resources.
  The cost of print and electronic resources rises
  10-20 each year, and the Library is increasing
  the amount spent on e-resources by 20 each year.
  There were 9.6 million units of information
  provided in FY04.
• The unit cost for provide custom research
  assistance has not changed since last year.
  Seventeen custom projects were undertaken in
  FY04.

84
Conclusions
85
Accomplishments for FY03

• Customers are very satisfied with the Librarys
  resources and services.
• The website has been redesigned in content
  management software, a new brand has been
  developed for the Library, and new print
  materials have been designed. These materials
  will be introduced to the NIH staff on February
  10, 2004.
• Full-Text Plus! software that will enable users
  to link easily from databases to journal articles
  to training and more has been implemented.
• Relais II has been implemented, which has
  streamlined the process used to fill document
  requests, and a web-based document request form
  is available to customers.
• The Unit Cost of Information remains steady as
  we add new electronic resources to the
  collection.

86
Initiatives for FY04

• We are in the process of developing a web-based
  survey, in conjunction with the NLM, to assess
  customer satisfaction with the informationist
  service.
• We will be conducting either the bi-annual user
  satisfaction telephone survey or a user needs
  assessment in FY04, and we will use the scorecard
  to gauge customer satisfaction with provide
  copies of publications and reference service
  provided at the information desk.
• The Parklawn Health Library will become a branch
  of the NIH Library.
• In FY04, MetaLib, software that will enable
  searching across many e-resources, will be
  implemented.
• The Web of Science software will be updated to
  the Web of Knowledge.
• Development of an in-house training curriculum to
  ensure all staff are knowledgeable about Library
  services and to retrain staff for new work will
  continue.
• To maintain the unit cost of information,
  resources and services will continue to be
  selected based on their usefulness to NIH staff
  and cost per use will be a major factor in the
  selection and retention of electronic resources.

87

• The End