Seeking Synchronicity: Evaluating Virtual Reference Transcripts

1
Seeking Synchronicity Evaluating Virtual
Reference Transcripts
Presented by Lynn Silipigni Connaway and Marie
L. Radford QuestionPoint Users Group
Meeting June 25, 2006 New Orleans, Louisiana
2
Seeking Synchronicity Evaluating Virtual
Reference Services from User, Non-User, and
Librarian Perspectives

• 1,103,572 project funded by
• Institute of Museum and Library Services
• 684,996 grant
• Rutgers, The State University of New Jersey and
  OCLC Online Computer Library Center 405,076 in
  kind contributions

3
Seeking Synchronicity Evaluating Virtual
Reference Services from User, Non-User, and
Librarian Perspectives

• Project duration
• 10/1/2005-9/30/2007
• Four phases
• Focus group interviews
• Analysis of 1,000 QuestionPoint transcripts
• 600 online surveys
• 300 telephone interviews

Interviews surveys with VRS users, non-users,
librarians
4
Phase II24/7 Transcript Analysis

• Generated random sample
• July 7, 2004 through June 27, 2005
• 263,673 sessions
• 25 transcripts/month 300 total
• 256 usable transcripts
• Excluding system tests and technical problems

5

• 6 Analyses
• Geographical Distribution
• Library receiving query
• Library answering query
• Type of Library
• Type of Questions
• Katz/Kaske Classification
• Subject of Questions
• Dewy Decimal Classification
• Session Duration
• Interpersonal Communication
• Radford Classification

6
n255
7
n238
8
n256
9
n273
10
n273
11
n273
12
n273
13
n273
14
Service Duration

• Mean Service Duration
• 1353
• Median Service Duration
• 1037

15
Transcript Reading

• Positive VRS experience
• Duration 1 hour 11 minutes
• Academic User
• Question Boston drug company - diabetes
• Relational Work
• Enthusiastic user
• Helpful librarian
• Less than positive VRS experience
• Duration 39 minutes
• Middle school or high school student
• Question physics car acceleration
• Poor reference work
• Extreme negative closure

16
Focus Group InterviewsReasons for Using VRS

• Convenient
• Efficient
• More reliable than search engines free
• Allows multi-tasking
• Email follow-up provision of transcript
• Pleasant interpersonal experience
• Librarian on first name basis more personalized
• Less intimidating than physical reference desk
• Feel comfortable abruptly ending session

17
Focus Group InterviewsReasons for not using VRS

• Graduate students
• Fear of
• Bothering librarian
• Looking stupid advisors finding out
• Questions may not be taken seriously
• Potential technical problems
• Bad experiences in FtF influence expectations of
  VRS
• Screenagers
• Virtual stalkers (psycho killers)
• Not finding a trusted librarian
• Unsure of what to expect

18
Focus Group Interviews Challenges for Users
Non-Users

• Speed and technical problems
• Delayed response time
• Librarians are not in users libraries
• Fear of no subject expertise
• Fear of overwhelming librarian

19
Focus Group Interviews Suggestions from Users
Non-Users

• Inclusion of multiple languages
• Access to subject specialists
• Better marketing and publicity
• Information on how to connect and use VRS
• Reassurance that users will not bother librarians
  the library wants the service to be used
• Faster technology
• Improved interface design
• More color
• More attractive

20
Next Steps

• Conduct
• Three focus group interviews VRS users
• Online survey telephone interviews with VRS
• Users
• Non-users
• Librarians
• Analyses
• Gender
• User Type
• Child/Young adult
• Adult
• Unknown

21
End Notes

• This is one of the outcomes from the project
  Seeking Synchronicity Evaluating Virtual
  Reference Services from User, Non-User, and
  Librarian Perspectives, Marie L. Radford and Lynn
  Silipigni Connaway, Co-Principal Investigators.
• Funded by IMLS, Rutgers University and OCLC,
  Online Computer Library Center.
• Project web site http//www.oclc.org/research/pro
  jects/synchronicity/

22
Questions Marie L. Radford, Ph.D. Email
mradford_at_scils.rutgers.edu www.scils.rutgers.edu/
mradford Lynn Silipigni Connaway, Ph.D. Email
connawal_at_oclc.org www.oclc.org/research/staff/conn
away.htm