Seeking Synchronicity: Evaluating Virtual Reference Transcripts

1
Seeking Synchronicity Evaluating Virtual
Reference Transcripts
Presented by Marie L. Radford and Lynn Silipigni
Connaway 2006 ALISE Conference San Antonio, TX,
January 16-19, 2006
2
Presenters Marie L. Radford, Ph.D. Associate
Professor, Rutgers University, SCILS Email
mradford_at_scils.rutgers.edu www.scils.rutgers.edu/
mradford Lynn Silipigni Connaway,
Ph.D. Consulting Research Scientist Email
connawal_at_oclc.org www.oclc.org/research/staff/conn
away.htm
3
Interpersonal Dimensions of Chat Research Stream
S.S. Green Award 44 transcripts (courtesy
LSSI) Maryland AskUsNow! 245 transcripts
(courtesy Maryland AskUsNow! Statewide
Consortium) Library Lawline 113 transcripts
(courtesy NELLCO Regional Consortium) Seeking
Synchronicity IMLS Grant Rutgers
OCLC Todays presentation 300 transcripts
(24/7, National International) Additional
1000 transcripts (QuestionPoint 24/7)
4
24/7 Transcript Analysis

• Generated random sample
• July 7, 2004 through June 27, 2005
• 263,673 sessions
• 25 transcripts/month 300 total
• 256 usable transcripts
• Excluding system tests and technical problems

5

• 3 Analyses
• Type of Questions
• Katz/Kaske Classification
• Subject of Questions
• Interpersonal Communication
• Radford Classification
• Manual and NVivo coding

6
Types of Questions
Combined 1st and 2nd Questions n273 questions
7
Subjects of Questions
Combined 1st and 2nd Questions n273 questions
8

• Classification Methodology
• Qualitative Analysis
• Development/refinement of
• category scheme
• Careful reading/analysis
• Identification of patterns
• Time intensive, but reveals complexity

9

• Research Questions
• Interpersonal Communication Analysis
• What relational dimensions are present?
• Are there differences in relational
  dimensions/patterns of chat users librarians?
  
• If so, what are they?
• How do users librarians compensate for lack
  of nonverbal cues?
• What is the relationship between content
  relational dimensions in determining quality?

10
Results Interpersonal Communication Analysis 2
Major Themes

• Relational Facilitators
• Interpersonal aspects of the chat conversation
  that have a positive impact on the
  librarian-client interaction and that enhance
  communication.
• Relational Barriers
• Interpersonal aspects of the chat conversation
  that have a negative impact on the
  librarian-client interaction and that impede
  communication.

11
Manual Coding Results

• 200 Transcripts
• 177 Usable Transcripts

12
Librarian Relational Factors (1-200)
Librarian Relational Facilitators Manual Analysis
n177 transcripts
13
Client Relational FacilitatorsManual Analysis
n177 transcripts
14
Comparison Relational Facilitators Manual
Analysis
n177 transcripts
15
Librarian Relational Barriers Manual Analysis
n177 transcripts
16
Client Relational Barriers Manual Analysis
n177 transcripts
17
Comparison of Relational Barriers Manual Analysis
n177 transcripts
18
NVivo Coding

• 100 Transcripts
• 79 Usable Transcripts

19
Librarian Relational FacilitatorsNVivo Analysis
n79 transcripts
20
Client Relational FacilitatorsNVivo Analysis
n79 transcripts
21
Comparison Relational Facilitators NVivo Analysis
n79 transcripts
22
Librarian Relational Barriers NVivo Analysis
n79 transcripts
23
Client Relational Barriers NVivo Analysis
n79 transcripts
24
Comparison Relational Barriers NVivo Analysis
n79 transcripts
25
NVivo Search Totals
Word Total Percent
sorry 20 25.32
thank you 51 64.56
thanks 34 43.04
thank u 3 3.80
n79 transcripts Includes words in scripts
26
NVivo Search Totals
Word Total Percent
sorry 20 25.32
thank you 51 64.56
thanks 34 43.04
thank u 3 3.80
thanx 0 0.00
TY 0 0.00
ty 0 0.00
n79 transcripts Includes words in scripts
27
NVivo Search Totals
Person Word Total Percent
Librarian sorry 14 17.72
Client sorry 7 8.86
Librarian thank you 36 45.57
Librarian thanks 18 22.78
Client thank you 28 35.44
Client thanks 24 30.38
Client thank u 3 3.80
n79 transcripts Includes words in scripts
28
NVivo 2.0 Advantages

• Advantages
• Move subnodes between nodes at any time
• Ability to create a hierarchical classification
  scheme
• Proximity and Boolean searching
• Group documents and nodes into sets for use in
  searching and analysis
• Programmatically calculate descriptive statistics
  after coding

29
NVivo 2.0 Disadvantages

• Disadvantages
• User interface does not match standard Microsoft
  application
• Cannot double click on a document or node to open
• Cannot scroll through a document using the mouse
  scroll wheel
• Cannot import nodes
• Cannot set certain tools to always appear
• Must manually display the Coder bar and coding
  strips each time for every document/transcript
• NVivo 7 to be released in Feb. 2006
• May alleviate disadvantages

30
End Notes

• This is one of the outcomes from the project
  Seeking Synchronicity Evaluating Virtual
  Reference Services from User, Non-User, and
  Librarian Perspectives.
• Funded by IMLS, Rutgers University and OCLC,
  Online Computer Library Center.
• Project web site http//www.oclc.org/research/pro
  jects/synchronicity/