BT Case Study Evaluation

1
BT Case Study - Evaluation

• Deliverable 26
• Alistair Duke

2
Case Study Recap

• Knowledge Sharing in a community of interest
• 3 related groups in BTexact
• 31 users
• Using OntoShare
• Knowledge sharing around a domain ontology
• Ontology evolves with usage
• RDF(S) info resource automatically created

3
Recap Tool Integration
OntoShare
Ontology
User
RQL
QuizRDF
Spectacle
Ontology
OntoEdit
Ontology
Ontology Data
OIL-Core
Usage Statistics
OMM
Bor
OntoEdit
OntoShare
Sesame
OIL-Core ontology repository
Ontology
Annotated Data Repository
RDF
RDF
Ontology Data
Ontology
pers05
731
par05
car
tel
about
Sesame
OntoExtract
OntoWrapper
Ontology Data
Data Repository (external)
Other Applications
4
Evaluation

• Statistical Analysis.
• Tracking functional usage
• Individual user granularity
• Questionnaire
• Pre Post trial
• Linked with usage
• Usability Study

5
Methodology
Ontology Workshop
Questionnaires
Usability Study
Analysis of Usage Stats
6
Workshop
7
OntoShare Usage
8
(No Transcript)
9
Sharing Documents

• 200 Documents Shared
• Only 6 users stored more that 5 documents
• Community relies on a subset of people who are
  eager to share information
• a typical community of practice model
• No of user comments on shared documents - 16
• Low - but could be due to user interface
• effort vs usage

10
Evolving Ontology

• User initiated concept addition - 9
• 0 user initiated concept moves or deletions
• New concepts were accepted and community added
  documents to them
• evolving relatively slowly (as expected)
• System concept modification suggestions
• well used
• User feedback Interesting / Not Interesting
• Low usage
• effort vs usage

11
Ontology Evolution
12
Negative Feedback

• As it was Java based, it took a long time to
  load and as time is a premium, this is probably
  the one single reason I hardly used it.
• HTML-based interface in standard browser would
  be better
• ... Jasper seems better because it does less.
  Ontoshare takes more effort to learn.

13
Negative Feedback

• These tools are great when you have (or make)
  time to use them. When spare time is short, I
  tend to not bother (even though I know the
  information would probably be of benefit to me).
• As for OntoShare alerts, well the general volume
  of email just puts these emails to the 'back of
  the queue' (and I never get round to viewing
  them).

14
Positive Feedback

• Evolving hierarchy of concepts was the best
  feature
• The expansion of interest groups into an
  evolving hierarchy of concepts is a big
  improvement.

15
Positive Feedback

• Good to see what ones peers are reading.
• The layout of the interface.
• ..intelligent suggestion of which concept a
  given item is relevant to

16
Learning Points

• Give careful consideration to the nature of the
  virtual community / cultural aspects
• Provide better interface access
• Provide wider access to functions
• Richer ontological representation
• Provide better support to new users
• Inform users about an ontology change

17
Learning Points

• Physical Presence is required
• Tool support is beneficial
• Relatively simple ontologies can be expected from
  domain experts in industry
• The On-To-Knowledge methodology provides an
  effective framework for the introduction of an
  ontology-based application