Addressing cognitive requirements in knowledge engineering with Jambalaya

1
Addressing cognitive requirements in knowledge
engineering with Jambalaya

• Neil A. Ernst, Margaret-Anne Storey, Polly Allen,
  Mark Musen

2
Outline

• introduction the increasing need for cognitive
  support in knowledge engineering
• identifying tasks requiring cognitive support
• survey
• literature review
• contextual inquiries
• list of tasks
• Jambalaya demonstration
• discussion of non-functional goals for cognitive
  support

3
Motivation/Background

• Knowledge engineering is at something of a
  crossroads
• Semantic Web initiative will see more ontologies
  and intelligent apps built, some (most?) by
  non-experts
• byproduct need for better metaphors for ontology
  modeling
• not a new problem really, just not looked at much
• better metaphors will need to consider techniques
  from information visualization
• amplifies human ability to reason and
  conceptualize
• early tools included this idea
• concept maps, SemNet, KEATS, CODE4
• our work focuses on the modeler, not end-users or
  designers

4
Requirements Analysis

• Why? First approach produced a tool, Jambalaya,
  which has seen little adoption or evaluation
• Approach combination of qualitative techniques
• Literature review
• Tallis et al. 1999 describe problems users
  encounter when doing KA tasks
• Blythe et al. 2001 describe concerns users have
  adding new knowledge to a system
• Ng 2000 implemented a tool for browsing a DL
  ontology
• Clark et al. 2001 non-KE able to enter
  knowledge accurately
• problem every domain has different challenges
  and users. We need to start synthesizing this
  work and finding commonalities.

5
Requirements Analysis (2)

• User survey
• Posted a survey to rdf-interest and
  protege-discussion about the uses of ontologies
  and information visualization
• Available at http//shrimpviews.org/jambalaya/user
  -survey.htm
• Some results
• large number of domains represented
• most ontologies still small, but a significant
  number of large (gt1000 frames),
• Variety of visualization features requested, and
  variety of tools in use

6
Requirements analysis (3)

• Contextual inquiries
• two separate studies of large-scale knowledge
  engineering projects at the NCI (cancer research)
  and FMA (anatomy modeling)
• observed typical users to gain insight into
  what tasks are hard to do with current tools
• interviewed managers and chief modelers to
  determine what their goals were, and where they
  saw visualization fitting in.
• Some results
• need for custom-built reporting tool to augment
  existing system
• requests for better navigation and search
  facilities
• need for better support for collaboration

7
Proposed tasks requiring support

• Using the contextual inquiry, survey, and lit
  review, we came up with a list of KE tasks we
  think need support in KE tools
• facilitate navigation
• overviews
• slot-based browsing
• view query results
• save, annotate, and share views
• support modeling
• graphical editing techniques
• editing navigation
• support reuse
• support verification
• identify incoming relationships
• incremental navigation
• support complex relationships
• in our paper we identify from which research
  techniques these were obtained

8
Jambalaya and its feature set

• Based on the requirements/tasks identified, we
  evaluated Jambalayas functionality
• Tools for manipulating a graph
• Treemap view
• Bookmarks
• Search
• Scripting
• Views on the graph
• Nested
• Tree
• Spring
• Treemap
• Brief demo of sample domain

9
Discussion

• We suggest using non-functional design goals to
  examine what level of cognitive support is needed
  for e.g.
• Customizability to support different domains and
  users
• Usability problems still getting in the way of
  evaluation
• Learnability of necessary functions often a
  problem
• Examine issues centred around presenting the best
  metaphor in a particular circumstance
• Expressivity being able to reprsenet whats in
  the model
• Scalability handling meaningful ontologies well
• These goals are necessarily a series of
  trade-offs
• Evaluation still difficult and under-emphasized

10
Summary of Workshop Discussion

• Themes
• Semantic Web and inserted assertions/inclusions
  information sharing
• make life easier for users ( cheaper and more
  effective)
• tools should be relevant to an example/domain not
  vice versa
• deal with distributed information and 3rd party
  sources
• query formulation, presentation, and manipulation
• present context of navigation/area of interest
• editing (need to reexamine)
• meaningful products needed
• more end-user focus (esp. tools e.g. Protégé vs
  IE (w/o plugins) (address previous topic)
• Web services domain needs help (w. execution
  models) and popular

http//www.cs.uvic.ca/nernst/vike