Bruce Porter University of Texas

1
Building KBs by Assembling Components An early
evaluation of the approach

• Bruce Porter (University of Texas)
• Peter Clark (Boeing)
• and Colleagues

2
Claims

• A component is a set of axioms that
• describe a consensus view of a common concept
• can be frequently reused with little modification
  
• to combine with other components
• to represent domain knowledge well.

3
Claims

• A component is a set of axioms that
• describe a consensus view of a common concept
• can be frequently reused with little modification
  
• to combine with other components
• to represent domain knowledge well.

versus an idiosyncratic view
4
Claims

• A component is a set of axioms that
• describe a consensus view of a common concept
• can be frequently reused with little modification
  
• to combine with other components
• to represent domain knowledge well.

versus infrequently reused without significant
modification
5
Claims

• A component is a set of axioms that
• describe a consensus view of a common concept
• can be frequently reused with little modification
  
• to combine with other components
• to represent domain knowledge well.

versus difficult or impossible to combine
(round holes and square pegs)
6
Claims

• A component is a set of axioms that
• describe a consensus view of a common concept
• can be frequently reused with little modification
  
• to combine with other components
• to represent domain knowledge well.

versus representations are incomplete or
inconsistent
Well evaluate each of these claims in turn
7
The Context for our Evaluation The Component
Approach in Practice

• We are building a component library while
  assembling a microbiology KB for the TKCP
• For each microbiology topic, we will
• Identify the core concepts used in textbook
  accounts
• Build components for these concepts
• Use these components to represent the topic in
  the KB

8
Do the components capture a consensus view?

• Internal evaluation
• Have multiple KEs (team members) independently
  encode each component.
• Measure the agreement among the representations
• External evaluation
• Translate components from KM into English,
  yielding dictionary definitions
• Have n subjects review the definitions then
  revise them if they deem appropriate.
• Have an independent subject extract the
  consensus view from the n definitions.
• Measure the agreement between the consensus view
  and the original ones.

9
Can the components befrequently reused with
little modification?

• Count the amount of reuse of each component.
• Count the number of modifications made for each
  instance of reuse, and weight it by its severity.

10
Do the componentscombine together well?

• For each microbiology topic
• Write a high-level design for its representation
  which shows
• The components that comprise it
• Their instantiations and inter-relationships
• Implement the design
• Measure and study the times when implementation
  details force changes to the design

11
Do the componentsrepresent domain knowledge well?

• For each topic represented in the KB
• Measure how much of its representation was
  provided by components the balance was coded
  specifically for this topic.
• The TKCP will measure the quality of the overall
  representation, in terms of consistency,
  completeness, support for inference, and so on.
• Analyze the KBs successes and failures at the
  TKCP to assign responsibility to the components
  (versus other aspects of the KB system).

12
Summary

• We have identified the strong claims of the
  component approach
• We plan to evaluate the claims early while
  building a microbiology KB
• The data we collect will be invaluable to study
  questions weve only begun to formulate