Week 8

1
Week 8

• MSE614 SP 08
• Ileana Costea

2
HW Questions on KA

• Due today, Week 8
• Assigned last session, Week 7
• A few verbal questions
• (see Transparency)

3
HW Question 1What is meta-knowledge?

• Answer
• K about K, or
• The systems K about how it reasons
• e.g.,
• how to use K in specific situations
• how to determine which K is relevant
• when the K is insufficient

4
Meta-knowledge (ctnd.)

• Meta-K allows the system to examine the operation
  of the descriptive (declarative) and procedural K
• Explanation in an ES can be viewed as Meta-K
• In the future Meta-K will allow ES more
• Create the rationale behind individual rules
• by reasoning from first principles
• Tailor explanations to fit the audience
• Able to change their internal structure through
• Rule correction
• Reorganization of KB
• System reconfiguration

5
HW Question 2Give four reasons why KA is
difficult.

• Representation mismatch
• between the human Expert and
• the program underdevelopment
• Large number of participants
• Transfer via a machine
• Difficulty of experts to describe their K
• More next slide

6
HW Question 2 (Ctnd.)Give four reasons why KA
is difficult.

• Other reasons, see slide.
• Overcoming difficulties
• Research on KA to reduce mismatch
• Develop systems able to accept advice
• KA to converse with expert in natural language
• Simplify the syntax of the rules so that an
  expert can build a system without training

7
HW Question 3Describe the process of Protocol
Analysis. (Ctnd.)

• The expert is asked to perform a real task.
• Then he/she is asked to verbalize (think aloud)
  the thought process.
• A record (protocol) of all the details is made.
• The records are transcribed.

8
HW Question 3Describe the process of Protocol
Analysis. (Ctnd.)

• Required Web search
• http//www.psy.fsu.edu/faculty/ericsson/ericsson.p
  roto.thnk.html
• Protocol analysis and Verbal Reports on Thinking
• An updated and extracted version from Ericsson
  (2002)
• http//www.epistemics.co.uk/Notes/176-0-0.htm
• Protocol Analysis Techniques
• Among others discusses Repertory Grid Technique
• http//en.wikipedia.org/wiki/Protocol_analysis

9
Protocol Analysis

• http//en.wikipedia.org/wiki/Protocol_analysis
• (from Wikipedia, the free encyclopedia)
• Protocol analysis is a psychological research
  method that elicits verbal reports from research
  participants. Protocol analysis is used to study
  thinking in cognitive psychology, cognitive
  science, and behavior analysis. It has found
  further application in the design of surveys and
  interviews, usability testing, and educational
  psychology.
• Usability testing is a technique used to evaluate
  a product by testing it on users.
• Cognitive psychology is a school of thought in
  psychology that examines internal mental
  processes such as problem solving, memory, and
  language.

10
Cognitive science(from Wikipidia)

• the scientific study either of mind or of
  intelligence.
• an interdisciplinary study drawing from relevant
  fields including psychology, philosophy,
  neuroscience, linguistics, anthropology, computer
  science, and biology.
• term coined by Christopher Longuet-Higgins in his
  1973 commentary on the Lighthill report, which
  concerned the then-current state of Artificial
  Intelligence research. In the same decade, the
  journal Cognitive Science and the Cognitive
  Science Society began.

11
Protocol Analysis (Ctnd.)

• in AI KA it is a method of KA
• it is one of the tracking methods
• it is the most common method of formal tracking.
• (See next slide for tracking)

12
Tracking Methods(Protocol Analysis Ctnd.)

• Process tracking a set of techniques that
  attempt to track the reasoning process of an
  expert.
• Popular method among cognitive psychologists who
• interested in discovering the experts train of
  thought while he/she reaches a conclusion
• Tracking methods are informal and formal

13
Protocol Analysis (Ctnd.)

• Particularly a set of techniques known as verbal
  protocol analysis,
• a common method by which KEngineer acquires
  detailed K from the expert

14
Protocol Analysis (Ctnd.)

• A protocol a record or documentation of the
  experts step-by-step info. processing (IP) and
  decision-making (DM) behavior
• Protocol Analysis Is similar to interviewing but
  more formal and systematic
• The expert is asked by the KE to perform a real
  task and verbalize his/her thought process
• Expert is asked to think aloud while performing
  the task or solving a problem under observation
• A recording made while expert thinks aloud
• Recording describes every aspect of the IP and DM
  behavior
• The recording becomes a record, or protocol, of
  the experts ongoing behavior

15
Protocol Analysis (ctnd.)

• Recording is
• transcribed by the KE for further analysis
• ( e.g., to deduce the decision process)
• Coded by the KE

16
Protocol Analysis (ctnd.)

• In contrast with interactive interviewing
  methods, PA involves mainly a one-way
  communication.
• KE prepares the scenario and plans the process
• during the session the Expert does most of the
  talking as he/she interacts with the data to
  solve problem
• KE listens and records the process
• Later KE must analyze, interpret, and structure
  the protocol (or record) into Knowledge
  Representation for a review by the Expert.

17
Protocol Analysis (ctnd.)

• for Procedure of PA
• Advantages and Limitations of PA
• ? see Transparencies

18
HW Question 4List the major difficulties of KA
from multiple experts.

• Different experts use different methods of
  problem solving, yet all may be correct.
• It is difficult to reconcile these different
  methods.
• It is difficult to get all the experts together
  at the same time and place.

19
List the major difficulties of KA from multiple
experts. (Ctnd.)

• The four possible scenarios to deal with multiple
  experts
• individual experts
• primary and secondary experts
• small groups
• panels

20
List the major difficulties of KA from multiple
experts. (Ctnd.)

• Major methods of dealing with multiple experts
• Consensus methods
• Reach consensus by using group dynamics or other
  methodologies
• Analytical approaches
• figure the average estimate of the group
• Selection of an appropriate line of reasoning for
  each occasion dont mix
• Automation of the process
• automatic decision by the computer
• Blackboard system
• divide the problem into sub-domains and use one
  expert for each sub-domain

21
HW Question 4Define evaluation, validation and
verification of K.
22
AAAIhttp//www.aaai.org/home.html

• Association for the Advancement of Artificial
  Intelligence (AAAI)
• (formerly the American Association for
  Artificial Intelligence)
• is a nonprofit scientific society devoted to
  advancing the scientific understanding of
• mechanisms underlying thought intelligent
  behavior, and
• their embodiment in machines

23
AI Applications

• http//www.aaai.org/AITopics/pmwiki/pmwiki.php/AI
  Topics/SiteMap
• a whole list of links of AI applications
• http//www.aaai.org/AITopics/pmwiki/pmwiki.php/AI
  Topics/AINews
• AI in the News