Expert Knowledge Elicitation

1
Expert Knowledge Elicitation

• mental and formal models

2
Assume

• Knowledge intensive processes are driven and
  constrained by the mental models of experts
• Product development is guided by expert knowledge
• Critical process relationships that are dynamic
• Biased by individual perspectives and goals
• Conditioned by experience
• Aggregate many system components and
  relationships
• Are often nonlinear

3
Mental models

• A mental model represents one possibility,
  capturing what is common to all the different
  ways in which the possibility may occur. Mental
  models represent explicitly what is true, but not
  what is false. These characteristics lead naive
  reasoners into systematic errors.
• The greater the number of models that a task
  elicits, and the greater the complexity of
  individual models, the poorer performance is.
  Reasoners focus on a subset of the possible
  models of multiple-model problems - often just a
  single model - and are led to erroneous
  conclusions and irrational decisions.
• Procedures for reasoning with mental models rely
  on counterexamples to refute invalid inferences
  they establish validity by ensuring that a
  conclusion holds over all the models of the
  premises. These procedures can be implemented in
  a formal system most artificial intelligence
  programs) do not use them.

http//www.tcd.ie/Psychology/Ruth_Byrne/mental_mod
els/theory.html
4
Mental models Meadows

• Each person carries in his head a mental model,
  an abstraction of all his perceptions and
  experiences in the world, which he uses to guide
  his decisions mental models are intuitive
  generalizations from observations of real-world
  events. (Meadows et al. 1974 4-5)

5
Mental models Senge

• Mental models are deeply ingrained assumptions,
  generalizations, or even pictures or images that
  influence how we understand the world and how we
  take action. Very often, we are not consciously
  aware of our mental models or the effects they
  have on our behavior. (Senge 1990 8)

6
Mental models Sterman

• In system dynamics, the term mental model
  stresses the implicit causal maps of a system we
  hold, our beliefs about the network of causes and
  effects that describe how a system operates, the
  boundary of the model (the exogenous variables)
  and the time horizon we consider relevant our
  framing or articulation of the problem. (Sterman
  1994 294)

7
Cognitive psychology and dynamic systems design

• Cognitive psychology tends to examine theories of
  knowledge represented in the mind as long term
  memory
• Mental models theory argues that there are
  specialized cognitive structures in long-term
  memory used for different tasks and situations
• These are transforming and are often implicitly
  held
• Some argue that novices tend to use models and
  experts rely on abstract rules
• Researchers also propose that there are a large
  number of specialized cognitive structures that
  act in long-term and short-term memory storage
  and recall
• Scripts and schema
• Most research using mental models is
  interdisciplinary as in human-computer-interactio
  n (HCI)

8
Role in deductive reasoning Laird-Johnson

• A mental model can be defined as a
  representation of a body of knowledge either
  long-term or short-term that meets the following
  conditions 1. Its structure corresponds to the
  structure of the situation that it represents. 2.
  It can consist of elements corresponding only to
  perceptible capable of being perceived by the
  senses entities, in which case it may be
  realized as a image, perceptual or imaginary. 3.
  Unlike other proposed forms of representation, it
  does not contain variables In place of a
  variable a model employs tokens symbols that
  are fixed rather than capable of assuming
  alternate values or states (Laird-Johnson 1989
  488)

9
In HCI

• knowledge about the system, external influences,
  and control strategies (Veldhuzen and Stassen
  1977)
• Special types of schema (Jagacinski and Miller
  1978)
• mental representations of a system (Young 1983)
• organized structures consisting of objects and
  their relationships (Staggers and Norcio 1993)
• abstract concepts that represent a persons
  knowledge of a decision problem (Coury et al.
  1992)
• representations that are active while solving a
  particular problem and that provide the workspace
  for inference and mental operations (Halford
  1993)
• a persons understanding of the environment It
  can represent different states of the problem and
  the causal relationships among states (Shih and
  Alessi 1993)
• a mediating intervention between perception and
  action (Wild 1996)

10
MMODS mental models of dynamic systems

• A definition must
• point out unique attributes or qualities of
  whatever is
• defined
• not be circular
• stated positively
• use clear terms
• (following Francfort-Nachmias and Nachmias
  1992)

• a mental model of a dynamic system is a
  relatively enduring and accessible, but limited,
  internal conceptual representation of an external
  system whose structure maintains the perceived
  structure of that system. (Doyle and Ford 1998)

11
Research potentials

• Mental models provide a unified account of
  deductive, probabilistic, and modal reasoning.
• Current challenges to model theorists are to
  explain
• causal reasoning
• deontic reasoning about what is permissible and
  impermissible
• defeasible reasoning in which the facts demand a
  change in one's beliefs
• strategic thinking that occurs in making
  decisions and in reasoning about another
  individual's inferences.

http//www.tcd.ie/Psychology/Ruth_Byrne/mental_mod
els/theory.html
12
To portray mental models

• Key parts
• An image (needed if the mental model is of a
  physical thing)
• A script (needed if the mental model has a
  process)
• A set of related mental models
• A controlled vocabulary
• A set of assumptions

http//www.boxesandarrows.com/archives/whats_your_
idea_of_a_mental_model.php
13
Tacit knowledge may drive expert decisions

• Tacit knowledge is difficult to describe, examine
  and use
• Improvement of complex processes has
• False starts
• Failures
• Institutional and interpersonal conflict
• Policy resistance
• Modelers have difficulty in eliciting expert
  knowledge and representing expert knowledge as
  useful models

14
SIGGI as neural agent uses the knowledge model

• Interdisciplinary teams are required to build the
  AI system
• Knowledge elicitation of system builders and
  operators required to structure and parameterize
  the model
• To build a useful model, modelers must elicit
  from experts information about system structure
  and governing policies

15
Tacit knowledge

• Tacit knowledge is
• Subjective
• Personal
• Context-specific
• Modeling requires
• Elicitation
• Articulation
• Description of expert knowledge

16
High-tech product development

• Drives expert knowledge elicitation
• This requires multiple knowledge-driven processes
• Design to produce the final product
• Quality assurance that transforms unchecked
  designs into approved designs or designs
  requiring changes
• Critical process relationships are dynamic,
  nonlinear, biased by individual perspectives and
  goals, conditioned by experience, and aggregate
  system components and relationships

17
Standard KE practice

• Knowledge elicitation techniques are generally
  used for
• Problem definition
• Model conceptualization
• Model boundary definition
• Recognized factors
• Purpose of the modeling effort
• Phase of the model-building process and type of
  task performed
• Number of people involved
• Time available
• Cost of the elicitation methods

18
Formal modeling requirements

• Goal description of tacit expert process
  knowledge with detail an dprecision to improve,
  complex, tacit models requires description of
  relations at an operational level
• Formal modeling requires specification of data,
  functional forms, and numerical estimates of
  parameters and behavioral relationships

19
Situational model
Endsley 2000 Figure 1.
20
Series of situation models
Endsley, English and Sundararajan 1997
21
SAGAT situational awareness global assessment
technique
Endsley, English and Sundararajan 1997
22
Parameter elicitation

• SIGGI is a unique contribution in this regard
• Little in the literature on estimating functions
  and parametric relationships ( archaeological
  types)
• Reference curves constructed
• Identify reference points
• Much of the skill is tacit, passed on from
  teachers to students in apprenticeships

23
Metaphor-analogy-model

• Objective of most methods is development of
  conceptual designs and models ( concept
  development)
• Nonaka and Takeuchi (1995) look at
  organizational knowledge in 3 steps
• Describe system knowledge with metaphors
• Integrate metaphors with analogies
• Model the resulting product concepts
• (research in Japanese self-organizing teams)
• product rough, non-specific descriptions or
  drawings

http//portal.acm.org/citation.cfm?id69173 http/
/jasss.soc.surrey.ac.uk/4/1/reviews/renault.html
24
Metaphor

• Common def. a figure of speech that suggests a
  similarity between two nonidentical things.
• Brown (2003) def. the phrase conceptual metaphor
  means any nonliteral use of language that results
  in a partial mapping of one term, image, object,
  concept or process onto another to reveal
  unsuspected similarities the phrase encompasses
  similarity, metaphor, analogy, abstraction,
  model, illustration, figure, hypothesis, theory
  and mathematics observational data can be
  related to models only through metaphors for
  interpreting the data.
• E.g., molecular models are metaphors because they
  represent a mapping from the domain of pictorial
  or three-dimensional model representation onto
  the domain of data from X ray diffraction and
  other experimental observations.

25
Analogy

• all knowledge is rooted in metaphorical or
  analogical modes of perception and thought
• Metaphors and analogies move novices and
  experts' mental models into to the unknown
• Experts frequently use analogies to build bridges
  from the known to the unknown.
• Constraints of a good analogy
• Similarity The source of the analogy and the
  target must share some common properties.
• 2. Structure Each element of the source domain
  should correspond to one element of the target
  domain, and there should be an overall
  correspondence in structure.
• 3. Purpose The creation of analogies is guided
  by the problem-solver's goals.

http//128.143.168.25/book/chap2/chapter2sec4.html

26
Causal loop diagrams

• Burchill and Fine (1997) use causal loops to map
  feedbacks in concept development for new products
• Careful coding of participant comments
• Produces causal loop diagrams that capture
  participant beliefs about processes governing
  concepts
• Does not yield quantitative information

27
Formal model

• Ford and Sterman (1997) identify sources for
  system dynamics modeling
• Forrester (1994)
• mental expansive and rich in information
• written codified and more easily accessible but
  lack richness, modelers cannot query for testing,
  and it is by nature biased by the authors
  viewpoint
• numerical narrowest in scope and lacking in
  supporting contextual information about the
  generating structure
• goal of generating a formal model pushes experts
• to describe relationships at the simulation level
• clarify and specify their knowledge

28
Phased elicitation

• Ford and Sterman (1997) identify 3 sequential
  phases
• Positioning phase
• Description phase
• Discussion phase

29
Positioning phase

• Establish context and goals for the description
  process
• Facilitator sets goals and provides structure
• Focus on one relationship at a time (graphic
  frames)
• Illustrate the method (relationship description
  worksheets)

30
Description phase

• sequential development of 4 different
  descriptions of the relationship (no interactions
  between experts)
• visual description (activate, bound and clarify
  mental images)
• verbal description ( transform mental image into
  explicit form and codify knowledge)
• textual description (codified description of the
  experts knowledge)
• graphic description
• points plotted on an empty graphic frame
• experts identify relationships as graphs with no
  direction

31
Discussion phase

• seeks to test, understand and improve the
  descriptions of different experts (Vennix and
  Gubbels 1994 estimate-feedback-talk protocol)
• examine individual descriptions (each expert
  shares verbal and graphic descriptions)
• compare descriptions (there will be differences
  complexity in knowledge bases)
• mental models will be identified and assumptions
  examined
• no attempt is made to resolve differences or
  reach consensus

32
Advantages of sequential phase model

• information losses during elicitation are reduced
  compared to single-step processes through use of
  several small, separate and explicit format
  transitions
• the generation of multiple descriptions in
  different formats by a single expert allows
  testing and improvement through triangulation
• the generation of multiple individually-generated
  descriptions in a group context allows testing
  and improvement of descriptions through
  comparison to the views of other experts while
  reducing the potential for group-think and
  premature convergence

33
Sequential phase model structure
Ford and Sterman 1997 Figure 1
34
Positives

• method helps generate useful products reflecting
  community assessment
• models are improved (cf. Morecroft 1994)
• verbal and textual descriptions provide data for
  triangulation
• behavior pattern validation is improved through
  better model calibration by drawin upon expert
  knowledge
• model analysis is improved by using experts
  assessments to select the ranges of variation as
  parameters and relationships
• structural behavior validation is enhanced by
  setting limits on extreme conditions of important
  model parameters
• formal model credibility is improved in the view
  of the participants by adding multiple experts to
  the knowledge base

35
Actions

• Knowledge held in mental models is usually not
  described in other forms because it is complex
  and tacit
• KE model uses multiple formats to elicit and
  capture this expert knowledge
• use of descriptive formats adds richness to
  improve information quality through triangulation
• individual experts seek consistency among their
  descriptions of a particular relationship
• experts compare their different descriptions
• use of graphical representations through
  successions of small steps reduces cognitive
  processing required of system experts
• explaining and providing complete documentation
  of the steps to be performed improves
  descriptions and experiences of experts
• effort is expended on knowledge that experts
  consider important and proprietary
• discussion provides immediate benefits to experts
  by allowing them to share and compare mental
  models in a form that facilitates learning by
  investigating underlying assumptions
• the process of describing and comparing
  individual descriptions in groups of peers
  increases error checking

36
Prognosis

• Goal 1 SIGGI training Situation models (Endsley
  2000) work well to create computer models that
  perform as well as human models
• Goal 2 model to understand archaeologists
  thinking create a model that designers can use
  to better understand human mental representations
• Proviso neural networks, because they utilize a
  learning approach to generate knowledge, can be a
  black box and provide little insight into how
  decisions are actually made

37
Expert systems as cognitive emulations

• In an expert system a rule contains an antecedent
  and a consequent (Barcelo 2001)
• The antecedent enumerates situations in which a
  rule is applicable
• When conditions are true, knowledge in the rule
  consequent has been activated
• The expert system in reality contains hundreds of
  rules and the associations between different
  units is complex
• Consequences of determined rules will condition
  activation of others

• If A and B, then C
• If x is X, then D
• If C and D, then H
• If B, then D

38
Modalities of reasoning

• Given empirical data (observations) about a
  specific archaeological case
• Given a set of rules (If .. Then or hypotheses
  and interpretations considered valid )
• The explanation is constructed in terms of
  knowledge stored in the rule base

39
Activating knowledge units

• Two criteria
• A unit of knowledge
• An association between this unit of knowledge and
  the unit of knowledge to be proved
• By defining associations, the activation of a
  concept will expand automatically, in turn
  triggering activation of new concepts
• Following Anderson (1983), the different expert
  system hypotheses will activate according to the
  relations maintained with the possible sources of
  activation (hypotheses previously activated and
  with which it is associated

40
The expert system works

• Because the inference engine causes a cascade
  movement between different knowledge units
• If too narrow little utility
• If too broad too many possible interpretations
• Activation of knowledge units is not in computer
  code and is part of the concept of abduction
  (Josephson et al. 1987)

41
Abductive inferences

• Abductive inference syllogism
• D is a collection of data (observations expressed
  in terms of verbal descriptions, numerical
  measures, digitized images)
• H explains D (If H were true, then would imply D)
• No of the Hypotheses explains D better than H
• Then, H is correct

• If (x,y,z) are proper empirical features of
  Object F1
• And (v,W) are proper definition terms of Concept
  F
• Anth there is some association (is-a) between F
  and F1
• Then F1 activates F
• Object (F1) is an instance of Concept F

Barcelo (2001)
42
Practicality

• Thagard (1988) admonishes that we establish a
  heuristic connection between two knowledge units
  based on practicality A explains B because A
  fulfills stated requirements
• The requirements are formal or quasi-formal
• These are derived from the stated goals
• We can have 1. A formal definition or 2. have an
  association a heruistic association

43
Expert knowledge used

• Using necessary information is controlled by
  prior knowledge about the problem
  (meta-knowledge)
• This knowledge allows the designer to stipulate
  conditions that successive states of the problem
  should fulfill to become an approximation of the
  solution

44
Gardin et al. 1987 on Expert Systems concerns

• 2 groups of 6 reservations
• Group 1
• Fragmentary nature of representations
• Arbitrary nature of the factual base
• Awkwardness or ill-fit of the rules drawn
• Group 2
• Construction of definitions from vague terms
• Use of analogies that not constructed upon
  underlying calculations based on measurement and
  probability coefficients
• Overall undertaking perhaps more akin to
  improvisation than science
• Many archaeologists think expert systems
  oversimplify, force knowledge, distort knowledge,
  and do not fully exploit expert knowledge (cf.
  Francfort 1993 308)

45
Following Barcelo

• Archaeological explanation can be represented as
  series of successive actions leading to empirical
  description
• These actions are in fact operators that function
  to link the initial state (description) to the
  final state (explanation)
• The problem space in effect constitutes a framed
  theory
• There should be a finite set of independent
  operations, each with its own heuristic validity
  criteria (propositional or topological)
• Fuzzy thinking and non-monotonic expert systems
  avoid pitfalls of natural language discourse
• The truth value of concepts becomes their
  applicability in schemata of action

46
Expert System SIGGI-AACS

• SIGGI is
• a Neural Agent
• who Learns Rules
• and Thinks Creatively

47
SIGGIs Thoughts

• To train a neural agent in an AI system we must
  make implicit referential systems explicit
• We extract expert knowledge and create
  hierarchical structures or decision trees
• Siggi learns to think like an archaeologist
• But Siggi has reference to much more
  authenticated data

48
SIGGI Prototype

• Domain Expert Lohse (1985), Columbia Plateau
  cultural area
• Rufus Wood Lake projectile point chronology used.
  
• Large collection with established provenience and
  radiocarbon dates.

• Lohse classification chosen because it was
  explicitly based on established types,
• had clean provenience information,
• a suite of radiocarbon dates,
• a clear analytical framework,
• and was statistically driven.

49
Expert Training Lohse 1985

• SIGGI has currently been trained to employ the
  classification system developed by Lohse (1985)
  for the Columbia Plateau
• Explicit
• Statistically based
• Authenticated data
• Community acceptance

50
1st Order Rules Lanceolates

• Variable forms are reduced to abstract geometric
  ideals
• SIGGI is taught that this is important and that
  this leads to further distinctions

51
1st Order Rules Triangular
52
Classifying Points in Multidimensional Space

• Handles real-world variability
• Creates measurable relationships
• Defines polythetic sets that offer discrimination
  between types
• Constructs discriminant functions that
  distinguish between sets and individual specimens
• Allows continued classification and objective
  assessments of class relations as the dataset
  expands

53
Series centroid plots

• 6 morphological type series
• Clear statistical separation
• Simple lanceolate and side-notched series show
  greatest separation

54
Discriminant functions

• A mathematical function is a Cartesian product of
  a set by itself where each pair of the product is
  assigned a real number
• Relations are sets of ordered pairs where
  functions are sets of ordered triples
• Measured points can be placed in matrices and
  distances plotted (correlation matrices)

55
Rotated standardized discriminant function
coefficients, lanceolate run (Lohse 1985)

• F1 haft length
• F2 neck width, blade width, shoulder angle,
  shoulder length
• F1 and F2 adequately explain 91 of the variation
  in lanceolate types

Table 11-6, Lohse 1985
Conclusion for lanceolates, haft length and
development of well defined shoulders, coupled
with blade width and neck width are the major
variables used to distinguish recognized
lanceolate from shouldered lanceolate types.
56
Rotated standardized discriminant function
coefficients, triangular run (Lohse 1985)
Table 11-7, Lohse 1985
Conclusion for triangular points, variance in
form is more subtle than in Lanceolate types,
and the primary discriminating variables are
shoulder Angle, basal margin angle and stem size
or overall proportion, reflected in Basal width
and the ratio between neck and basal width.
57
Canonical Discriminant Runs, Lohse (1985)
58
Cases correctly classified, Lohse (1985)
59
Rufus Wood Lake Projectile Point Chronology
Modal types - descriptive - historical
60
Measurement Grids
Triangular Forms
Lanceolate Forms
61
SIGGI doesnt use discriminant analysis

• SIGGI is short for use of sigmoid functions as
  part of curve fitting related to form recognition
• Sigmoid functions are often used in neural
  networks to introduce nonlinearity in the model
  and to make sure that certain signals remain
  within a specified range.
• A popular neural net element computes a linear
  combination of its input signals, and applies a
  bounded sigmoid function to the result this
  model can be seen as a "smoothed" variant of the
  classical threshold neuron.

http//www.answers.com/topic/sigmoid-function
62
Curve fitting sigmoid functions
http//mathworld.wolfram.com/SigmoidFunction.html
63
Actual outlines are reduced to geometric forms

• Objects defined in a descriptive system can be
  conceived as points in space, whose dimensions
  are descriptors of the objects
• - 2D space (2 measured axes elements in the
  vectors) distributions are regular, linear,
  clustered or random
• - 3D space (Euclidean 3D space 5D etc.
  coordinates constitute vectors) clusters now
  occur in multiple dimensions
• N2 clusters in the 5 empirical dimensions
  constitute 2 new abstract dimensions composed of
  multiple descriptors

64
Importance of Latent Dimensions

• Data elements are transformed into latent
  dimensions in order to manipulate solutions at
  the abstract or symbolic level
• Latent dimensions can be found easily on the
  computer
• The latent dimensions are archaeological
  structures ( polythetic sets)
• Identification of latent dimensions allows
  removal of unnecessary information using Bayesian
  inference models

http//reach.ucf.edu/aln/pyle/theory.html
65
Intersecting Variables
66
TYPES

• RULES FOR INCLUSION OF DATA
• 1- A defined type must have a clearly proscribed
  range of variation defined quantitatively or
  qualitatively
• 2- The named type must have been recovered in
  definable archaeological contexts, and is
  isolatable in specific stratigraphic sequences

67
SITES

• RULES FOR INCLUSION OF DATA
• 1- Sites must have been excavated in cultural
  stratigraphic levels and not in natural or
  arbitrary levels.
• 2- Provenience information must be available for
  all recovered artifacts that specifies cultural
  units as to stratum, feature and association.
• 3- A detailed descriptive report covering
  excavation methodology and analysis must be
  published for the site or be in the process of
  publication, and excavation notes and photographs
  must be on file at a recognized repository.

68
ASSEMBLAGES

• RULES FOR INCLUSION OF DATA
• 1- An artifact distribution to qualify as a site
  activity assemblage must be defined in a discrete
  vertical and horizontal distribution associated
  with a recognizable cultural feature.
• 2- The cultural and natural stratigraphy must
  indicate that the assemblage represents a
  discrete prehistoric activity. The assemblage is
  not an analytical construct but a found context.
• 3- The assemblage indicates a discrete series of
  tasks or task-related activities. The assemblage
  is not an amalgam of activities over an extended
  period of time reflecting different seasons of
  site use nor different uses in different years.

69
TYPES AND ASSEMBLAGES

• Assemblage Windust
• a composite
• Types Windust, Cascade and Cold Springs S-n
• Windust and Cascade
• are distinctive to region (cultures)
• Large Side-n non-
• distinctive as regional
• indicator ( associated
• temporal marker)

Ames et al. 1998 reproduced from Leonhardy and
Rice 1970
70
Inspection of Interstices

71
Plateau Arche-Types
Rabbit Island
Cascade
Windust
Lind Coulee
72
Plateau Archetype Schema
73
Common Structures
74
Middle Archaic C-n Forms
How to identify common and unique types?
Columbia Plateau these are Columbia C-n, large
and small Plains these are Pelican
Lake specimens Basin these are Elko
C-n specimens
http//www.abheritage.ca/alberta/archaeology/i_pel
ican.html
75
Late Archaic S-n Forms
How to identify common and unique types?
Columbia Plateau these are Plateau S-n Plains
these are Avonlea specimens Basin these are
Desert S-n specimens
http//lithiccastinglab.com/gallery-pages/avonleag
rouplarge.htm
76
SIGGI as a user-defined smart interface
77
Loading images
78
Defining outlines eroding
79
Simple outlines
80
Probabilities of membership morphological
categories
81
Probabilities of membership Historical types
82
To Do

• Obtain information from domain experts
• Columbia Plateau
• Northwestern Plains
• Great Basin
• Classify methods by interactions with domain
  experts
• Classify by types of information elicited from
  domain experts

83
Expert Interactions 1

• Interviews (concept mapping, cognitive structure
  analysis, data flow models, task action mapping)
• Case Studies (context-driven)
• Protocols (actions and mental processes)
• Critiquing

84
Expert Interactions 2

• Role Playing
• Simulations
• Prototyping (expert evals, iterative system
  evals, rapid prototyping, storyboarding)
• Teach-back
• Basic observation

85
Need for authenticity

• Only data that can be verified will be entered
  into the database
• used to train SIGGI
• used to test SIGGI
• Explicit rules for inclusion can be constructed
• sites to include
• assemblages to include

86
Conclusions

• SIGGI was developed as
• an explicit classification system based on
  accepted typologies
• a smart user interface to
• manipulate large relational databases
• explore the nature of archaeological
  classification
• SIGGI as a prototype is smart enough to
• classify specimens into general accepted types
• begin explicit comparisons between cultural areas
• begin knowledge elicitation experiments with
  archaeological experts

87
Interface design

• Simplicity since mental models simplify reality,
  interface design should simplify actual computer
  functions.
• Familiarity an interface should allow users to
  build on prior knowledge, especially knowledge
  gained from experience interacting in the world.
• Availability an interface should provide visual
  cues, reminders, lists of choices, and other
  aids, either automatically or on request (users
  emphasize recognition vs. recall).
• Flexibility an interface should support
  alternate interaction techniques, allowing users
  to choose the method of interaction that is most
  appropriate to their situation (users should be
  able to use any object in any sequence at any
  time).
• Feedback a system should provide complete and
  continuous feedback about the results of actions.
  (feedback a user gets that supports their current
  mental model strengthens it).
• Safety a user's actions should cause the results
  the user expects.
• Affordancesan affordance refers to the
  properties of an object (use real-world
  representations of objects)

http//www.lauradove.info/reports/mental20models.
htm
88
References

• Barcelo, Juan A., 2001, Expert systems as
  cognitive emulation An archaeological viewpoint.
  http//seneca.uab.es/prehistoria/Barcelo/ExpertEmu
  lation.html
• Brown, Theodore L. Brown, 2003, Making Truth
  Metaphor in Science. Urbana University of
  Illinois Press.
• Ford, David N. and John D. Sterman, 1998, Expert
  Knowledge Elicitation to Improve Mental and
  Formal Models, System Dynamics Review 14(4)
  309-340.
• Nehaniv, Chrystopher L. (ed.), 1999, Computation
  for Metaphors, Analogy, and Agents. Lecture Notes
  in Artificial Intelligence 1562. Berlin
  Springer-Verlag.
• Nonaka, Ikujiro and Hirotaki Takeuchi, 1995, The
  Knowledge-Creating Company, How Japanese
  Companies Create the Dynamics of Innovation. New
  York Oxford University Press.
• Weidmann, Karl-Heinz, Metaphors, Conceptual
  Models and Evolutionary Epistemology.