Forum Minerva

1
Forum Minerva

• Literature study on knowledge-intensive
• assistance
• Jouke Verlinden

2
Agenda

• Literature study knowledge intensive assistance
• My research topic Dialogue Management
• Preliminary results on the web
  http//dutoce.io.tudelft.nl/jouke/pres23082000/li
  tstudy.doc

3
Study Knowledge-intensive assistance

• Why?
• What?
• Status?
• Some examples?

4
Study Knowledge- intensive assistance

• Why?
• My focus is the semantics in communication on
  shape design.
• To get familiar with the field (vocabulary,
  concepts, etc.). And to find gaps in existing
  knowledge.
• Questions
• What are the key technologies and methods ?
• What is the current state of research in these
  areas?
• What are good examples of these?

5
Study Knowledge- intensive assistance

• What?
• Knowledge-intensive
• intelligent (AI)
• knowledge-based
• Assistance
• collaborative
• agent-based

6
Literature - starting points

• What?
• UIST - User Interface Software Technologies
  (1990-1999),
• IUI - Intelligent User Interfaces (1993-2000),
• CHI - Computer-Human Interfaces (1993-2000)
• Journal 'User Modeling and User-adapted
  interaction' (Kluwer) - 1995 - 2000
• Journal 'Interacting with Computers' (Elsevier) -
  1996-2000
• Journal 'Knowledge-Based Systems' (Elsevier) -
  1993 - 2000
• Intelligent CAD Systems I, ten HagenTomiyama,
  1987.
• Intelligent CAD Systems II, Akman, ten
  HagenVeerkamp,1989.
• Intelligent CAD III, Yoshikawa, Arab
  (Tomiyama),1991.
• 'Readings in Intelligent User Interfaces',
  MayburyWahlster (eds.), 1998.

7
Literature study

• Status?
• Human Factors, AI
• 90 Read
• 50 Documented
• Plan finish in October.

• Design
• 20 Read
• ? Documented

8
Two topics in knowledge-intensive assistance
Dialogue
Model
1. Programming by Example (PBE) 2. Design
Critique
9
Programming by Example (PBE)

• Tries to assist user by automating user input
• (macros).
• agent watches the interaction of the user with
  the application
• interaction history inferencing algorithm
  gtgeneralized macros
• agent suggests macros
• Macros on Steroids Lieberman, 2000

10
PBE Example Eager

• Short movie (from web)
• Cypher, 1991

11
Possible application in Design

• repetitive task to parts of a design (e.g.
  multiple filleting or creating a complete façade
  after drawing one window)
• larger, compound actions to be reapplied to a
  completely different design (style)
• ...

12
Principles of PBE

• Machine Learning (generalization)
• Management of behaviors
• Initiative (activation, responsiveness)
• Mondrian (2D drawing) Lieberman, 1993
• MetaMouse (2D alignment) Maulsby, 1993
• APE (learning alg.) Ruvini, 2000

13
PBE Issues

• Explicit vs implicit training
• Generalization/learning algorithms
• Felicity conditions
• Visualization

14
Design Critiquing

• Originally from AI - expert systems.
• Reflect on a design while designing

Model
Designer
Agent
Analysis
15
Design Critique - example 1

• Fisher, 1993 Hydra
• Interactive CAD of kitchen interiors.
• Critiquing

16
Design Critiquing - principles

• ADAIR Model Robbins, 1998
• Multiple critics/experts

Activate
Advise
Record
Improve
Detect

• Correctness
• Completeness
• Consistency
• Optimization
• Evolvability

• Presentation
• Tool
• Experiential
• Organizational

17
Design Critiquing - example 2

• ArgoUML Robbins, 1998
• A design system for software (OO based)
• Demo

18
Design Critiquing - issues

• the right information at the right
  time(depends on domain, task, user)
• Room for explanation (why do you suggest .?)

19
My Research

• Dialogue management
• ICA Vision (periphery)
• Dialogue Management - functionality
• Contributions

20
Vision of our research

• A design studio with digital extensions
• Seamless integration of virtual and real design
  space
• Capable of capturing more than just an end-result
  (a design is just one possible end node in the
  process of design)

21
Vision of our research - ingredients

• Advanced representations
• shape
• physical behavior
• processes

• Representation management
• ontologies (exchange, schemes, associations)

• Analysis/
• critiquing tools
• ergonomics

• 3D printing
• large stacked foam models

• Conversational dialogue
• facilitation
• coping with incompleteness

• Multimodal manipulation
• 3D gestures
• multimodal input

22
Vision of our research - ingredients

• Advanced representations
• shape
• physical behavior
• processes

• Representation management
• ontologies (exchange, schemes, associations)

• Analysis/
• critiquing tools
• ergonomics

• 3D printing
• large stacked foam models

• Conversational dialogue
• facilitation
• coping with incompleteness

• Multimodal manipulation
• 3D gestures
• multimodal input

23
Dialogue Management

• 1. Model-based dialogue specification
• 2. Coping with incompleteness
• 3. Facilitate knowledge-intensive assistants

User
System/Environment
does not know what the system supports. (tools/med
ia)
does not understand what the user
does. (intentions)
24
1. Model-based dialogue

• No hard-coding of dialogue.
• Models for user, domain and interaction determine
  behavior of system

25
2. Coping with incompleteness
Raw data
Not understandable
Hypothesis
Ambiguity
Conclusion
Incorrect operation
Model
Not presentable
Presentation
26
2. Incompleteness

• Different strategies to resolve these issues
  (recovery strategies) Multimodal Fusion,
  discourse analysis, inferencing, subdialogues.
• Heavily based on Computational Linguistics and
  Multimodal architecture research.

27
3. Agent facilitation

• Plug-and-play agents
• Facilities to track behavior and models - most
  important discourse model

28
My research - contributions

• Dialogue models for shape design (user, domain,
  interaction).
• Algorithms (interaction and subdialogues,
  disambiguation, inferencing).
• Software architecture (platform).

29
My Research - plan

• Finish literature study (october)
• Experiments exploring good/bad dialogue
  management, inventarisation of incompleteness
  (this quarternext).
• Some pilot implementations (software)
  evaluation of existing dialogue platforms. (this
  year).

30
The end