Quentin Limbourg

1
Multi-Path Development of User Interfaces

• Quentin Limbourg

Louvain-la-Neuve, 4th November 2004
2
Presentation Plan

• Context
• Problem Statement
• Thesis
• Proposal
• Ontological proposal
• Methodological proposal
• Case Studies and tools
• Conclusion Perspectives

3
Context

• Transformational-driven Development of User
  Interfaces
• Endorses the motivation of SE for HCI
• User-Centred Design of UIs
• Deriving a UI is heuristic and is not a straight
  line

4
Context (contd)

• UIs are running fastafter change
• Task redefinitions or reallocation
• Organizational adaptation
• Domain evolution
• Obsolescence of languages and New Computing
  languages
• New platforms
• These changes give rise to the ellicitation
  development scenario

5
Context contd

• These development scenarios are relevant to
  several problem areas
• Computer-assisted derivation of UIs
• UI maintenance and refactoring
• Multi-context UI development
• Reverse engineering of UIs

6
Problem Statement

• Ontological shortcomings
• Explicitness, rigorousness, extendibility,commitme
  nt, communication
• Methodological shortcomings
• Expliciteness, rigorousness, consistency in
  application, predictability, flexibility,
  modifiability, communication.

7
Thesis Statement

• It is possible to address the previously outlined
  shortcommings by
• A definition of an explicit and rigorous
  ontological framework for defining concepts
  involved in UI development
• A definition of an explicit and rigorous
  methodological framework that is transformation
  driven and multi-path

8
Multi-Path Development of UI
Forward engineering
Reverse Engineering
Adaptation
Any Relevant Composition

• Multi-path development qualifies a methodology
  that support the realization of multiples types
  of development paths withing a single framework

9
Proposal

• To support this approaches in a single framework
  we need
• An ontological framework that provide a
  description of concepts for each development
  stage
• A methodological framework that allows a
  definition, organization, and execution of
  development paths

10
Ontological Framework

• An ontology is a formal specification of a
  conceptualisation

11
Ontology Abstract Concepts

• Structured in viewpoints and in models
• Task (CTT some improvements)
• Domain (UML Class/Object diagram some
  improvements)
• Abstract User Interface (vocabulary independent
  of the modality)
• Concrete User Interface (vocabulary independent
  of the toolkit)
• Context of use (ltUser, Platform, Environmentgt)
• Inter-model relationship mappings (traceability,
  integration of all views)

12
Ontology Syntax

• Abstract syntax
• Directed, labeled, attributed, typed graphs.
• Nodes are concepts
• Edges are relationships between these concepts
• Result a UI specification is a BIG WHOLE graph
  
• Concrete syntax USIXML and AGG visual notation
• USer Interface eXtensible Mark-Up Language
• (graph structure is achieved by defining
  explicitly relationships)
• AGG Visual Notation

13
Methodological Framework


Task and
Task and
Task and
Task and
Reification
Reification
Reification
Domain
Domain
Domain
Domain


Abstraction
Abstraction
Abstraction
u
z
z
Abstract
Abstract
Abstract
Abstract
Translation
Translation
Translation
User Interface
User Interface
User Interface
User Interface
Code Generation
Code Generation
Code Generation
v


y
y
Code Reverse
Code Reverse
Code Reverse
Engineering
Engineering
Engineering
Concrete
Concrete
Concrete
Concrete
User Interface
User Interface
User Interface
User Interface
?
w
x
User Interface
User Interface
User Interface
Final User Interface
Code
Code
Code
14
Model-to-Model Transformation

• In the literature
• Opportunistic algorithms
• APIs
• XSLT
• Drawbacks
• Execution semantics is poorly defined
• Consistency in applying steps
• Consistency of resultant model
• Procedurality
• Cryptic syntax
• Low predicatbility
• Hidden rationale (entails low re-use)

15
Conditional Graph Rewritting

• Generalization of string grammars
• Grounded execution semantics (pushout
  construction)
• Side-effect free
• Attractive syntax
• Declarativeness
• Determinism (application strategy)
• Seamlessness with ontological world (rules
  manipulate patterns of specification)
• Tools
• Extension (PAC, NAC)

16
How it works

• Find an occurrence of LHS in G (this occurrence
  is called a match). If several occurrences exist,
  choose one non-deterministically.
• Check preconditions of both type PAC and NAC. If
  not verified, then skip.
• Remove the part of G which corresponds to (LHS
  K), where K is the morphism specified between LHS
  and RHS.
• Add RHS K into G (LHS K) as it is given by
  the corresponding relation between RHS K and K.

• Check postconditions of both type PAC (and
  notably that the resulting graph is
  properly typed) and NAC. If not verified, then
  undo the transformation rule.

17
Methodological World and Transformation World
A development library stores (in usiXML textual
format) paths, steps and sub-steps definition and
their associated transformation systems and
transformation rules
18
Substeps From AUI to CUI

• Definition of AUI structure
• Which objects should be logically grouped ?
• Definition of AIC types
• Which  functionnality  should assume AICs and
  what data do they manipulate ?
• Definition of spatio-temporal arrangement
• How should AIC be arranged in space and/or in
  time ?
• Definition of dialog control
• What is the valid flow of action on AIOs ?

19
Substeps From AUI to CUI

• Define CUI structure
• Which AIC is a window ?
• Define CIC type ?
• Which  widget  should represent which AIC ?
• Define placement ?
• What layout can be specified between CICs, CCs
• Define navigation ?
• Which container can be started or closed from
  which container ?
• Define dialog control ?
• What is the valid flow of action on AIOs

20
Typed Model-to-Model Transformation
Meta-Meta-Model
Graph Structure
is instance of
Meta-Model
Our Meta-Model
Meta-Model Subset 1
Meta-Model Subset 2
e.g., TaskDomain Model
e.g., Concrete UI Model
Uses language
is instance of
is instance of
Initial UI Model
Resultant UI Model
Transformation Rule
e.g., MyTaskAndDomainModel
e.g., MyConcreteUIModel
Our transformation catalog
21
Example Widget Replacement (1)
lt?xml version"1.0" encoding"UTF-8"
standalone"yes"?gt ltcuiModel name"MyModel"gt
ltversion modifDate"2004-03-24T170917.4020100"
xmlns""gt7lt/versiongt ltauthorName
xmlns""gtYourilt/authorNamegt ltwindow
height"500" width"600" name"Formulaire (2/5)"
id"window_1"gt ltbox relativeHeight"100"
name"box1_0" id"box1_0"gt ltbox
type"vert" name"boxTodo" id"boxTodo"gt
... ltbox type"horiz"
name"box_2_2_2_1" id"box_2_2_2_1"gt lttextCompone
nt defaultContent"Sexe" isBold"true"
id"label_2"/gt ltradioButton groupNamegrupo01"
defaultContent"Femme"
defaultState"false"
id"radiobutton_0"/gt ltradioButton
groupName"grupo01" defaultContent"Homme"

defaultState"true" id"radiobutton_1"/gt
lt/boxgt ... lt/boxgt
lt/boxgt lt/windowgt lt/cuiModelgt
Excerpt for an usiXML CUI specification.
22
Example Widget Replacement(2)
23
Example Widget Replacement(3)
The usiXML graph before aplying any rule
24
Example Widget Replacement(4)
Rule 1 Create a new comboBox with the same id
and name as the name of the group of radioButtons.
NAC
RHS
LHS
25
Example Widget Replacement(5)
Rule 1 Create a new comboBox with the same id
and name as the name of the group of radioButtons.
The usiXML graph after aplying the first rule
26
Example Widget Replacement(6)
Rule 2 Convert every radioButton within the
group x into an item for the comboBox x, we
have just created.
LHS
RHS

27
Example Widget Replacement(7)
Rule 2 Convert every radioButton within the
group x into an item for the comboBox x, we
have just created.
The usiXML graph after aplying the second rule
28
Example Widget Replacement(8)
lt?xml version"1.0" encoding"UTF-8"
standalone"yes"?gt ltcuiModel name"MyModel"gt
ltversion modifDate"2004-03-24T170917.4020100"
xmlns""gt7lt/versiongt ltauthorName
xmlns""gtYourilt/authorNamegt ltwindow
height"500" width"600" name"Formulaire (2/5)"
id"window_1"gt ltbox relativeHeight"100"
name"box1_0" id"box1_0"gt ltbox
type"vert" name"boxTodo" id"boxTodo"gt
... ltbox type"horiz"
name"box_2_2_2_1" id"box_2_2_2_1"gt lttextCompone
nt defaultContent"Sexe" isBold"true"
id"label_2"/gt ltcomboBox id"comboBox001"
name"label_3" isDropDown"true"gt ltitem
id"radiobutton_0" name"radiobutton_0"
defaultContent"Femme"/gt ltitem
id"radiobutton_1" name"radiobutton_1"
defaultContent"Homme"/gt lt/comboBoxgt
... lt/boxgt
lt/boxgt lt/windowgt lt/cuiModelgt
Excerpt from the final transformated usiXML
specification
29
Example Widget Replacement(10)
30
Case Studies

• 2 case studies were considered
• Modus Operandi

31
TransformiXML API/GUI

• API set of transformation functions
• GUI (Video)

32
Tool Support

• Prototypes
• TransformiXML API/GUI transformation tool
  (previously shown)
• IdealXML Task, Domain and AUI editors
• GrafiXML CUI Hi-Fi Code Generator (Java
  Swing, XHTML)
• VisiXML CUI Lo-Fi, MS Visio Plug-in
• FlashiXML flash renderer
• ReversiXML reverse engineering from HTML to CUI

33
Tools IdealXML
Demo
34
Tools GrafiXML
35
Tools VisiXML
36
Tools FlashiXML
Demo
37
Conclusion Validation

• External Feasiblity of the approach has been
  exposed by the realization of two case studies
  and the provision of transformation catalogs
• Internal
• Ontological requirements
• Expliciteness, Expressivity, Human readability,
  Formality, Machine readability, Separation of
  concerns, Verifiability, Homogeneity, Reuse,
  Extendibility, Standards
• Methodological requirements
• Expliciteness, Flexibility, Formality,
  Executability, Separation of concerns,
  Extendibility Traceability, Correctness, Tool
  interoperability, Reuse.

38
Conclusion Main Contributions

• Ontological contribution
• Definition of a formal language for describing
  viewpoints on a UI system
• Methodological contribution
• Explicit, open and formal definition of
  development steps and paths
• Unification of several problem area in a unique
  framework thanks to the paradigm of multi-path
  development of user interface
• Communication and incremental research and
  development effort

39
Conclusion Secondary contributions

• For SE community
• Application of transformational development
  concepts to UIs.
• Graph Transformation Community
• Application of theoretical concepts to a new
  problem domain

40
Obstacles and Shortcomings

• Main obtacles and shortcomings
• Ontological
• Lack of expressivity of models
• Methodological
• Complexity of certain sub-steps
• Difficulty of finding an approprite level of
  generalization
• Difficulty in ordering rules and steps
• Lack of disjunction in the rule expression
• Difficulty to get meta-information
• Difficulty in managing large sets of
  transformation catalogs

41
Future Works

• Extension to other types of UIs
• Extension to other types of models
• Define high-level building blocks (constructive
  patterns)
• Extension of methodological steps definition and
  transformation catalogs
• Extend formal foundations
• Validate catalogs by human factors experts
• Hide complexity
• Evaluate transformation for run-time scenarios
• Continue the development of ongoing tools

42
Thank You !