1Forum doctoral LouvainlaNeuve, IAG, 21 Dcembre, 2006

1
Un petit dessin vaut mieux quun long discours
Adrien Coyette Université catholique de Louvain
(UCL) School of Management (IAG) Information
Systems Unit (ISYS) Belgian Lab. of
Computer-Human Interaction (BCHI) http//www.isys.
ucl.ac.be/bchi
2
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

3
Motivations
User interface determines how easily a user may
control underlying functions of a computer
program. Thus, a program equipped with powerful
functionalities and low quality user interfaces
may be under-exploited or misused
4
Motivations
Additionally, the proportion of code dedicated to
user interface in application development is
considerable from 50 to 70 of the
application code
Myers and Al, 2000
5
Motivations
How to build a graphical user interface
efficiently?
Start ? all programs ? an Editor ? build your
UI
6
Motivations
Even if a final result is quickly available,
the UI created is likely to require a huge set of
expensive modifications There are no rewards
in life for being the first one with the wrong
answer.
Snyder, 1996
7
Motivations

• As it was the case in software engineering, HCI
  communities wanted to develop a well-structured
  method for developing user interfaces.
• However, the systematization, and the
  reproducibility found in software engineering
  methods cannot be transposed equally in HCI the
  development life cycle remains inherently open,
  ill-defined, and highly iterative

Limbourg, 2004
8
Motivations

• Designers often consider that informal techniques
  based on low fidelity mock up is the best
  alternative for the early design phase.
• It allows to cope with the fact the such process
  is eminently open, iterative and incomplete

Sumner, 1997
9
Motivations

• Such approach allows to reduce the time needed
  between the cycle
• Permit to involve the end user in the process
  since few technical background is required
• And rises as many usability problems than using
  high fidelity mock-ups

Virzi and Al, 1996
10
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

11
Paper prototyping

• What is it used for?
• Early design phase
• Get a sense of the user's needs and goals
• Quickly prune the tree of possible designs
• Remove locked-in choices

Snyder, 1996
12
Paper prototyping

• How to Create a paper prototype ?
• Paper
• Eraser
• Pencils
• Scissors
• Glue/Tape
• Post-it

Snyder, 1996
13
Paper prototyping
Snyder, 1996
14
Paper prototyping

• Paper prototypes are as much about
• seeing how the user uses the design
• as it is about seeing what ideas (user and
  designer) come up DURING the design
• Iteration is key

Snyder, 1996
15
Paper prototyping

• Paper prototyping appears thus to be the a
  natural answer
• familiar and unconstrained approach
• fast to learn and quick to produce
• focus on basic structural issues instead of
  unimportant details
• it is very appropriate to convey ongoing,
  unfinished designs, and it encourages creativity,
  
• can be performed collaboratively between
  designers and end-users

16
Design Tab

• But drawbacks also exist
• changes are hard to accomplish
• The designer has to redraw the common features
  that the design retains
• lack of interaction between the paper-based
  design and a user (a designer needs to"play
  computer" )
• Can't emulate complex interactions
• But...it isn't designed for those purposes
• It is designed to solve problems before any money
  or time is invested in a solution

17
Paper prototyping

• Designers almost never use paper prototyping with
  clients
• Why?
• Doesn't impress?
• Too simple and cheap to do anything valuable?
• Probably a little of both

Snyder, 1996
18
Paper prototyping

• The idea of developing a computer-based tool for
  sketching UIs naturally emerged from these
  observations
• Such tools would extend the advantages provided
  by sketching techniques by easily creating,
  deleting, updating or moving UI elements.
• Some research was carried out in order to propose
  a hybrid approach, combining the best of the
  hand-sketching and computer assisted interface
  design, but as we will se trough the state of the
  art, this marriage presents a set of
  shortcomings.

19
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

20
Computer Assisted Prototyping
DENIM(James A. Landay, James Lin, Mark W. Newman,
Jason I. Hong) http//dub.washington.edu/denim/
21
Computer Assisted Prototyping

• Advantages
• Support for scenario-based design
• Several levels of granularity
• Good documentation
• Toolbox of generic widgets
• Mature product, based on experimental testing
• Zooming facility from local design (e.g. a web
  page) to a global design (e.g., a portion of a
  web site or an entire web site)
• Storyboarding facilities based on patterns

22
Computer Assisted Prototyping

• Shortcomings
• No shape recognition and interpretation, thus
  loosing the effort
• No code generation
• No preview mode
• Only dedicated to web sites

23
Computer Assisted Prototyping
JavaSketchIt (Joaquim A. Jorge, Manuel João
Fonseca, Anabela Caetano, Néri Goulart
) http//immi.inesc-id.pt/project_page.php?project
_id21
24
Computer Assisted Prototyping

• Advantages
• Performance (speed and accuracy)
• Multi-stroke gestures
• Recognizes rotated shapes
• Computationally light
• Open source
• Requires standard and freely available libraries

25
Computer Assisted Prototyping

• Shortcomings
• No scenario editor
• Mono-window
• Only generates java (and no UI spec)
• Limited widget set
• Shape interpretation can only take as input a
  construct made of a maximum of two vectorial
  shapes
• No zoom

26
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

27
SketchiXML Demonstation
28
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

29
SketchiXML Development
Requirements elicitation

• Avoidance of Effort loss
• Large conceptual coverage
• Robust recognition
• Language neutrality
• Ease of use (naturalness)
• Flexible processing
• Context independency

30
SketchiXML Development
Requirements elicitation

• Context independency
• Well defined editing functionalities.
• Save and export facilities.
• Zooming capabilities
• Robust scenario editor
• Recognition flexibility

31
SketchiXML Development
SketchiXML as a Multi-Agent System

• Adaptive
• Modular
• Flexible

32
SketchiXML Development

• Recognition engine Cali library
• A fast, simple and compact approach to identify
  Scribbles (multi-stroke geometric shapes)
• Combines temporal adjacency, Fuzzy Logic and
  geometric features to classify scribbles.
• Recognizes elementary geometric shapes, such as
  Triangles, Rectangles, Diamonds, Circles and
  some gesture commands, such as, Delete, Cross,
  WavyLine, Move, Copy and Tap.
• Shapes are recognized independently of changes in
  rotation, size or number of individual strokes. 
• The recognizer has a recognition rate of 97 and
  a fast response

33
SketchiXML Development
Recognition engine Trainable Gesture Recognizer

• Handdrawn input (sketch) is superimposed with a
  grid
• Each sampling point is mapped to its closest grid
  node
• For adjacent nodes, a stroke direction (out of 8
  different directions) is taken
• A string is generated from direction coding of
  input
• Complex task of comparison of drawing inputs is
  reduced to simple calculation of string edit
  distances

34
SketchiXML Development
Levenstein  distance

• Distance is shortest sequence of edit commands
  that transform s to t
• Simplest set of operations
• Copy character from s over to t
• Delete a character in s (cost 1)
• Insert a character in t (cost 1)
• Substitute one character for another (cost 1)

35
SketchiXML Development
USIXML

• (USer Interface eXtensible Markup Language)
• XML-compliant markup language that describes the
  UI for multiple contexts of use
• Character User Interfaces, Graphical User
  Interfaces, Multimodal User Interfaces,
• Language Supporting Multi-Path Development of
  User Interfaces
• Forward engineering, Reverse engineering,
  Middle-out approach,

Limbourg, 2006
36
SketchiXML Development
Transformation paths, steps, sub-steps
Limbourg, 2006
37
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

38
Surveys
Building the widget catalogue
39
Surveys
40
Surveys
Recognition rate of the application
Overall recognition rate for widget 94.12
(only) No difference between end user and
computer scientist
41
Surveys

• Evaluation of the usability of the application
• Evaluation of the time needed to draw a simple
  form with the application
• The time needed decrease strongly as user get
  used to the application (learning effect)
• End user are significantly faster than computer
  scientist to draw the same user interface
• Evaluation of the application with a multiple
  choice questionnaire at the end of the test
• Most of the participant enjoyed the test
• Most of them found the application easy to use
• Most of the participant were satisfied with the
  results (despite the low recognition rate)

42
Presentation plan

• Motivations
• Paper prototyping
• Computer Assisted Prototyping
• SketchiXML Demonstration
• SketchiXML Development
• Surveys
• Conclusion

43
Conclusion

• We have introduced a new and innovative sketching
  tool that generates a user, platform, and
  environment independent output
• We have met most of the requirements that were
  identified as important shortcomings of existing
  tools
• we have also conducted a survey on 60 peoples
  from different activity sectors with different
  backgrounds, in order to identify how these
  people would intuitively represent the widgets to
  be handled by SketchiXML
• We have conducted a survey on 40 peoples with
  different background in order to test the
  application
• SketchiXML extends a set of tools based on
  UsiXML, allowing to initiate the design process
  from the early design phase to the final concrete
  user interface for different platforms, with
  tools support for every stages

44
Future Works

• Meet all the requirements elicited in the state
  of the art and particularly the Development of a
  scenario editor allowing to represent transition
  between screen.
• Perform larger scale user testing so as to
  evaluate the performance of the application,
  compare the time needed for a specific project
  with regard to standard approaches
• Improve performances of the differents algorithms

45
Thank you very much for your attention
http//www.usixml.org User Interface eXtensible
Markup Language
http//www.usixml.org/index.php?viewpageidpage2
9 SketchiXML home page
http//www.similar.cc European network on
Multimodal UIs
For more information and downloading,http//www.i
sys.ucl.ac.be/bchi