Intuitive Communication through

1
"Seeing is Believing"
Intuitive Communication through
Visual Languages
Monojit Choudhury Research Scholar, CSE Indian
Institute of Technology Kharagpur, INDIA - 721302
2
A picture is worth

• A thousand words?
• Or
• A million words?
• Or
• An infinite number of words?
• literally indescribable

3
(No Transcript)
4
So a Visual Medium is

• Semantically dense
• We can focus on and zoom in any part and gather
  new information
• Multi-dimensional
• Whereas, natural language (speech text) is
  one-dimensional

5
Why not VLs?

• In fact, we do use VLs
• Here, we explore
• The evolution of VLs From cave paintings to
  computer desktops
• VLs Computers How the tryst can empower
  society
• Design, Cognition Computation

6

• Looking through the glasses of
  TIME

The
Evolution
of Visual Languages
7
Earliest Expressions
8
Maps Mudras
9
Methods Molecules
10
Looks Gestures
11
Virtual Reality
12
Empowering the Society
Visual Languages
Computers
13
Why Visual Languages?

• Intuitive easy to remember
• Can be interpreted faster
• Low learning cost
• Easy to use with the help of a computer
• Transcend the language barrier

14
VLs in Education

• Effective teaching medium for
• children
• pre-literate adults
• physically challenged
• Technical education
• Karnaugh Map
• Phasor diagrams

15
VLs in Communication

• Transcend the Language Barrier

Domain specific cross-cultural inter-lingual
communication An easy way out for machine
translation
Vedo-vidi developed by Steven Tanimoto et al.
aimed at human communication on the internet
(domaintravel)
Helping the neo-literate and preliterate people
in specialized and formal communication
16
VLs in AAC
Intuitive easy to use for people with
neuro-motor disabilities

• Clicker4 (left), MinSpeak, MindExpress,
  Sanyog are some examples of AAC devices using
  Visual Languages

17
VLs in HCI

• HCI uses Metaphors for visual representations
• Visual Programming
• Visual Query Languages
• Desktops Computer Icons

18
VLs for Visualization

• Process Visualization
• Algorithms, experiments
• Data Visualization
• Graphs, charts, diagrams
• Includes
• Graph drawing aesthetics
• Animations

19
Design
Cognition
Computation
20
The Model Beneath
Design
Visualization Creation/manipulation
V i s u a l I n t e r f a c e
Perception
Reasoning
Symbolic Processing
Parsing Interpretation
Creation/manipulation
Cognitive Cycle
Computational Cycle
21
VLs are Multi-dimensional
Linguistics
Design
Semiotics
Aesthetics
Philosophy
Visual Language
Human Computer Interaction
Artificial Intelligence
Neuro- biology
Cognitive Psychology
22
Atoms of VL
Flames/Fire Icons
Hot/Inflammable Index
Purity Symbol

• Issues
• Associated learning cost
• Ideal mixture

23
The Pieces of the Puzzle
Visual Language
Modelling Computation
Evaluation
Design
Definition
Cognitive Experiments
Complexity Decidability
Icon Design
Grammar
Parsing
Test-bench design
Vocabulary
24
A Philosophical Question

• How do we reason about diagrams?
• Symbolic Processing?
• map to string languages
• model using grammars (CFG, PG etc)
• parse using standard techniques
• But images are not strings!
• Are we looking for a PARADIGM SHIFT?

25
A Look at the future

• nothing even convincing, much less exciting,
  has yet emerged from such efforts on visual
  programming I am persuaded that nothing will
• F. P. Brooks. No Silver Bullet Essence and
  accidents of Software Engineering.
• IEEE Computer, 1987

26
A Look at the future

• Overall, we believe that in many respects
  visual programming languages offer substantial
  gains over conventional textual languages, but at
  present their HCI aspects are still
  under-developed
• T. R. G. Green M. Petre. Usability analysis of
  visual programming environments A cognitive
  dimensions framework.
• Journal of Visual Languages Computing, 1996

27
A Look at the future

• Only by acknowledging the cognitive reality in
  which the language is employed by programmers and
  end users, can we recognize that visual
  programming does introduce a new paradigm.
• D. E. Mahling D. L. Fisher. The cognitive
  engineering of visual languages.
• Journal of Visual Languages Computing, 1990

28
THANK YOU