CS376 User Interface Toolkits

1
User Interface Toolkits
Jeffrey Heer 26 May 2009
2
Final Project Presentations

• Tuesday June 9, 330-630pm, 104 Gates
• 8 minute presentations
• 6 min for research, 2 min for questions
• Start with an overview
• 1 sentence statement of your research result
• 1 slide / 4 sentences of what you did and why
• Rest of time on details. Assume audience is
  familiar with HCI issues focus on your work

3
In the beginning
cryptonomicon.com/beginning.html
4
The Xerox Alto (1973)
5
Event-Driven UIs

• Old model (e.g., UNIX shell, DOS)
• Interaction controlled by system, user queried
  for input when needed by system
• Event-Driven Interfaces (e.g., GUIs)
• Interaction controlled by user
• System waits for user actions and then reacts
• More complicated programming and architecture

6
Periodic Table of Motif Widgets
7
Toolkit Example Java Swing

• GUI toolkit with a widget set and an API

8
User Interface Builders
9
Event Dispatch Loop

• Event Loop (runs in dedicated thread)
• Remove next event from queue
• Determine event type
• Find proper component(s)
• Invoke callbacks on components
• Repeat, or wait until event arrives

• Event Queue
• Queue of input events

Mouse moved (t0,x,y)

• Component
• Invoked callback method
• Update application state
• Request repaint, if needed

10
Event Dispatch

• Event Queue
• Mouse moved (t0,x,y)
• Mouse pressed (t1,x,y,1)
• Mouse dragged (t2,x,y,1)
• Key typed (t3, F1)
• (queues and dispatches incoming events in a
  dedicated thread)

Window
Panel
TextArea
Panel
Label
/ callback for TextArea / public void
mouseMoved(e) // process mouse moved
event
Button
Button
11
Success of Tools

• Todays tools are highly successful
• Window Managers, User Interface Toolkits, and
  Interface Builders are ubiquitous
• Most software built using them
• Are based on many years of HCI research
• Brad A. Myers. A Brief History of Human Computer
  Interaction Technology. ACM interactions. Vol.
  5, no. 2, March, 1998. pp. 44-54.

12
Why Tools?

• The quality of the interfaces will be higher.
• Why?
• Rapid prototyping.
• Easier to incorporate changes motivated by
  evaluation.
• Re-use affords investment in high quality tools.
• Consistency of interface design.
• Enable collaboration among specialists.

13
Why Tools?

• The user interface will be easier to create and
  maintain.
• Why?
• Less code to write due to component re-use.
• Better modularization, separation of concerns
• Tools may abstract complex systems or algorithms.
• Easier to port an application to different
  hardware or software environments if device
  dependencies are isolated in the user interface
  tool.

14
What should tools do?

• Help design the interface given a specification
  of the tasks.
• Help implement the interface given a design.
• Help evaluate the interface after it is designed.
• Create easy-to-use interfaces.
• Allow the designer to rapidly investigate
  different designs.
• Allow non-programmers to create user interfaces.
• Provide portability across different machines and
  devices.
• Be easy to use themselves.

15
Tools might do

• Provide sets of standard UI components
• Guide the implementation
• Help with screen layout and graphic design.
• Validate user inputs
• Handle user errors
• Handle aborting and undoing of operations
• Provide help and prompts
• Deal with field scrolling and editing
• Insulate the application from all device
  dependencies and the underlying software and
  hardware systems.
• Support features in the interface that allow the
  end user to customize the interface.

16
Application Types

• What application domains are deserving of
  specialized toolkit support?

17
Application Types

• Each has own unique UI style, and implementation
  challenges
• Word processors
• Drawing programs (CAD/CAM)
• Painting programs
• Mail readers
• Spreadsheets
• Programming environments / code editors
• WWW
• Interactive games
• Visualizations
• Automated-teller machines (ATM)
• Virtual Reality
• Multi-media
• Video
• Animation
• Controlling machinery

18
DiamondSpin Toolkit

• Toolkit for tabletop user interfaces
• Shen, Vernier, Forlines, Ringel CHI 04

19
(No Transcript)
20
Tabletop UI Needs

• Multi-user support
• Identity-aware widgets
• Multiple menus
• Public and personal spaces
• Resolving conflicting actions
• Arbitrary orientation of UI elements
• Techniques to control orientation and layout
• Rotation sensitive components

21
iStuff Toolkit

• Physical UI components for ubiquitous computing
  environments (multiple users, devices, and
  applications)
• Ballagas, Ringel, Stone, Borchers, CHI 03

22
iStuff Design

• iStuff components
• Device proxy (smarts are in the proxy)
• PatchPanel
• Translate between iStuff events and
  application-specific events
• Run-time retargetable events
• Address dimension mismatches

23
(No Transcript)
24
Discussion of Themes

• Address the useful important aspects of UIs
• Narrower tools have been more successful than
  ones that try to do everything
• Do one thing well
• Threshold / Ceiling
• Research systems often aim for high ceiling
• Successful systems often seem to instead aim for
  a low threshold
• Impossible to have both?

25
Threshold Ceiling
Programming in C
Visual Basic
Director (v6)
HyperCard
C Programming
MFC
C Programming
DifficultyofUse
xCmds
Click andCreate
Lingo
HyperTalk
Goal
Basic
Sophistication of what can be created
26
Discussion of Themes, cont.

• Path of Least Resistance
• Tools should guide implementers into better user
  interfaces
• Goal for the future do this more?
• Predictability
• Programmers do not seem willing to release
  control
• Especially when system may do sub-optimal things
• Moving Targets
• Long stability of Macintosh Desktop paradigm has
  enabled maturing of tools

27
Evaluating User Interface Tools

• An API is a user interface where programmers are
  the users
• ?? Evaluate toolkit as you would a UI?
• Factors
• Expressiveness (Ceiling)
• Development Rate (of skilled user)
• Learning Rate (to become skilled)
• Performance
• Portability

28
The Future of Interface Tools

• Supporting
• Prototyping
• Collaboration
• Evaluation
• of interfaces built with tools
• of tools themselves how to prototype, test,
  iterate?
• Emerging interface styles, such as
• mobile
• recognition-based UIs (speech, pens, vision)
• multiple devices

29
Current Research Challenges

• Complex design space
• e.g., do we have to update the toolkit every time
  someone creates a new sensor or actuator?
• Ambiguous input
• Speech, gestures, computer vision, etc. arent
  recognized as accurately as mouse clicks. Should
  the toolkit handle the recognition?

30
Summary

• Toolkits provide reusable interface components to
  simplify UI development
• Toolkit trap its tempting to only make UIs that
  the toolkit makes easy, instead of making whats
  best for a specific app
• Toolkit types
• WIMP (Garnet, Swing, Motif, etc)
• Specialty (Phidgets, iStuff, Papier-Mache,
  DiamondSpin, GroupKit, Peripheral Displays
  Toolkit, etc)

31
Next Time Adaptive Interaction

• You should use Sharons online paper viewer for
  the next set of readings.