Toolkit Details: Input

1
Toolkit Details Input Communications

• CS 160, Spring 2002
• Professor James Landay
• May 6, 2002

2
Toolkit Details Input Communications

• CS 160, Spring 2002
• Professor James Landay
• May 6, 2002

3
Outline

• Output review
• Event overview
• Windowing systems
• Window events
• Event dispatching
• Widget communication

4
Review

• Models for images ?
• strokes, pixels, regions
• what is good about the stroke model?
• saves space computation
• what cant it represent well?
• images
• what is aliasing how do we fix it?
• jaggies due to low resolution -gt anitalias
  (partially fill in adjacent pixels)
• Coordinate systems ?
• device, window, physical, model
• what is the difference between these?
• Drawing
• what abstract object do we draw on?
• canvas
• Cipping ?
• drawing only regions that are visible to the user
• Color models ?
• RGB, HSV, CMY

5
Sequential Programs

• Program takes control, prompts for input
• Examples include
• command-line prompts (DOS, UNIX)
• LISP interpreter
• The user waits on the program
• program tells user its ready for more input
• user enters more input

6
Sequential Programs (cont.)

• Architecture
• Program reads in a line of text
• Program parses the text
• Program evaluates the result
• Maybe some output
• Loop back to beginning
• But how do you model the many actions a user can
  take?
• for example, a word processor?
• need to do printing, editing, inserting, whenever
  user wants to

7
Sequential Programs (cont.)

• Usually end up with lots of modes
• lots of state variables
• Other examples of modes
• paint programs (line, bucket-fill, rectangle,
  etc)
• universal remotes with TV / VCR mode
• vi edit mode command mode
• Problems with modes?

8
Sequential Programs (cont.)

• Problems with modes?
• gets confusing if too many modes
• can be easy to make errors
• how to switch between modes?
• Need feedback as to what mode you are in
• Well need a more advanced model to simplify
  windows programming

9
Event-Driven Programming

• Instead of the user waiting on program, have the
  program wait on the user
• All communication from user to computer is done
  via events
• An event is something interesting that happens
  in the system
• mouse button goes down
• item is being dragged
• keyboard button was hit

10
Event Example
11
Major Issues

• How to decompose the UI into interactive objects?
• How to distribute input to the interactive
  objects
• How to partition between application system
  software?
• Models for programming interactive objects
• Models for communications between objects

12
Windowing Systems

• Partitioning to prevent chaos
• Infrastructure to support common services
• Two major aspects
• software services to applications
• create organize windows
• implement interaction in those windows
• window manager
• UI allowing user to control size placement of
  windows

13
Interactor Tree

• Decompose interactive objects into a tree
• interactive objects also known as widgets
• based on screen geometry of objects
• nested rectangles
• Used for dispatching events
• events are dispatched (sent) to code in widget
• the code then handles the event
• Variety of methods for dispatching events
• return to this later

14
Interactor Tree
Display Screen
F\cs160\Public window Inner Window
title bar horizontal scroll bar contents
area CDJukebox folder Home Ent
folder size control Web Newspaper
window
15
Interactor Tree
Display Screen
Outer Win black
?????
16
Interactor Tree
Display Screen
Outer Win black
Inner Win green
Result Win tan
Result String
Keypad Teal
button
- button
button
0 button
17
Interactor Tree (Java)
Display Screen
Frame black
Panel green
Text Entry tan
Result String
Keypad Panel Teal
Button()
Button(-)
Button()
Button(0)
18
Windows

• Top level windows known as root windows
• provide UI abstraction for multiple apps
• windowing system arbitrates interactive resources
• Each root window belongs to an application
• all descendant windows belong to same app.
• violated by ActiveX
• Windows vs. widgets/controls
• X, NeXTStep, MS Windows
• everything is window

19
Networked Windowing Systems

• XWindows NeWS designed to allow apps to run on
  remote machines
• Uses client-server model

20
XWindows

• Note backwards terminology
• user is on server not client
• X Server
• interprets X commands can send events
• determines which window receives events
  forwards over network to proper client
• X Client
• software interface to X (Xlib)
• assembles the output from Xlib routines into
  packets for transmission to server

21
XWindows

• Interaction Problems?

Network
X Server std system software
Client app software
User
Network Bandwidth is bits per second Network
Latency is time to transfer and process data
Latency! Relation to Model Human Processor?
22
Network Round Trips (NRT)

• Problem?
• every mouse move on thumb involves NRT
• Solutions?
• download code that knows how to scroll
• NeWS used display PostScript to do this

23
Window Events

• User interacts with input device
• action translated into software events
• must distribute events to appropriate window
• doesnt need IPC, use method/procedure call
• Events have
• type
• mouse position or character key
• the window the event is directed to

24
Input Events

• Mouse button events
• mouse up down
• modifier (shift keys, etc.)
• double click (X doesnt have this -gt fakes it)
• Mouse movement events
• implement painting with mouse
• mouse drag
• Mouse enter and exit events
• e.g., if you entered / exited a button region

25
Implementing Buttons
Button
Button
(But using mouse move events would be overkill)
26
Events (cont.)

• Keyboard events
• must translate raw scan codes into ASCII
• Windowing events on window
• creation / destruction
• opening / closing
• iconifying / deiconifying
• selection / deselection
• resize
• redraw
• redraw newly exposed portions of the window
  (rect.)

27
Main Event Loop

• Main event loop
• Initialization
• While (not time to quit)
• Get next event E
• Dispatch event E
• The meat of the program is in the code that
  handles the dispatch

28
Event Dispatch
Dispatch (event E) switch (E.window)
... case FIVE-KEY if (E.type
left-down) cur 5 10cur display
(cur) last-op NUMBER ...
Hit the 5 key
29
Event Dispatch
Dispatch (event E) switch (E.window)
... case TWO-KEY if (E.type left-down)
cur 2 10cur display (cur)
last-op NUMBER ...
Hit the 2 key
30
Event Dispatch
Dispatch (event E) switch (E.window)
... case ENTER-KEY if (E.type
left-down) push (cur) cur 0
last-op COM ...
Hit the enter key
31
Event Dispatch
Dispatch (event E) switch (E.window)
... case SIX-KEY if (E.type left-down)
cur 6 10cur display (cur)
last-op NUMBER ...
Hit the 6 key
32
Event Dispatch
Dispatch (event E) switch (E.window)
... case PLUS-WIN if (E.type
left-down) if (last-op NUMBER) push
(cur) result pop() pop() push
(result) display (result) cur
0 last-op COM
Hit the key
33
Event Dispatch
34
Administrivia

• Presentations on Wed. (random picks)
• presentations code must be locked down
• any major changes after Wed by permission only
• Final is 8-11 AM on Thur, May 23rd in ??
• Project Fair is 11-130 on Wed., May 15th in 306
  Soda (good review of class)
• we supply lunch you supply poster (template
  online)
• Any questions on project? presentations? project
  fair?

35
Event Queues

• Input events are placed in a queue
• Ensures events are processed in order
• Main event loop removes them from the queue
  (get-next-event) dispatches for processing

Mouse move (22, 33) Mouse move (40, 30) Mouse
down left (45, 34) Mouse up left (46, 35)
36
Event Queues (cont.)

• Can use event masks to filter unwanted events
• e.g., filter mouse moves in a forms-based program
• just get enter/exit events

37
Object-Oriented Event Handling

• Older methods prone to programmer error
• OO languages more naturally handle passing
  messages between independent objs
• Basis for NeXTStep, Mac App, Visual C

38
Object-Oriented Event Loop

• Toolkit defines an application class
• provides a run method which contains event loop
• technique used by Visual C and MacApp

Application myApp Intialize windows
application data structures Set any special
event masks by sending messages to
myApp myApp.Run()
39
Dispatching Events

• If user scrolls the text, the software must
• direct the mouse events to the scroll bar
• update the scroll bar display during the drag
• notify the text editing window it needs to scroll
  itself so that the text appears to have moved

40
Dispatching Events (cont.)

• Algorithm selects the bottom-most, front-most
  region in the interactor tree
• scroll bar or contents over outerwin
  (bottom-most)
• scroll bar over contents (front-most)
• each window need only consider its own events
• difficult to impose a high level of control
• known as bottom-first event dispatch
• Top-down event dispatch
• events passed to top-most, front-most window
• it dispatches to one or more of its children...

41
Widget Communication

• How do widgets communicate and interact with
  each other?
• e.g., if scrollbar is moved, corresponding text
  should also move

42
Widget Communication (cont.)

• Pseudo-events
• create new events for inter-widget communication
• send pseudo-event to the right widget
• pseudo because it does not represent real input

43
Widget Communication (cont).

• Direct sibling access
• specify sibling as property (Tk)
• specify sibling in code (subArctic, AWT)
• Example

44
Widget Communication (cont.)

• Parent notification
• each widget has a unique identifier
• widgets notify parent widget of any changes that
  have occurred
• parent does the right thing

45
Event Focus

• Where should keyboard events go?
• mouse-based
• attach mouse position to all key events and
  dispatch events in the same way as mouse events
• click-to-type (Mac)
• send all key events to last window where mouse
  down occurred
• key focus
• windows take give away keyboard focus
• Mouse focus
• long narrow scrollbar... problem?

46
Simple Event Handling

• Event tables (in the early days)
• indexed by event types (integer from 0 - 255)
• holds pointers to functions that handle each
  event
• one table per / window
• lots of things to maintain when attached to a
  widget that you want to make reusable
• Callbacks
• separate things like labels/colors into resources
  read from files
• each kind of widget defines a set of named
  callbacks which it will invoke

47
Callback Example

• How do we notify text window to scroll when the
  scroll bar is moved?
• create a vertical scroll bar widget
• write a callback procedure which has code to
  notify text windows of their new position
• register callback with scroll bar as callback to
  invoke when the scroll bar is moved
• also register a pointer to the text window as the
  callback data -gt knows which window to scroll

48
Simple Event Handling (cont.)

• WindowProc style (MS Windows)
• newer and better than older models
• define window classes, each of which have a
  WindowProc (similar to callback)
• whenever event dispatch algorithm identifies a
  window that should receive an event, that
  windows WindowProc is invoked
• body of WindowProc is a switch on the event type
  with a handler for each event
• 100s of events, but most is inherited/delegated

49
Summary

• Windowing systems
• special problem with networked WS?
• latency
• Input events, such as
• keyboard, mouse, window, etc.
• Main event loop
• used to dispatch events
• Interactor trees used for
• figuring out where to dispatch events
• Dispatching events
• two main ways
• Event focus determines
• what widget current events go to

50
Next Time

• Presentations