Glove Technology

1
Glove Technology
Marlon L. Smith CS 4473 August 1999
2

• Agenda
• Glove data-input devicesApplications
• Why glove data-input devices?
• What do they measure?
• How do they work?
• History
• Product Comparisons
• - VPL DataGlove
• - Exos Dextrous Hand Master
• - Mattel/Nintendo/AGE Inc. PowerGlove

3
(Product Comparisons continued) -
Virtual Technologies CyberGlove - FakeSpace
PINCH Gloves - Fifth Dimension Technologies
5DT Data Glove
4

• Glove Data-Input Devices--Applications
• Starting a program by pointing your finger at an
  icon on your
• screen, then closing the program by waving
  goodbye.
• A vocally-impaired person having a device which
  translates
• sign language into sound.
• A piece of equipment to register a doctor's hand
  motion and
• allow him to perform remote surgery on a patient
  miles away.

Researchers have worked long and hard to make
such scenarios come true. There are a variety of
glove-like devices currently available which
provide for complex data entry and manipulation
by hand gestures.
5

• Why use a glove device?
• Traditional data input devices have a limited
  range of the amount of data they can input at a
  given time because they are limited to one, two
  or three degrees of freedom.
• Degrees of freedom are a measure of the number
  of positions at which the device can be read as
  inputting a different data value.
• Gloves offer far superior data input potential
  since they provide multiple degrees of freedom
  for each finger and the hand as a whole.

6

• By tracking orientation of fingers and relative
  position of hand, glove devices can track an
  enormous variety of gestures, each of which
  corresponds to a different type of data entry.
• This gives the glove remarkably rich expressive
  power, which can be used in the inputting of
  extremely complicated data.

7

• What do glove devices measure?
• Measure finger flexure and hand orientation to a
  greater or
• lesser extent. Each type of glove measures
  either four or five of the fingers
• (the pinky finger is sometimes excluded) for the
  degree of
• flexure. Measurements range from the fingers
  being extended
• straight in a line with the palm to being curled
  up against the
• palm, as in making a fist.
• Gloves can track hand orientation by measuring
  roll, pitch
• and yaw or position of the hand as a whole.

8
How do gloves work?
All data glove devices track the orientation of
the hand and fingers using either fiber optics,
ultrasonics, magnetics, electrical resistance, or
some combination of these methods. However, any
glove device feeds data about hand and finger
positions to a tracker, a piece of equipment that
processes the data so that it can be understood
by the computer. The computer than matches the
orientation to a file of gestures and fires an
event corresponding to the matching gesture.
9

• Fiber Optics
• optical fibers along the fingers measure finger
  flexion
• as the fibers bend attenuating the transmitted
  light.
• signal strength for each of the fibers is sent
  to a processor
• that determines point angles based on
  precalibrations for
• each user.
• fragile and costly for mass production.

10

• Ultrasonics
• frequency above the audible range of human ear.
• calculate difference in time it takes ultrasonic
  signal
• to reach each of three sensors (microphones),
  system
• is able to triangulate hand's position.

11

• Magnetics
• by housing a small magnet within each of the
  joints, able to
• measure flexure of all three joints per finger.
• strength of magnetic signal within each joint
  varies
• according to flexure of joint and translates into
  bend of
• each finger.
• gloves often use Polhemus Tracker (magnetics) to
  track
• orientation of hand as a whole as discussed in
  class.

12

• Electrical Resistance
• uses fibers with conductive material running up
  through each finger measuring the change in
  electrical resistance as the fingers are bent.

13

• History
• The beginning
• - 1983, Digital Data Entry Glove, Dr. G.
  Grimes,
• ATT Bell Labs
• first glove-like device (cloth) onto which
  numerous touch, bend, and inertial sensors were
  sewn.
• measured finger flexure, hand-orientation and
  wrist-position, and had tactile sensors at
  fingertips.
• orientation of hand tracked by video camera
  required clear
• line-of-sight observation for the glove to
  function.

14

• designed as alternative to keyboard matched
  recognized gestures/hand orientations to specific
  characters, specifically to recognize the Single
  Hand Manual Alphabet for the American Deaf
  circuitry hard-wired to recognize 80 unique
  combinations of sensor readings to output a
  subset of the 96 printable ASCII characters a
  tool to finger-spell words.
• finger flex sensors, tactile sensors at the
  fingertips, orientation sensing and
  wrist-positioning sensors positions of sensors
  were changeable.
• US Patent 4,414,537 Patented Nov. 8, 1983
  'Digital data entry glove Gary J.Grimes, Bell
  Telephone Lab. Inc interface device' .

15
VPL DataGlove-1987

• designed by by Tom Zimmerman, produced by VPL
  Research optical flex sensor patented in 1985.
• consists of a Lycra glove with optical fibers
  running up each finger with photodiode at one end
  and light source at other.
• point angles of each finger joint must be
  calibrated for each individual user in order to
  get accurate measures.
• combines with Polhemus tracking device.

16

• monitored 10 finger joints (lower two of each
  finger, two for thumb) and six DOF of the hand's
  position and orientation (magnetic sensor on back
  of glove).
• some had abduction sensors to measure angle
  between adjacent fingers.
• formal testing and informal observations have
  shown actual flex accuracy to be closer to 5 or
  10 degrees (not accurate enough for fine
  manipulations or complex gestural recognition
  originally rated at 1 degree) flex data was
  reported in 8-bit field.
• speed of approximately 30Hz is insufficient to
  capture rapid hand motions for time-critical
  applications.
• essentially first commercially available glove
  yet priced in excess of 9000 (w/o Polhemus
  Tracker)

17
Exos Dextrous Hand Master www.exos.com (under
const.)

• lightweight aluminum exoskeleton for the hand.

• unlike DataGlove, measures flexure of all three
  finger joints via magnet housed within each
  joint.
• much more accurate than DataGlove, measures 20
  degrees of freedom with 8 bits of accuracy, at up
  to 200 Hzunrivalled at the time.
• typically uses Polhemus tracker for 6 DOF hand
  tracking.
• costs approximately 15,000.

18
Mattel/Nintendo/AGE Inc. PowerGlove- 1989

• Mattel toy company released PowerGlove as an
  accessory for Nintendo video game system.

• based on design of DataGlove, PowerGlove was
• developed by Abrams-Gentile Entertainment (AGE
  Inc.) for Mattel through a licensing agreement
  with VPL Research.
• PowerGlove flopped after short production cycle.
• users began employing it in simple virtual
  reality
• environments as cheap substitute for other glove
  devices.

19

• consists of a sturdy Lycra glove with flat
  plastic strain gauge fibers coated with
  conductive ink running up each finger measures
  change in resistance during bending to measure
  the degree of flex for the finger as a whole.
• measures bend for only one segment per finger
  and guesses at degree of flexure of other
  segments sensors on first four fingers each
  bend reported as a two-bit integer.
• employs ultrasonic system (back of glove) to
  track roll of hand (reported in one of twelve
  possible roll positions), ultrasonic transmitters
  must be oriented toward the microphones to get
  accurate reading pitching or yawing hand changes
  orientation of transmitters and signal would be
  lost by the microphones poor tracking mechanism.
  (4D - x, y, z, roll)

20

• interface boxes exist that allow easy connection
  to almost any machine through a standard RS-232
  serial port (such as UIUC's PowerGlove Serial
  Interface or Menelli Box).
• series of buttons along back of glove complete
  data entry possibilities originally designed to
  be firing and moving buttons for video games.
• Mattel claims glove is only usable at up to
  roughly 45 degrees, and within five to six feet
  of the receivers, its (x, y, z) coordinate
  information is accurate to within 0.25 inches.
• sold for between 80 to 100 new.

21
Virtual Technologies CyberGlove
www.virtex.com

• invented by James Kramer of Stanford University
  to develop forms of communication between
  vocally-impaired people and speaking people.
• measures flexure of first two knuckles of each
  finger by means of strain gauges like those used
  in PowerGlove except much more accurate uses
  patented resistive sensors (not fiber optic or
  electromagnetic).
• two models 18 DOF and 22 DOF (22 measures
  fingertip joint).

22

• measures abduction between fingers and a number
  of additional measures around thumb (since it has
  5 DOF) measures wrist.
• combines with Polhemus or Ascension Tracker.
• award-winning instrumented glove.
• available for both hands.
• requires calibration.
• sensor resolution of 0.5 degrees.
• at 115200 baud, update rate of 112 records/sec
  filtered, 149 records/sec unfiltered (update rate
  of approximately 25 to 30 frames per second on an
  SGI Indigo2 Extreme showing two low resolution
  hands with communication at 38400 baud) computer
  hardware is limiting factor.

23

• GesturePlus lets you train it to recognize hand
  formations that are of interest to you when user
  makes one of the gestures, it sends a byte to the
  host computer telling it which gesture was
  recognized then host program takes prescribed
  action.

24

• CyberTouch is a tactile feedback option
  featuring vibrotactile stimulators on each finger
  and the palm of the CyberGlove stimulators can
  be individually programmed to vary strength of
  touch sensation array of stimulators can
  generate simple sensations such as pulses or
  sustained vibration, and can be combined to
  produce complex tactile feedback patterns
  effective for simulating perception of touching
  solid object in simulated virtual world.

25

• CyberGrasp provides haptic interface for entire
  hand providing realistic force feedback via
  force-reflecting exoskeleton that fits over a
  CyberGlove can be programmed to prevent user's
  fingers from penetrating or crushing a virtual
  object forces can be specified individually
  allows full range-of-motion of the hand and does
  not obstruct the wearer's movements.
• available for IRIX and for Windows NT.
• written in C supports OpenGL and Performer.
• costs around 6,000.

26
FakeSpace PINCH Gloves www.fakespace.com

• gesture recognition system to allow users to
  work within virtual environment.
• uses cloth gloves with electrical sensors in
  each fingertip contact between any two or more
  digits completes conductive path, and a complex
  variety of actions based on these simple "pinch"
  gestures can be programmed into applications.
• compatible with Ascension and Polhemus trackers.
• no calibration since not measuring anything.

27

• baud rates up to 19200.
• costs around 1950.

28
Fifth Dimension Technologies 5DT DataGlove 1995
www.5dt.com

• correspondingly less accurate, with only finger
  bend measuring (one sensor per finger) and no
  thumb flex or abduction measures 8-bit
  fiber-optic resolution for each finger.
• measures roll and pitch of a user's hand with
  2-axis built-in
• tilt sensor for 60 degree range.
• 19.2 kbaud (full duplex) sampling rate

29

• right hand and left hand gloves.
• requires calibration.
• costs around 485.

30
Recommendation Buy the award-winning Virtual
Technologies CyberGlove.
31
References AKA90 David W. Aha, Dennis Kibler,
and Marc K. Albert. Instance-based learning
algorithms. Draft submission to Machine Learning,
1990. Aus95 Deafness Resources Australia. Sign
language and the deaf community course notes,
July 1995. Address available in
http//www.cse.unsw.edu.au/ waleed/local_deaf.html
. Bol80 R. A. Bolt. Put that there Voice and
gesture at the graphics interface. In Proceedings
SIGGRAPH. ACM Press, 1980. CH67 T. M. Cover and
P. E. Hart. Nearest neighbour pattern
classification. IEEE Transactions on Information
Theory, IT-13(1)21--27, January 1967. Cha93
Eugene Charniak. Statistical Language Learning.
MIT Press, 1993.
32
CM92 C. Charayaphan and A. Marble. Image
Processing system for interpreting motion in
American Sign Language. Journal of Biomedical
Engineering, 14419--425, September 1992. Cov68
Thomas M. Cover. Estimation by the nearest
neighbour rule. IEEE Transactions on Information
Theory, IT-14(1)50--55, January 1968. DH94
Brigitte Dorner and Eli Hagen. Towards an
American Sign Language Interface. Artificial
Intelligence Review, 8(2--3)235--253, 1994.
Dor94 Brigitte Dorner. Chasing the Colour
Glove Visual hand tracking. Master's thesis,
Simon Fraser University, 1994. Available at
ftp//fas.sfu.ca/pub/thesis/1994/BrigitteDornerMS
c.ps.
33
DS93 James Davis and Mubarak Shah. Gesture
recognition. Technical Report CS-TR-93-11,
University of Central Florida, 1993. Fel94 S.
Sidney Fels. Glove-TalkII Mapping Hand Gestures
to Speech Using Neural Networks -- An Approach to
Building Adaptive Interfaces. PhD thesis,
Computer Science Department, University of
Toronto, 1994. FH93 S. S. Fels and G. Hinton.
GloveTalk A neural network inteface between a
DataGlove and a speech synthesiser. IEEE
Transactions on Neural Networks, 42--8, 1993.
Gat72 Geoffrey E. Gates. The reduced nearest
neighbour rule. IEEE Transactions on Information
Theory, pages 431--433, May 1972.
34
Gri83 G. Grimes. Digital Data Entry Glove
interface device. Patent 4,414,537, AT T Bell
Labs, November 1983. Hag94 Eli Hagen. A
flexible American Sign Language interface to
deductive databases. Master's thesis, Computer
Science, Simon Fraser University, 1994. Available
at ftp//fas.sfu.cs/pub/theses/1994/EliHagenMSc.ps
. Har68 Peter E. Hart. The condensed nearest
neighbour rule. IEEE Transactions on Information
Theory, pages 516--517, May 1968. HB78 D. J.
Hand and B. G. Batchelor. Experiments on the
edited condensed nearest neighbor rule.
Information Sciences, 14171--180, 1978.
35
HSM94 Chris Hand, Ian Sexton, and Michael
Mullan. A linguistic approach to the recognition
of hand gestures. In Designing Future Interaction
Conference. Ergonomics Society/IEE, April 1994.
Also available at http//www.cms.dmu.ac.uk/People/
cph/Publications/DFI94/gestures.html ITK92
Koichi Ishibuchi, Haruo Takemura, and Kumio
Kishino. Real-time hand shape recognition using a
pipe-line image processor. In Proceedings of the
IEEE International Workshop on Robot and Human
communications, pages 111--116, 1992. JKP94
George H. John, Ron Kohavi, and Karl Pfleger.
Irrelevant features and the subset selection
problem. In Proceedings of the Machine Learning
Conference, pages 121--129, 1994.
36
Joh89 Trevor Johnston. Auslan Dictionary a
Dictionary of the Sign Language of the Australian
Deaf Community. Deafness Resources Australia Ltd,
1989. JRC88 Raymond C. Jeanes, Brian E.
Reynolds, and Bernadette C. Coleman. Basic Signs
for Communication with the Deaf. Deafness
Resources Australia, 1988. Reprinted extracts
from the Dictionary of Australasian Signs with
permission from the Victorian School for Deaf
Children. KL89 Jim Kramer and Larry Leifer. The
Talking Glove A speaking aid for non-vocal deaf
and deaf-blind individuals. In Proceedings of
RESNA 12th Annual Conference, pages 471--472,
1989. KL90 James Kramer and Larry J. Leifer. A
Talking Glove'' for nonverbal deaf individuals.
Technical Report CDR TR 1990 0312, Centre For
Design Research -- Stanford University, 1990.
37
Kra91 Jim Kramer. Communication system for deaf,
deaf-blind and non-vocal individuals using
instrumented gloves. Patent 5,047,952, Virtual
Technologies, 1991. Kru90 Myron W. Krueger.
Artificial Reality. Addison-Wesley, Reading
Mass., second edition, 1990. MT91 Kouichi
Murakami and Hitomi Taguchi. Gesture recognition
using recurrent neural networks. In CHI '91
Conference Proceedings, pages 237--242. Human
Interface Laboratory, Fujitsu Laboratories, ACM,
1991. OTK91 T. Onishi, H. Takemura, and E.
Kishino. A study of human gesture recognition for
an interactive environment. In 7th Symposium on
Human Interaction, pages 691--696, 1991.
38
PW90 Randy Pausch and R. D. Williams. Tailor
Creating custom user interfaces based on gesture.
Technical Report TR-90-06, University of
Virginia, 1990. PW91 R. Pausch and R. D.
Williams. Giving CANDY to children User-tailored
gesture input driving an articular-based speech
synthesiser. Technical Report TR-91-23,
University of Virginia, 1991. Rhe91 Howard
Rheingold. Virtual Reality. Touchstone Books,
1991. Rub90 Dean Rubine. The Automatic
Recognition of Gestures. PhD thesis, Computer
Science, Carnegie-Mellon University, 1990. SP95
Thad Starner and Alex Pentland. Visual
recognition of American Sign Language using
Hidden Markov Models. Technical Report TR-306,
Media Lab, MIT, 1995. Available at
ftp//whitechapel.media.mit.edu/pub/tech-reports/T
R-306.ps.Z.
39
Squ94 Brett Squires. Automatic speaker
recognition, an application for machine learning.
Undergraduate thesis, University of New South
Wales, 1994. Sta95 Thad Starner. Visual
recognition of American Sign Language using
Hidden Markov Models. Master's thesis, MIT Media
Lab, July 1995. Available at ftp//whitechapel.me
dia.mit.edu/pub/tech-reports/TR-316.ps.Z. Stu92
D. J. Sturman. Whole Hand Input. PhD thesis,
MIT, February 1992. Available at
ftp//media.mit.edu/pub/sturman/WholeHandInput/.
Stu94 Student Chapter of the Association for
Computing Machinery. Power Glove Serial
Interface. Student Chapter of the Association for
Computing Machinery, UIUC, 2.0 edition, 1994.
40
SZ94 D. J. Sturman and D. Zeltzer. A survey of
glove-based input. IEEE Computer Graphics and
Applications, 14(1)30--39, January 1994. TK91
Tomoichi Takahashi and Fumio Kishino. Gesture
coding based in experiments with a hand gesture
interface device. SIGCHI Bulletin, 23(2)67--73,
April 1991. Uni89 Griffith University. Signs of
life. Video, 1989. Vam Peter Vamplew. The
SLARTI sign language recognition system A
progress report. Unpublished report.
41
Waron Simon Keith Warfield. Segmentation of
Magnetic Resonance Images of the Human Brain. PhD
thesis, University of New South Wales, In
preparation. Wex93 Alan Daniel Wexelblat. A
feature-based approach to continuous gesture
analysis. Master's thesis, MIT, 1993. Available
at ftp//media.mit.edu/wex/MS-Thesis.ps.gz.
WK91 Sholom M. Weiss and Casimir A. Kulikowski.
Computer Systems That Learn. Morgan Kaufman,
1991. Zim91 Thomas G. Zimmerman. Optical flex
sensor. Patent, VPL Inc, 1991. ZL87 Thomas G.
Zimmerman and Jaron Lanier. Hand gesture
interface device. In CHI GI Conference
Proceedings, pages 189--192, 1987.
42
ZL92 Thomas G Zimmerman and Jaron Lanier.
Computer data entry and manipulation apparatus
method. Patent Application 5,026,930, VPL
Research Inc, 1992.