Biomimetic Sensing for Robotic Manipulation

1
Biomimetic Sensing for Robotic Manipulation
Neil Petroff, Ph. D. Candidate University of
Notre Dame
Lerner Research Institute Cleveland, OH December
8, 2005
2
Outline

• Me on Me
• Grasping
• biology as motivation for current work
• Robotic Manipulation
• Nonholonomic motion planning
• Motion planning for stratified systems
• Open-Chain Manipulators
• Forward kinematics
• Inverse kinematics
• Biomimetic Robot Sensors
• Vision, touch
• Control Perspective on Deep Brain Stimulation
• The Rest of the Story

3
Hand Orthosis
Target Group C5 - C7 SCI

• 3 Grasps
• Fingertip, key, cylindrical
• Increase Autonomy
• Mercury Orthotics
• Rehabilitation technology
• therapeutic
• quality of life

4
Grasping

• Interaction
• Creation
• Task Execution

Grasping Hand Orthosis Robotic
Manipulation Fuzzy Logic Open-Chain
Manipulators Biomimetic Robot Sensors
Work to Date
5
Grasping
Robots Humans
Poor at fine motion good at fine motion
No feedback vision, proprioception
structured adaptive
precise robust
rapid slow
strong variable
stamina need to rest
Can we improve robotic manipulation by imbuing
robots with useful human characteristics?
Grasping Hand Orthosis Robotic
Manipulation Fuzzy Logic Open-Chain
Manipulators Biomimetic Robot Sensors
Work to Date
6
Biological Motivation

• Haptic Recognition
• Force feedback
• Compliance is Useful for Manipulation
• Brain Model
• Fuzzy logic
• Hierarchical Control

Grasping Hand Orthosis Robotic
Manipulation Fuzzy Logic Open-Chain
Manipulators Biomimetic Robot Sensors
Work to Date
7
Biological Control Loop
current configuration
desired task
motion planning algorithm
inverse kinematics
encoder counts
PID
Robot
fuzzy supervisor
encoder counts
sensor readings
trajectory adjustment
8
Testbed
9
Robotic Motion Planning

• Steering Using Piecewise Constant Inputs
• This is a geometric analysis
• Provides a systematic approach for establishing
  controllability
• Applicable to underactuated systems with
  nonholonomic constraints
• Exact for nilpotent systems of the form
• Driftless
• Not all gis may exist
• a system is nilpotent if all Lie brackets greater
  than a certain order are zero
• Lie bracket motions
• allows the system to move in a new direction

10
Lie Bracket Motions

• Flow along g3 can be approximated by flowing
  along g1 and g2

Higher order brackets can be generated, e.g.
11
Example
Parallel parking a car
12
Example
Car equations
l

g1
g2
Extended System
13
Car Simulation
14
Why Didnt it Work?

• The Car Model is not Nilpotent
• g5 points in the same direction as g3
• Motion along lower order brackets induces motion
  along higher order brackets
• Solution
• Iterate
• Feedback nilpotentization
• Other Drawbacks
• Small Time or Small Inputs
• obstacle avoidance
• Open Loop
• highly susceptible to modeling errors
• no error correction

15
Stratified Systems

• Extends motion planning algorithm to systems with
  discontinuities
• Intermittent contact
• locomotion
• manipulation

16
Control Architecture
Desired task
motion planning algorithm
17
Open-Chain Manipulators
Forward kinematics
P
s
T
18
Inverse Kinematics

• The inverse kinematics solution is not unique

1
1
1
1
19
Inverse Kinematics

• PUMA geometry makes an analytical solution
  tractable

20
Inverse Kinematics
14 diameter circle
21
Control Architecture
Desired task
motion planning algorithm
inverse kinematics
current configuration
encoder counts
PID
Robot
current counts
fuzzy supervisor
22
Biomimetic Sensing
23
Force Sensors

• Feedback at Finger/Object Junction
• Piezoelectric
• Used in biomedical testing
• Compliant
• Tend to drift under static load
• Flexiforce Sensor

24
Finding an Object
25
Control Architecture
current configuration
desired task
motion planning algorithm
inverse kinematics
encoder counts
PID
Robot
fuzzy supervisor
encoder counts
sensor readings
trajectory adjustment
26
Summary

• So Far
• Built a closed loop system to perform robotic
  manipulation
• stratified motion planning
• inverse kinematics solution
• force feedback
• To Do
• Manipulation
• Currently working on simulation
• apply to robots

27
Control Perspective on DBS(or What the heck am
I doing here?)

• Underlying manipulation technique is a geometric
  approach to nonlinear controls
• Nonlinear control lies at the forefront of modern
  control methods
• One of the most intriguing aspects of
  nonlinearity is that of chaos
• Nonlinear control techniques have been used to
  suppress cardiac arrythmia, a chaotic process
• Is neuron transmission chaotic?
• at the heart of successful treatments using deep
  brain stimulation is the ability to control chaos
• Robust and nonlinear control techniques provide
  an analytical foundation on which to study such
  systems
• Soft computing techniques provide an additional
  approach that while not at rigorous may yield
  equal or better results

28
Open Questions on DBS

• By approaching DBS from a control Theory
  Standpoint, Can We
• Control with external stimulation locally?
• Filter the signals?
• Characterize which signals cause which
  disruptions
• stimulation can suppress dyskinesia
• tremors tend to lessen during movement
• Keep symptoms from returning with fatique?
• Muscle spasticity
• Completely eliminate meds?

29
The Rest of the Story

• 54,000 SCI
• Additional 2,800 / yr at C5 C6 level
• Parkinsons affects 750,000 1 million people in
  the U.S.
• Other Pathologies
• Hemiplegic stroke
• Multiple sclerosis
• Muscular dystrophy
• Rehab
• Funding
• Competition for startup money
• Who Can Pay?
• Hand Mentor from KMI
• 3,950
• Coverage from private insurance companies in only
  2 states
• Currently no medicare coverage
• State of Indiana Home and Community Based Care
  Act
• Provides funding for community and home-based
  care
• 2002 84 / 16
• Medicaid savings of 1,300 per client per month

30
My Plea

• As researchers, I believe we have a
  responsibility to pursue noble goals
• Obligation of the Engineer
• conscious always that my skill caries with it
  the obligation to serve humanity
• Hippocratic Oath
• I will remember that I do not treat a fever
  chart, a cancerous growth, but a sick human
  being, whose illness may affect the person's
  family and economic stability. My responsibility
  includes these related problems, if I am to care
  adequately for the sick.
• will remember that I remain a member of society,
  with special obligations to all my fellow human
  beings, those sound of mind and body as well as
  the infirm.

31
On a Lighter Note
32
Motion planning algorithm
Solve for vs from desired trajectory
Expand vector exponentials and equate coefficients
Solve for hs by equating Bs of above
33
3rd order bracket
34
Fictitious Input Flow
35
Revolute Joint Lemmas
Position Preservation
Distance Preservation
36
Stratified Motion Planning
If t4 t6 and t1 t3,
Motion planning performed on S12 with projected
vector fields
37
Contact Coordinates

• Mapping from R3gR2
• Shows evolution of finger on object
• EOMs on the sheets

38
Grasp Constraints

• End effector motion is limited due to contact
  with object
• Present control system such that it is in
  standard form
• Relative contact velocities are control inputs
• Defines joint torques

39
Extended System
Motion planning for smooth systems (extended)

• The vis are fictitious inputs
• for extended system, pick trajectory,

Can write any flow
40
Lie Bracket
Given two vector fields, g1 and g2, we
can generate a third which points in a new
direction
This is an approximation by TSE
Higher order brackets can be generated, e.g.
41
Inverse Kinematics
Finding
42
Two twists
43
Orthosis Design and Feedback
Requires joint angle feedback - difficult
Change in DIP Angle of the Second Metacarpal of
the Right Hand for Three Subjects During Flexion
(Both Angles are Relative to the MCP)
44
Measuring the MCP Angle
45
Mercury Orthotics
2004-2005 Notre Dame Business Plan Competition
Semi-Finalist Mission Provide state-of-the art
rehabilitation technology for therapeutic and
quality-of-life aid

• Problem I Rehabilitation of Hand Injuries
• Current structure of therapy is restrictive
• facility based
• requires dedicated therapist
• Problem II Assistive Aid for Long-Term Care
• Current techniques are invasive or incomplete
• Solution HandStand System
• Provides a method to automate certain therapy
  functions
• facility or home based
• Automatic storage of relevant information and
  progress assessment
• therapist is freed to focus on patient care while
  treating more patients
• Provides a noninvasive method for restoring basic
  hand function
• responds to user commands

46
Phillip Hall Basis

• Forms a basis for a Lie algebra
• A basis is like spice?
• Vector fields are elements of the algebra
• Some of the vector fields created by Lie bracket
  operations are in this basis
• These generate new directions in which a system
  can move
• Once we have enough to span the space, any point
  is reachable from any other point

47
General Solution Approach

• Determine the kinematic equations of motion,
  velocity constraints
• Determine the vector fields which annihilate the
  constraints
• Directions in which the system is able to move
• Determine the Phillip Hall basis
• Eliminate additional, linearly dependent vector
  fields
• Describe the extended system comprised of
  actual inputs and fictitious inputs generated by
  Lie bracket motions
• Define a nominal steering trajectory
• Determine the inputs
• The span of the set of remaining linearly
  independent vector fields determines the
  involutive closure,
• The dimension of equals the dimension of the
  configuration space meaning the system is small
  time, locally controllable
• Since the distribution is involutive it can be
  integrated

48
Configuration Space

• Partition of M into submanifolds
• Different EOMs on each stratum
• Restricted to each stratum - equations are smooth
• Cyclic strata switches
• manipulation

Consider the sequence

49
Fuzzy Logic
Mamdani Inference System