Practical Structural Design and Control for Digital Clay

1
Practical Structural Design and Control for
Digital Clay
PhD Defense Presentation

• Haihong Zhu

Woodruff School of Mechanical Engineering Georgia
Institute of Technology
www.imdl.gatech.edu/haihong
2
PhD Reading Committee Members

• Dr. Wayne J. Book, Chair, Advisor, ME
• Dr. Imme Ebert-Uphoff, ME
• Dr. Mark Allen, ECE
• Dr. David Rosen, ME
• Dr. Jarek Rossignac, COC

3
Outline of This Presentation

• Introduction to Digital Clay
• Cell of Digital Clay
• Cell array of Digital Clay
• Implementations of the multi-cell system
• Conclusions and recommendations

• Basic idea
• Background context
• Hardware of Digital Clay
• Control of Digital Clay
• Advantages and potential applications

• Overview objectives
• Cell level control
• Control methods
• Control states, switching logic and user gesture
  interpretation
• Experimental system results
• Displacement measurement
• PWM speed control and displacement estimation
• Non-contacting resistance displacement sensor
• Displacement sensor embedded micro actuator
• 1x5 prototype
• Summary

• Overview objectives
• N2 by 2N fluidic matrix drive
• Surface refresh methods for the fluidic matrix
  drive
• Control architecture based on fluidic matrix
  drive
• Summary

• Overview objectives
• Mechanical structure design
• Functional modules
• Realization of N2 by 2N fluidic matrix drive
• Displacement sensor embedded actuator array
  assembly
• Pressure sensor array mounting base
• Electronic system
• Functional block diagram of the electronic system
• Displacement sensor array multiplexing
• 5x5 cell array prototype
• Summary

4
Outline of Current Section

• Basic idea
• Background context
• Hardware of Digital Clay
• Control of Digital Clay
• Advantages and potential applications

Introduction to Digital Clay
5
Basic Idea
Introduction to Digital Clay

• 3D human-machine haptic interface
• Input / output using tangible 3D shape/surface
• Computer controlled
• Haptic/semi-haptic style
• Can be edited / transferred digitally

Video
Haptic? Sense of touch
6
Background Context
Introduction to Digital Clay

• Haptic manipulators
• Tactile array
• Haptic display interfaces

7
Hardware of Digital Clay
Introduction to Digital Clay

• General structure
• Formable crust
• Formable body
• Planar pin-rod array
• Composition
• Actuator array
• Sensors array
• Fluidics system
• Control system

Actuator Array
Video
Fluidics System
On-board Controller
8
Control Structure
Introduction to Digital Clay

• Cell Level Control
• Surface Level Control
• User API

User AP I
User API
Surface Level Control
Surface Level Controller
API Interface
Feedback Processor
Command Bus
Feedback Bus
Cell Level Control
Cell-Level Controller
Cell-Level Controller
Cell-Level Controller
9
Advantages and Applications
Introduction to Digital Clay

• Advantages
• Natural, direct, fast and efficient communication
  
• Unleash the mind power of creation, perception
  and intuition
• Applications
• Engineering design science research
• Medical diagnosis
• Vision Impaired assistance
• Military civil map
• Art
• Communication
• Education, Entertainment, etc.

Video
10
Outline of Current Section

• Overview objectives
• Cell level control
• Control for solenoid valve based system
• Control states, switching logic and user gesture
  interpretation
• Experimental system results
• Displacement measurement
• PWM speed control and displacement estimation
• Non-contacting resistance displacement sensor
• Displacement sensor embedded micro actuator
• 1x5 prototype
• Summary

Cell of Digital Clay
11
Overview and Objectives
Cell of Digital Clay

• Overview
• Elementary unit of Digital Clay
• Mimics a point on a material surface
• One dimensional actuation type
• Challenges
• Control
• Haptic effect compromised by on-off valves
• User gesture interpretation without other help
• Volume change using unidirectional force (push)
• No suitable displacement sensor
• Micro actuator suitable for massive production
• Objectives
• Control algorithm to mimic a point on a material
  surface
• Sensing methods
• Actuator

12
Cell Level Control (I)
Cell of Digital Clay

• Control for solenoid valve based hydraulic system
  
• Testing system setup
• Pressure surge caused by solenoid valve
• Pressure signal filtering
• Position control vs. pressure control

13
Cell Level Control (II)
Cell of Digital Clay

• Control states, switching logic and user gesture
  interpretation

14
Cell Level Control (III)
Cell of Digital Clay

• Experimental system results

• Elastic state

• Plastic state

• Keep stationary

• Shaping state

• Exit shaping state

15
Where are we?
Cell of Digital Clay

• Overview objectives
• Cell level control
• Displacement measurement
• PWM speed control and displacement estimation
• Non-contacting resistance displacement sensor
• Displacement sensor embedded micro actuator
• 1x5 prototype
• Summary

• Why is this topic important?
• Sensor and actuator are critical
• Huge number of sensors and actuators are needed
• No suitable existing products are found

16
Displacement Measurement (I)
Cell of Digital Clay

• PWM Speed Control and Displacement Estimation
• Experimental system preliminary test results
• PWM Frequency lt100 Hz High Linearity Bad haptic
  sense
• PWM Frequency gt150 Hz Low linearity Good haptic
  sense

17
Displacement Measurement (II)
Cell of Digital Clay

• PWM Speed Control and Displacement Estimation
• Analytic model used for curve fitting

• Phase I
• Flow 0
• Phase II
• Phase III
• Phase IV

! Definitions of terms can be found in the
thesis
18
Displacement Measurement (III)
Cell of Digital Clay

• PWM speed control and position estimation test
• Test setup
• Control structure
• Test result

Video
19
Displacement Measurement (IV)
Cell of Digital Clay

• Displacement Sensor Embedded Micro Actuator

• Non-contacting Resistance Sensor
• Resistance to displacement signal
• Capacitor picks up the signal
• Structure
• Micro glass tube graphite piston
• Uniform thin film deposited
• Advantages
• Ultra-compact size
• Low cost
• Interchangeable with LVDT
• Nonlinearity lt 0.5
• Resolution Theoretically Infinity

20
Where are we?
Cell of Digital Clay

• Overview objectives
• Cell level control
• Displacement measurement
• 1x5 prototype
• Summary

21
1 x 5 Cell Array Prototype of Digital Clay
Cell of Digital Clay

• A line on a material surface
• Structure Features
• Micro Solenoid Valve
• No Displacement Sensor
• SLA10120 Base
• Control
• Direct control on each cell using proposed PWM
  method

Video
22
Summary
Cell of Digital Clay

• Position control method is suitable for solenoid
  valve based hydraulic system
• Proposed control states, switching logic and user
  gesture interpretation are effective for
  hydraulic system to mimic material mechanics
  properties with haptic senses
• Novel displacement measurement methods suitable
  for large number and micro size hydraulic system
  are presented
• PWM speed control and displacement estimation
• Non-contacting resistance displacement sensor
• Displacement sensor embedded micro actuator
  (patent in application)
• 1x5 prototype gives one solution to realize the
  Digital Clay
• Good experimental system results are shown

23
Outline of Current Section

• Overview objectives
• N2 by 2N fluidic matrix drive
• Surface refresh methods for fluidic matrix drive
• Control architecture based on fluidic matrix
  drive
• Summary

Cell Array of Digital Clay
24
Overview and objectives
Cell Array of Digital Clay

• Overview
• Forms the human-machine interactive tangible
  surface
• Planar pin-rod array (bed of nails)
• Huge number of identical components involved
• Challenges and solutions
• Objectives
• Conceptual design of practical structure suitable
  to realize cell array that has huge number of
  cells
• Control architecture suitable for large scale
  subsystem array

• Practical structure at current stage of
  technology
• One dimensional actuation
• 2.5 D

• Hardware
• Raw material cost
• Manufacturing
• Structural simplicity
• Fluidic Matrix Drive
• Control
• Control resource
• Dynamic control resource allocation

25
N2 by 2N Fluidic Matrix Drive (FMD) (I)
Cell Array of Digital Clay

• 2N ( 1 or 2) control valves control an N by N
  actuator array (needs 2N2 valves usually )
• Column and row matching style
• Independently addresses every actuator
• Greatly reduced the amount of valves and control
  resourse
• example. N100, 21002 20,000 gtgt 201
• Relatively slow

26
N2 by 2N Fluidic Matrix Drive (FMD) (II)
Cell Array of Digital Clay

• Working principle of the control adapter

Control Adapter
Column Control Valve
Pressure Selection Valve
Row Control Valve
High Low Control Pressure
27
Surface Refresh Methods for FMD (I)
Cell Array of Digital Clay

• Model of the FMD Node

Flow rate
Actuator displacement

• d1 and d2 are the PWM duty cycles applied to the
  valves
• k is a constant
• PWM waves are of the same phase

28
Surface Refresh Methods for FMD (II)
Cell Array of Digital Clay

• Reducing the FMD node model
• Keep row control valve only on or off (PWM duty
  cycle 0 or 100)

29
Surface Refresh Methods for FMD (III)
Cell Array of Digital Clay

• Matrix representation of surface refresh
• Working surface matrix representation
• Surface refresh

• a and ß are the PWM duty cycle vectors applied on
  the column and row control valve arrays

30
Surface Refresh Methods for FMD (IV)
Cell Array of Digital Clay

• One-time refresh method
• Process
• Fully open one row valve
• Control the column valve array until the
  actuators in that row reach the desired final
  position
• Close that row valve
• Open the next row valve and repeat step 2.
• Advantage and disadvantages
• Simple
• Slow
• Bad visual effect and haptic effect

In this example, actual total time taken is
around 3.5 seconds
31
Surface Refresh Methods for FMD (V)
Cell Array of Digital Clay

• Gradual refresh method
• Process
• Divide the desired final surface into several
  intermediate surfaces
• Use one-time refresh method to achieve each
  intermediate surfaces.
• Advantage and disadvantages
• Good visual effect and haptic effect
• Relatively complicated
• Slower

In this example, actual total time taken is
around 5.5 seconds
32
Surface Refresh Methods for FMD (VI)
Cell Array of Digital Clay

• Gradual approximation refresh method
• Process
• Divide the desired final surface into several
  intermediate surfaces
• Decompose and translate intermediate surfaces
  into certain sub-surfaces
• Realize each sub-surface once a time
• When realizing each sub surface all the valves
  are activated at certain PWM duty cycle.
• Advantage and disadvantages
• Good visual effect and haptic effect
• Most complicated
• Very fast
• Need further research

In this example, total time taken is around 1
second
33
Control Architecture Based on FMD (I)
Cell Array of Digital Clay

• Cell level control
• Surface refresh coordinator
• Dynamic control resource allocation
• Hot area processor

34
Control Architecture Based on FMD
Cell Array of Digital Clay
Desired material property for each cell to
simulate
Desired actuator arrays displacement matrix and
the speed to achieve the displacement
The matrix contains the information of user
actions and intentions for each cell in the hot
area
Desired MP
GUI
User API
CAD Model
Other User Inputs
User Motion
Desired X V
Current X , P
To compensate the surface discrepancy caused by
the delayed refresh on the surface other than the
hot area

• Surface Level Control
• Position matrix decomposition
• Contact detection
• Control source allocation
• Aftermath compensation

Signal to tell surface refresh coordinator lower
down its priority, and tell Hot area processor to
work
Desired actuator arrays displacement matrix in
the next surface refresh cycle
Current X P
Hot areas locations, sizes, etc.
Desired ?X
Compensation matrix
User Motion
Contact Process Signal
Contact Process information
Multiplexers Feedback Processor

• Surface Refresh Coordinator
• PWM vector generation
• Compensate structural variation

Current X
PWM Vector
Current actuator arrays displacement matrix and
pressure matrix
Valve Controllers

• Hot Area Processor
• Haptic reaction
• PWM vector generation
• Aftermath compensation

Current X P
PWM Vector
Valve Controllers
35
Summary
Cell Array of Digital Clay

• N2 by 2N fluidic matrix drive is novel and has
  great benefits for large scale fluidic subsystem
  array. (Patent application)
• Greatly reduces the control valves and control
  channels needed
• Makes the cell array (with huge number of units)
  practical
• Relatively slow speed maybe compensated using
  proper surface refresh method
• Suitable surface refresh methods for fluidic
  matrix drive make it possible for the system
  using FMD to achieve smooth and fast surface
  refresh.
• Carefully designed control architecture for FMD
  can both reduced hardware cost and computing
  resource.

36
Outline of Current Section

• Overview objectives
• Mechanical structure design
• Functional modules
• Realization of N2 by 2N fluidic matrix drive
• Displacement sensor embedded actuator array
  assembly
• Pressure sensor array mounting base
• Electronic system
• Functional block diagram of the electronic system
• Displacement sensor array multiplexing
• 5x5 cell array prototype
• Summary

Implementations of the Multi-cell System
37
Overview objectives
Implementations of the Multi-cell System

• Practical structural implementation
• Aims at N by N cell array
• Challenges
• Objectives
• Design for manufacturing
• Modular design
• Structural simplicity
• Design for mass production
• Design for scalability
• Structural expandable
• Size and resolution scalable
• Vertically modular

• Large number of identical components
• Material cost, fabrication cost, assemble cost
• Manufacture and assemble difficulty
• Large number of feedback measurements
• Hardware cost
• DAQ resource limitation

38
Mechanical Structure Design (I)
Implementations of the Multi-cell System

• Functional Modules
• Row control hydraulic board
• Column control hydraulic board
• Pressure sensor array assembly
• Fluidic channel concentrating block
• Actuator-sensor array assembly

Actuator-sensor Array Assembly
Row Control Valves
Column Control Valves
Fluidic Channel Concentrating Block
Pressure Sensor Array Assembly
Column Control Hydraulic Board
Row Control Hydraulic Board
39
Mechanical Structure Design (II)
Implementations of the Multi-cell System

• Realization of N2 by 2N fluidic matrix drive
• Design of the control adapter

40
Mechanical Structure Design (III)
Implementations of the Multi-cell System

• Displacement sensor embedded actuator array
  assembly

41
Mechanical Structure Design (IV)
Implementations of the Multi-cell System

• Pressure sensor array mounting base

42
Electronic System (I)
Implementations of the Multi-cell System

• Functional block diagram of the electronic system

43
Electronic System (II)
Implementations of the Multi-cell System

• Displacement sensor array multiplexing
• Simple multiplexing scheme

44
Electronic System (III)
Implementations of the Multi-cell System

• Displacement sensor array multiplexing
• Simple multiplexing scheme results

45
Electronic System (IV)
Implementations of the Multi-cell System

• Displacement sensor array multiplexing
• Multiplexing scheme using grounding resistor

46
Electronic System (V)
Implementations of the Multi-cell System

• Displacement sensor array multiplexing
• Results of multiplexing scheme using grounding
  resistor

47
Electronic System (VI)
Implementations of the Multi-cell System

• Displacement sensor array multiplexing
• Other multiplexing schemes
• Numerically compensate
• Two Digital Switches

48
5x5 Cell Array Prototype
Implementations of the Multi-cell System

• Designed and controlled using proposed solutions
• Key Features
• Stereolithography Technology
• 5 x 5 actuators in a linear pattern
• Grid size (center to center) is 5mm
• Hydraulic Matrix Drive
• Non contacting resistive sensors and modified
  pressure sensors
• Reduced control signals for multiplexers
• Controlled by RT Linux on a host PC

49
Summary
Implementations of the Multi-cell System

• Vertically modular design reduces the complexity
  of fabrication and assembly, improves the
  reliability and convenience for maintain, and
  suitable for mass production.
• Successfully Designed FMD control adapter
  realized the concept of FMD
• Displacement sensor embedded actuator array
  assembly makes fabrication of large number
  actuator-sensor array become simple and fast.
• Pressure sensor array mounting technology reduces
  the cost and makes the multi-cell system
  expandable.
• Carefully design electronic system can reduce the
  complexity of the control hardware, amount of
  components and improve the feasibility of
  realizing the Digital Clay.
• Displacement sensor array multiplexing using
  grounding resistor is a simple but effective way
  to realize the large scale multiplexing.
• 5x5 cell array prototype is designed under the
  guidelines suitable for N by N cell array, and
  can be expanded to larger size array. Test
  results preliminarily validated the design and
  control methods presented.

50
Conclusions (I)

• System development
• Cell level control architecture and realization
• Haptic control for solenoid valve based hydraulic
  system (Position control, shaping state, user
  gesture interpretation)
• Surface refresh methods for FMD
• Surface level control architecture based on the
  FMD
• Vertical modular design for multi-cell system
• Key components design
• Displacement sensor embedded micro actuator
• Fluidic matrix drive for multi-cell system with
  huge number of cells
• Pressure sensor array assembly

51
Conclusions (II)

• Measurement technology
• PWM displacement estimation
• Non-contacting displacement sensing
• Multiplexing technology for huge amount AC
  signals
• Control signal reducing for sensor arrays

52
Conclusions (III)

• Prototype development and manufacturing process
• Single cell system prototype validates the cell
  level control for single cell system
• 1x5 cell array prototype validates the PWM
  control method and the horizontal modular design
• 10x10 cell array prototype validates the concept
  of FMD
• 5x5 cell array prototype validates the N x N
  planar pin-rod Digital Clay structure and control
  
• Micro displacement sensor actuator array mass
  production process Key step to the success of
  Digital Clay realization
• Pressure sensor array assemble process Key step
  to the success of Digital Clay realization
• FMD realization using SLA technology Key step to
  the success of Digital Clay realization

53
Recommendations on Future Work

• Actuator and sensors
• Current displacement sensor needs further
  comprehensive test
• Further investigations on the assembly of the
  pressure sensors array
• Micro valve for the Fluidic Matrix Drive
• Embed MEMS valve into the cell array system using
  fluidic matrix drive
• Refresh method for Fluidic Matrix Drive
• Gradual approximation refresh method shows
  promising merits, but the matrix decomposition
  needs to be solved before implementation
• Other general topics
• Control architecture for 2 valves per cell
  driving scheme
• The manufacturing process to realize multi-cell
  array system

54
Questions Answers