Software Enabled Control for Intelligent Uninhabited Aerial Vehicles UAVs

1
Software Enabled Controlfor Intelligent
Uninhabited Aerial Vehicles (UAVs)

• Principal Investigators
• Georgia Tech Daniel Schrage, George Vachtsevanos
• Key Personnel
• Georgia Tech Bonnie Heck, J.V.R. Prasad, Linda
  Wills
• Boeing Bryan Doerr, Kevin Wise

http//uav.ae.gatech.edu/sec
2
Presentation Outline

• Organizational Chart
• Project Overview
• Overview of Technical Approach
• Statement of Work Tasks
• Task I Mid-level Coordination for Mode Switching
  and Reconfigurable Control
• Task II Control Integration and Simulated
  Demonstration
• Task III Vertical Take-Off Landing (VTOL) UAV
  Demo
• Issues and Concerns
• Discussion

3
Organizational Chart
4
Software Enabled Control
Project Overview
http//uav.ae.gatech.edu/sec
5
Project Objectives

• Develop software-enabled control methods for
  complex dynamic systems with application focus on
  intelligent UAVs
• Develop and implement a plug-and-play, real-time
  software architecture for SEC integration
• VTOL UAV hardware-in-the-loop simulation and
  flight test demonstration
• Collaboration with other research teams and
  toolkit providers

6
Quad Chart
Internal Failures
Situation
External Threats
NEW IDEAS
High-Level
Awareness
Fault
Reactive Control
Diagnosis

Intelligent control methods for mode switching
Mode
Fault
Real-Time Distributed
and fault tolerance in
UAVs
Tolerance
Selection
Reconfigurable
Architecture

Interchangeable control modules that allow for
Mode
Reconfigurable
Real Time Sensor
Switching
Control
Processing
changing the mission and modes quickly

Open, distributed, plug-and-play software
architecture for interoperability among
heterogeneous components
SCHEDULE
IMPACT
(2-year project)

Improved mission effectiveness for
UAVs
Months
(smoother operation, improved
Mid-level Coordination
maneuverability and robust to failures)

Rapid response to mission or operational
changes through
reconfigurable
software
architecture for
UAVs
Control Integration and Simulated Demonstration

Increased interoperability among
Multi-level controllers
heterogeneous components
Run-time infrastructure and
and sensor processing
Hardware-in-the-loop

Reduced development costs due to reuse of
software architecture developed.
modules integrated.
simulation demonstrated.
generic control algorithms and integration
Intelligent VTOL UAV Demonstration
patterns
Flight
Infrastructure developed.
Test support developed.
test.
7
The Challenges

• VTOL UAV (Yamaha R-50 helicopter)
• need for truly autonomous flight very nonlinear
  and highly coupled system limited payload
• Control Methods
• smooth mode transition health and safety issues
  quick mission change
• Software / Hardware
• computational limitations open plug-and-play
  architecture real-time requirements system
  integration

8
Georgia Tech Leveraged Programs

• Center of Excellence in Rotorcraft Technology
  (CERT)
• Since 1982 the largest and most successful
  university center in the world for conducting
  research in advancing rotorcraft technology
  currently one of three centers under the National
  Rotorcraft Technology Center (NRTC) which
  partially supports 15 faculty and 35 students
  conducting research in six different areas
  flight controls research will be directly
  leveraged
• Autonomous Scout Rotorcraft Testbed (ASRT)
• U.S. Army / Georgia Tech sponsored technology
  demonstration (1994 - 1997) with the objectives
  of
• Demonstrating intelligent automation technologies
  for VTOL UAVs
• Demonstrating an IPPD approach applicable for
  technology demonstration programs
• Some equipment (DGPS, Sensors, R-1 Avionics
  Package) will be directly leveraged

9
Georgia Tech Leveraged Programs (cont.)

• Intelligent Controls Laboratory (ICL)
• Initiated in 1984 to conduct research and to
  introduce students to the concepts of artificial
  intelligence, i.e. neural networks and fuzzy
  logic and their utility in system identification,
  modeling and control applications. Also includes
  applications for industrial controls and
  manufacturing and industry participation, e.g.
  Honeywell, Ford, GE, etc.
• Mission-ORiented Architectural Legacy Evolution
  (MORALE)
• As part of the DARPA Evolutionary Design of
  Complex Software (EDCS) program supports the
  software evolution process using software reverse
  engineering and visualization, architectural
  evaluation, and change requirements elicitation
• Uninhabited Aerial Vehicle (UAV) Research
  Facility
• Established in 1997 for applications in neural
  network and fuzzy logic augmented control of
  UAVs. Includes two Yamaha R-50 Unmanned
  Helicopters which serve as Experimental Flight
  Controls Research Testbeds and will be directly
  leveraged for this project

10
Yamaha R-50
11
Vehicle Details

• Performance
• Max. Take Off Weight 67Kg (147.7 lb.)
• Practical Payload 20Kg ( 44.1 lb.)
• Endurance 40 Minutes
• Control Range Visual Observation Range
• Engine Yamaha water Cooled 2-Stroke
• Engine Displacement 98cc ( 6.0 in3 )
• Max Horse Power 12 Hp
• Fuel Auto Gas and Oil Mix
• Specification
• Overall Length 3580mm (11.8 ft)
• Fuselage Length 2655mm (8.7 ft)
• Fuselage Width 700mm (2.3 ft)
• Overall Height 1080mm (3.6 ft)
• Main Rotor Dia 3070mm (10.1 ft)
• Tail Rotor Dia 520mm (1.7 ft)
• Empty Weight 44Kg (97 lb.)
• Fuel Tank Capacity 4000 cc. (0.9 Gal)

• Special Feature
• YACS (Yamaha Attitude Control System)
• YACS is a flight attitude control system in which
  three fiber optic gyroscopes and three
  accelerometers are fitted to the helicopter body
  to supply data to a computer unit that regulates
  all control stick operations (Roll, Pitch, Yaw
  and Throttle).
• As a result, the Helicopter maintains a stable
  fight pattern without the operator having to make
  constant remote control adjustment

12
Aerial Robotics Mission Figure
The Millennial Event
International Aerial Robotics Competition
AD 1998, 1999, 2000
13
Mission Scenario
14
Mission Scenario
15
Mission Abstraction
16
Software Enabled Control
Technical Approach Overview
http//uav.ae.gatech.edu/sec
17
Hierarchical Intelligent Control Architecture
18
UAV Testbed Functional Architecture
19
UAV Mission Planning and Control
20
Hardware Configuration
21
Tasks

• Task I Mid-Level Coordination for Mode Switching
  and Reconfigurable Control
• Task II Control Integration and Simulation
  Demonstration
• Task III VTOL UAV Demonstration

22
Schedule (2-year project)
Months
Mid-level Coordination
Control Integration and Simulated Demonstration
Multi-level controllers and sensor
processing modules integrated.
Run-time infrastructure and software architecture
developed.
Intelligent VTOL UAV Demonstration
Flight test.
Infrastructure developed.
Test support developed.
23
Software Enabled Control
Task I
Mid-Level Coordination for - Mode Switching
(MS) - Reconfigurable Control (RC)
http//uav.ae.gatech.edu/sec
24
Mid-Level Coordination for Mode Switching and
Reconfigurable Control Subtasks

• Situation awareness for external events and
  internal fault conditions
• Diagnostics and prognostics for fault tolerant
  control
• VTOL UAV mode switching and reconfigurable
  control
• Identify generic, reusable control structures and
  collaborate with toolkit providers

25
MS RC Objective

• Design Control Algorithms to assist an autonomous
  vehicle for smooth and stable transitioning
  (switching) from one flight mode to another

26
MS RC Challenges

• Unknown transitioning dynamics
• Tight stability envelopes
• Curse of dimensionality

27
MS RC Enabling Technologies

• Gain Scheduling (governed by scheduling variable)
• Phase-space Control System Design (Phase-space
  Navigator, Perfect Moment, etc.)
• Blending Mode Controllers (Fuzzy Logic approach)

28
MS RC Requirements

• Input from High-Level Mode Selection Module
• Output to Low-Level Flight Controllers

29
MS RC Approach

• A Hybrid Analytical/Intelligent Methodology
• Design Local Controllers
• design linear or nonlinear regulators for the
  start and goal modes
• Model Combined Dynamics
• model the local and coupling dynamics of the
  start and goal modes
• Determine Blending Gains
• determine gains using the Phase Portrait
  Assignment Algorithm
• Stability and Robustness Analysis

30
MS RC Example

• An Example Hover to Forward Flight
  ControllerMode to Mode ControllerStructure/Preli
  minary Results

31
Fault Tolerant Control (FTC) Objectives

• Develop Fault-Tolerant Control algorithms for an
  autonomous vehicle so that critical external and
  internal events may be monitored in real time
• Assess as expeditiously and as accurately as
  possible the potential impact of such events on
  the vehicles mission and/or the health of its
  components
• Generate appropriate plans that will mitigate
  such threats
• Execute the plans without severely compromising
  the integrity of the vehicle itself or the
  mission objectives

32
FTC Challenges

• Small vehicle inertia (easily driven to an
  unstable regime)
• Severe monitoring / processing requirements
• Large disturbances and large-grain uncertainty
• Small reaction times
• No UAV is currently capable of performing a
  mission in an autonomous or even semi-autonomous
  manner under uncertain environmental conditions

33
FTC Enabling Technologies

• LQ-Based Redesign Procedure
• Control Mixer Algorithm
• Heuristic Method

34
FTC Requirements

• Input from situation awareness module
• Output to control reconfiguration and flight
  control modules

35
FTC Approach
36
FTC Approach (cont.)

• View the UAV as a large-scale interconnected
  system, S, decomposed into isolated subsystems,
  Si.
• Characterize qualitative properties of S via a
  scalar Lyapunov Function, V, and define an
  appropriate performance measure subject to
  constraints.
• The objective is to find a decentralized
  hierarchical feedback controller which minimizes
  V, possesses features of structural flexibility,
  may be implemented through parallel
  architectures, and utilizes the interconnections.

37
FTC Proposed Solution

• Partition the controller and subsystem structure
  and decompose similarly the cost functional.
  Given the decentralized LSS with each of the
  autonomous pairs (Si, S0) completely controllable
  and the overall system stabilizable, define a
  two-level hierarchicalcontrol strategy with S0
  the coordinator level and Si the subsystem
  level.

38
FTC Proposed Solution (cont.)

• The gains of the suboptimal decentralized
  controller may be computed from the solution of
  an optimization problem. The objective
  functional, subject to stability constraints, is
  optimized to determine the feedback gains.

39
Candidate Failure Modes

• Component
• tail failure, battery failure, loss of data link
• Sensor
• altitude sensor, RPM, GPS, engine temperature
• Actuator
• main rotor control, tail rotor control
• Operational
• engine stall and surge, rotor stall

40
FTC Proposed Tail Rotor Failure(for Simulation)
With fault tolerant and reconfigurable control
system
Control reconfiguration using main rotor controls
Translatory descent to a clear area
Gain altitude using main rotor collective
Control reconfiguration for autorotation
Tail rotor failure
Without fault tolerant and reconfigurable
control system

Autorotational landing
41
FTC Proposed Main Rotor Actuator Failure(for
Demo)
With fault tolerant and reconfigurable control
system
Failure detection and control reconfiguration
with RPM control
Main rotor control failure
Without fault tolerant and reconfigurable control
system
42
Software Enabled ControlTask II Control
Integration and Simulated Demo
http//uav.ae.gatech.edu/sec
43
Task II Control Integration and Simulated Demo

• Software architecture and run-time infrastructure
• Integration of mid-level and low-level control
  algorithms
• Integration of inputs from other contractors
• Simulation of VTOL UAV
• Demonstration of software-enabled control
  routines via simulation

44
Task II Details

• Simulation model development of the R-50
  helicopter to include necessary functions for
  failure simulation and fault tolerant and
  reconfigurable control evaluations
• Carry out a detailed failure mode analysis of the
  R-50 helicopter
• Develop adaptive neural networks based flight
  control structure for mode selection and mode
  switching, optimize network structure and network
  parameters

45
Model Inversion with Adaptive Neural- Net / Fuzzy
Logic Flight Controller
46
Task II Details (cont.)

• Integration of mid-level and low-level control
  algorithms and software development
• Development, analysis and simulation evaluations
  of control reconfiguration strategies
• Simulation demonstration of selected failure
  scenarios
• Flight test evaluation of selected failure
  scenarios

47
Control Integration Objectives

• Interoperability among distributed, heterogeneous
  components
• Real-time, asynchronous communication
• Mission-oriented customization at all levels of
  the hierarchical control architecture
• Reuse of generic control and integration
  mechanisms in new applications

48
Control Integration Approach

• Component based design and modeling of all
  modules
• Use Boeings Bold Stroke Open System
  Architecture as facilitator for the run-time
  architecture
• Use Design Patterns to factor out common features
• Transfer of control algorithms and integration
  patterns to toolkit providers

49
Hierarchical Software Architecture
Mid Level Control Mode Switching Reconfiguration
Fault Tolerant Control Optimization
Trajectory Commands
Low Level Control Stability Augmentation Set
Points Trajectory following
Actuator Commands
High Level Commands

Vehicle Dynamics External Stimuli
High Level Control Situation Awareness Reactive
Control
Sensor Suite Vision GPS IMU Altimeter
State and Navigation Info.
50
Architecture for Algorithms (1)
High Level Reconfiguration Commands
Trajectory Commands
Mid Level Control
High Level Control
Mode Reconfiguration Implementor
Mid Level Control
Mode Transitioning Module
Low Level Control
Sensor Suite
Mode Strategy

• Flight Management
• Collision Avoidance
• Trajectory Optimization
• Formation Management

Mode Strategy
Framework
Reconfigurable Component interface
Algorithm Hooks
51
Architecture for Algorithms (2)
Trajectory Reconfiguration Commands
Actuator Commands
Low Level Control
High Level Control
Control Reconfiguration Implementor
Control Transitioning Module
Mid Level Control
Low Level Control
Control Strategy
Sensor Suite

• Pure PID
• Model Inversion Control
• Neural Net Augmented Control

52
Architecture for Algorithms (3)
Reconfiguration Commands
State and Navigation Info.
Sensor Suite
Sensor Reconfiguration Implementor
High Level Control
Transitioning Module
Mid Level Control
Low Level Control
Filter Strategy

• Filter type
• Filter constants etc.

Sensor Suite
GPS
Vision
IMU
Altimeter
Vehicle Dynamics and External Stimuli
53
Architecture for Algorithm Implementation

• Use UML for design of these components
• Use Design Patterns to abstract common factors
  and provide hooks for algorithms using a common
  interface
• Software integration occurs through the common
  interface

54
Example Observer (a.k.a. Publish-Subscribe)

• Intent Define a one-to-many dependency between
  objects so that when 1 object changes state, all
  its dependents are notified and updated
  automatically.

Structure of Solution
Behavior
Source Design Patterns, Gamma et. al
55
Example Strategy

• Intent Define a family of algorithms,
  encapsulate each one, and make them
  interchangeable. Strategy lets the algorithm
  vary independently from clients that use it.
• Structure of Solution

Source Design Patterns, Gamma et. al
56
Concrete Example of a Strategy Pattern
Navigator DetermineRoute(WayPoints) ...
PathPlanning Strategy ShortestPath(Graph)
Hill Climbing Search ShortestPath(Graph)
A Search ShortestPath(Graph)
BeamSearch ShortestPath(Graph)
57
Communications Architecture

• Facilities we need to implement the above
• Distributed computing in real-time
• Platform independence (NT, VxWorks)
• Decoupling of the interacting components making
  applications plug-and-play
• Data flow and Control flow facilities
• Timer and Time stamping facilities

58
Boeings Bold Stroke Open Systems Architecture

• Real-time CORBA-based integration of distributed,
  heterogeneous components.
• Builds on the Real Time (RT) Object Request
  Broker (ORB) developed as part of the TAO project
  at Washington University in collaboration with
  Boeing.

59
Boeings Bold Stroke OSA Key Features

• Extends CORBA to provide real-time, asynchronous
  communication without CORBAs high overhead
• Key features
• real-time event dispatching and scheduling based
  on priorities and resource requirements
• synchronization
• efficient event filtering and correlation
  mechanisms
• real-time, predictable (QoS) performance
• optimized memory management
• wall clock across modules and timer services

60
CORBA Common Object Request Broker Architecture

• ORB facilitates distributed communication across
  diverse platforms.

61
Example Interface Description in Interface
Definition Language (IDL)
module Sensors interface InertialSensor
attribute long Weight attribute long
Volume attribute float Resolution
interface PositionSensor IntertialSensor
float MeasurePosition() interface
VelocitySensor IntertialSensor float
MeasureVelocity()
Sensor.idl
Interface (in IDL)
Interface (in IDL)
Client Stub (in C)
Server Skeleton (in C)
TAO (RT-ORB)
62
Component Communication Example
Simulation
Propagation Strategy
Rigid Body Dynamics (50 Hz)
Swash Plate Servo dynamics (50 Hz)
UAV Interface
PID
Aerodynamics
Control rotor dynamics (100 Hz)
Rotor dynamics (100 Hz)
Controller Strategy
Neural Net
Controller Interface
Open Systems Architecture
Testbed
Actuators

• Distributed objects
• Plug-and-play
• Encapsulation
• Reconfiguration

UAV Interface
Sensor Suite
63
Event Channels

• Suppliers push data into the channel
• Consumers read data at the rate they need it
• De-coupled data sources and targets
• Event filtering allows consumers to subscribe to
  only events of interest to them.
• Event correlation makes consumers wait for
  multiple events to occur before executing.

64
Component Point of View
Data Flow
Data Flow
Object
20 Hz
MR_coll
Helicopter Simulation/ Airframe
20 Hz
Y?,?,?,pn,pe,pd
B1
A1
Health
TR_coll
5 Hz
Start()
Shutdown()
Initialize_systems()
Test_Subsystems()
Control Flow
65
Transfer to Toolkit

• Types of reusable structures that may be
  transferred
• generic algorithmic structures for mid-level
  control (fault-tolerant control and mode
  switching)
• design patterns for fault-tolerance,
  reconfiguration, and component integration
• other commonly used components such as GUIs and
  math utilities

66
Hardware-in-The- Loop Simulation
Simulated Sensor data
Pilot and/or Ground Station Operator
PC based Helicopter Simulation
Flight Hardware
Actuator responses

• Real time PC based dynamic simulation of
  helicopter
• Network Communications between PC based
  simulation and flight control computer
• Dynamic response of simulation is used as sensor
  data
• Flight control laws are run in real time based on
  actual pilot input and simulated response
• This capability allows testing of control laws
  and presence of various digital implementation
  effects including time delays and facilitate
  flight hardware and software qualification

67
Proposed Tail Rotor Failure(for Simulation)
With fault tolerant and reconfigurable control
system
Control reconfiguration using main rotor controls
Translatory descent to a clear area
Gain altitude using main rotor collective
Control reconfiguration for autorotation
Tail rotor failure
Without fault tolerant and reconfigurable
control system

Autorotational landing
68
Software Enabled ControlTask III VTOL UAV
Demonstration
http//uav.ae.gatech.edu/sec
69
Task III VTOL UAV Demonstration

• Ground-based and onboard software-enabled control
  implementation
• Demonstration of software-enabled control
  technologies on a small VTOL UAV

70
Proposed Main Rotor Failure(for Demo)
With fault tolerant and reconfigurable control
system
Failure detection and control reconfiguration
with RPM control
Main rotor control failure
Without fault tolerant and reconfigurable control
system
71
Hardware Configuration
72
System Integration and Flight Test Support

• System Integration addresses all integration
  aspects, both software and hardware, and is the
  key to a successful program
• Safe demonstration of advanced real-time
  computing frameworks and intelligent SEC
  algorithms requires an extensive system
  integration effort and build-up flight test
  program prior to the VTOL UAV Demonstration
• Flight testing requires support from the GST
  Ground Station (including operator), a certified
  Yamaha R-50 pilot, and maintenance support
  personnel

73
Software Enabled ControlIssues, Concerns
Conclusion
http//uav.ae.gatech.edu/sec
74
Schedule of Work
75
Issues and Concerns

• Although Georgia Tech will leverage considerable
  resources from previous projects (CERT, ASRT, UAV
  Lab, etc.) and on-going research (Flight Controls
  Research), along with cost sharing, there is a
  shortage of sponsored funds available in FY 1999
  and FY 2000 to successfully accomplish the
  objectives at a reasonable risk
• It is estimated that approximately 200K will be
  required in FY 1999 and 50K in FY 2000 for the
  following items
• Task II Control Integration and Simulated
  Demonstration
• System integration and flight testing (Un-funded)
• Task III Flight Test Development and Support
• Flight test development and support (Un-funded)
• Integration feasibility of other SEC contractors
  software algorithms is unknown without knowledge
  of the software and its delivery schedule

76
Contractor Interaction

• Georgia Tech/Boeing
• Vehicle specifications and instrumentation
  available on the Web
• Computer interface specifications will be
  available on the Web
• Our needs methods for high level mode selection
  and situation awareness
• We provide design patterns for toolkit
  developers, the software architecture, and the
  testbed

77
Team Collaboration
The Repository
Testbed
Code conforming to interface guidelines committed
over network
Digital Data Link
Run-time Code
Component Algorithm Configurations
Internet
Boeing R-50 So
Other Algorithm Developers
Georgia Tech Algorithm Developer
Mission Control Station
78
Potential Phase II Follow On

• Further integration of other contractor toolkits
• Additional VTOL UAV demonstrations of SEC
  methodologies
• Extension of SEC technology to other intelligent
  UAV applications such as NASA/Air Force X-36,
  DARPA/Boeing Canard Rotor Wing (Dragonfly),
  Navy/Bell Tiltrotor UAV (Eagle Eye)

79
Technology Transfer

• Transferring information among contractors, DARPA
  and AFRL
• Web Page and Central Repository for disseminating
  information
• Possible 2nd meeting in Atlanta so people can see
  facilities
• Interconnections between the academics, the
  facilitators (industry) and the end users (AFRL)

80
Summary and Conclusions

• Georgia Tech is very excited about working with
  DARPA, AFRL, Boeing and the other SEC contractors
  in advancing the state-of-the-art in
  software-enabled control using evolving
  object-oriented computing architectures
• We believe this is a natural extension for the
  work we have done in the area of uninhabited
  vertical takeoff and landing (VTOL) aerial
  vehicles over the past five years
• We have presented our technical approach and the
  computing architecture we plan on using and look
  forward to working with the other contractors in
  integrating their SEC algorithms
• Our only issue/concern is having sufficient
  resources to properly execute this challenging
  system integration effort and provide the flight
  test support for the SEC implementation and
  demonstration

81
Software Enabled ControlExtra Slides
http//uav.ae.gatech.edu/sec
82
System Integration
Client Protocol
User Interface
Integration Infrastructure
Custom Software
Server Protocol (CORBA, DCOM, Sockets)
Additional Standards and Protocols
83
The Autonomous Scout Rotorcraft Testbed (ASRT)
Project

• ASRT was a 2.5 year (1994-1997), 2.5M Army
  sponsored project to demonstrate advances in
  autonomous vehicle technologies for VTOL aircraft
  and to evaluate the use of an Integrated
  Product/Process Development (IPPD) for a
  technology demonstration
• Due to constraints on availability of a suitable
  VTOL platform most of the software developed
  (except for the flight controller) was included
  in the ground station and telecommunicated to
  the aerial platform, a small hobby size
  helicopter, the GST 300
• The ASRT Project was successfully completed and a
  hour video documents the project, both the
  autonomous vehicle technologies developed along
  with the IPPD approach
• The object-oriented software developed, along
  with the operator-interface is considered
  state-of-the-art for actual real time
  demonstration on an autonomous VTOL UAV