Unified Robotic Software Development using CLARAty

1
Unified Robotic Software Development using
CLARAty
Theme The Long Road to Technology Infusion
Issa A.D. Nesnas Mobility and Robotic Systems
Section Autonomous Systems Division July 20,
2005 http//claraty.jpl.nasa.gov Briefing to the
Office of the Chief Technologist Sponsors Mars
Technology ProgramIntelligent Systems Program
2
Presentation Overview

• Historical Antecedents
• Problem and Objectives
• What is CLARAty?
• Interoperability Success Stories
• Navigation
• Single-cycle Instrument Placement
• Challenges of Software Interoperability
• Challenges in Technology Infusion
• Architecture Overview
• Challenges Ahead

3
Some JPL Robots
4
Historical Antecedents

• Late 80s - Early 90s parallel robotic
  developments
• RSI, MOTES, Satellite Servicing, Robby,
  Mircorover
• No shared hardware or software
• Mid 90s Mars rover research centralized with
  Rocky 7
• First flight rover
• Late 90s Expansion and diversification of rover
  work
• No software interoperability (Rocky 7, FIDO,
  Athena, DARPA)
• Autonomy demonstration of Remote Agent Experiment
  (ARC and JPL)
• MDS investigates reusable software for spacecraft
  control.
• 99-Early 00 Exploration Technology Program
  develops concept for a unifying autonomy
  architecture
• Unifying autonomy and robotic control
• Started the CLARAty task
• Today
• Unification of several robotic developments at
  JPL, ARC, and CMU
• Two flight rovers with several new robotic
  capabilities

5
Problem and Approach

• Problem
• Difficult to share software/algorithms across
  systems
• Different hardware/software infrastructure
• No standard protocols and APIs
• No flexible code base of robotic capabilities
• Objectives
• Unify robotic infrastructure and framework
• Capture and integrate legacy algorithms
• Simplify integration of new technology
• Operate heterogeneous robots

6
What is CLARAty?

• CLARAty is a unified and reusable software that
  provides robotic functionality and simplifies the
  integration of new technologies on robotic
  platforms

A tool for technology development and maturation
7
Interoperability Software Hardware
CAPABILITY Navigation
SRI Stereo
ARC Stereo
Sojouner PoseFIDO 3DEKF 6D EKF
Stereovision JPL_STEREO
Stereovision JPL_STEREO
Stereovision JPL_STEREO
Pose Estimation MER_SAPP
Obstacle Avoidance MORPHIN
Drivemaps
Pose Estimation MER_SAPP
Obstacle Avoidance MORPHIN
Pose Estimation MER_SAPP
Pose Estimation MER_SAPP
Obstacle Avoidance MORPHIN
GESTALT
CLARAty Reusable Software
Robot Adaptation
QNX
VxWorks
Linux
8
Interoperability Success Stories

• Navigation
• Single-cycle Instrument Placement

9
Capturing Flight Algorithms MER GESTALT on FIDO
10
Navigation Interoperability on FIDO and Rocky 8

• Complex Algorithms on different Platforms
• I/O, motion control
• Trajectory Generation
• Rough Terrain Locomotion
• Odometry Pose Estimation
• Stereo Processing
• Visual Odometry
• Navigation (Morphin)
• Obstacle avoidance
• Path Planning

11
Interoperability Success Stories

• Navigation
• Single-cycle Instrument Placement

12
Single-cycle Instrument Placement
Changes in FOV
1st Frame
37th Frame after 10 m
13
FALCON Visual Target Tracker on Rocky 8

• Integration of Complex Algorithms
• I/O, motion control
• Trajectory Generation
• Rough Terrain Locomotion
• Odometry Pose Estimation
• Stereo Processing
• Visual Odometry
• Obstacle avoidance
• Mast Control
• Visual Tracking

14
Integrated Single-Cycle Instrument Placement
15
CLARAty Framework for Single-cycle Instrument
Placement
Possible Alternatives
SIFT Tracker
SIFT Tracker
K9 Mast Pointing
Maestro Ground System
Visual Odometry
2D/3D Visual Tracking
2D/3D Visual Tracking
R8 Mast Pointing
Visual Odometry
On -board Rover Software Infrastructure
Visual Tracker
Haz Camera Tracking
Gaze Pointing
Pose Estimator
HIPS
Vision-guided Manip
Wheel Odometry
Base Placement
MSL Base Placement
Obstacle Avoider
Camera Hand-off
Locomotor
Morphin Navigator
KIM Hand-off
R8 Locomotor
FIDO EKF
Mesh Registration
6DOF EKF
GESTALT
Visual Odometry
Bundle Adjustment
Drivemaps
16
Challenges in Interoperability

• Mechanisms and Sensors
• Hardware Architecture
• Software Algorithms

17
Different Mobility Mechanisms
18
Different Sensors and Appendages
4 DOF Mast
19
Challenges in Interoperability

• Mechanisms and Sensors
• Hardware Architecture
• Software Algorithms

20
Centralized Hardware Architecture
Video Switcher
RS232 Serial
IMU
PC104 x86 Arch Framegrabbers Digital I/O Analog
I/O Wireless Ethernet
FIDO
PID Control in Software
Potentiometers
Actuator/Encoders

21
Distributed Hardware Architecture
Sun Sensor
1394 Bus
RS232

• Compact PCI
• - x86 Arch
• Wireless E/net
• 1394 FireWire
• - I2C Bus

IMU
Rocky 8
I2C
Rocky Widgets Single-axis controllers Current
sensing Digital I/O Analog I/O

Distributed Motion Control and Vision
Potentio- meters
Actuator/Encoders

22
Challenges in Interoperability

• Mechanisms and Sensors
• Hardware Architecture
• Software Algorithms

23
Software Challenges for Algorithm Infusion

• The new algorithms to be integrated may
• Have architectural mismatches with the framework
• Include multiple orthogonal functionalities
• Make implicit assumptions about the platform
• Duplicate functionality in the framework
• Use incompatible data structures
• Are complex and hard to tune
• Require highly specialized domain expertise
• Are poorly implemented

24
Architecture and Process
25
A Two-Layered Architecture
CLARAty Coupled Layer Architecture for
Robotic Autonomy
THE DECISION LAYER Declarative model-based
Global planning
INTERFACE Access to various levelsCommanding
and updates
THE FUNCTIONAL LAYER Object-oriented
abstractionsAutonomous behaviorBasic system
functionality
Adaptation to a system
26
Multi-level Abstraction Model
Use abstractions
Interface at different levels
27
Collaborations Interactions
JPL Internal Programs
Other NASA Programs
RTD, MDS, DRDF
Legacy AlgorithmsFlight Algorithms
Technology Tasks
Technology Tasks
Technology Tasks
Competed Mars Technology Program
CLARAty
Flight FocusedTechnology Programs
NASA Centers andUniversities Technology Tasks
NASA Centers andUniversities Technology Tasks
TechnologyValidation Tasks
NASA Centers andUniversities Technology Tasks
Jet Propulsion Lab
NASA Centers andUniversities Technology Tasks
TechnologyValidation Tasks
NASA ARC
CMU
U. Minnesota
Rover Hardware
Operator Interface
Rover Simulation ROAMS
Science InstrumentsSimulation
28
CLARAty Test Bed for Regression Testing
FIDO2 Stack
ATRV Jr.
Dexter ManipulatorBench top
Rocky 8 PPC Bench top
29
Challenges Ahead

• Mature framework and robotic capabilities
• Investigate relevance to flight and define
  migration path
• Develop regression tests for long-term
  maintainability of robotic capabilities - very
  hard and open research topic
• Maintain current capabilities on existing
  platforms
• Develop new capabilities (e.g. continuous motion)
• Integrate new technologies from competed programs
• Develop a releasable version
• Develop formal documentation and tutorials
• Identify and deploy on low-cost rover platform
• Open source

30
Summary

• Developed a unified and reusable software
  framework
• Deployed at multiple institutions
• Deployed on multiple heterogeneous robots
• Integrated multiple technologies from different
  institutions
• Delivered algorithms for formal validation
• Enabled new technology developments on multiple
  platforms
• Integrated flight algorithms for detailed
  performance characterization and operation on
  research rovers.
• Taking a technology from inception, to
  development in CLARAty, to validation, and now to
  integration into flight

31
Current CLARAty Core Team

• Jet Propulsion Laboratory
• Antonio Diaz Calderon
• Tara Estlin
• John Guineau
• Won Soo Kim
• Richard Madison
• Michael McHenry
• Mihail Pivtoraiko
• Issa A.D. Nesnas
• Babak Sapir
• I-hsiang Shu
• OphirTech
• Hari Das Nayar

• NASA Ames Research Center
• Clay Kunz
• Eric Park
• Susan Lee
• Carnegie Mellon University
• David Apelfaum
• Nick Melchior
• Reid Simmons
• University of Minnesota
• Stergios Roumeliotis

Full Credits for all Developers and Contributors
athttp//keuka.jpl.nasa.gov/main/project/team/in
dex.shtml
32
Questions?
33
Distributed Software Development
AFS Backbone
Authentication
...
CMU
JPL
UW
ARC
U. Minnesota
K9
CLARAty
VxWorks
Rocky 8
3rd Party
Web
FIDO
Rocky 7
Number of employees and not FTEs
34
Unify Mechanism Model
35
Some CLARAty Statistics

• 320 modules in repository (increase of 6 from
  FY03) goal is to limit modules
• 60 modules are researched technology algorithms
  (20)
• About 500,000 lines of C code revise and
  reduce
• Five adaptations Rocky 8, FIDO, Rocky 7, ATRV,
  K9
• Most technology modules are at Level III
• None are at Level IV or Level V (formally
  reviewed, documented, and open source)

• CLARAty Integration Levels
• Level I Deposited
• Level II Encapsulated
• Level III Refactored
• Level IV Formally reviewed
• Level V Open source and fully
  documented

36
Relevance to the Missions

• Why is this work relevant to the missions?
• Provides a common environment for development,
  test, and comparison of advanced robotic
  technologies
• Provides an infusion path for robotics
  technologies into flight missions
• Demonstrates technologies on relevant robotic
  systems
• Makes research rovers viable test platforms for
  flight algorithms (e.g. navigation)
• Is robust to changes in rover hardware designs
• Can be easily adapted to flight and new research
  rovers

37
Measuring Success or Failure

• We succeed IF we
• Significantly reduce integration time of new
  technology software onto real robotic systems
• Support multiple platforms with different
  hardware architectures
• Provide a service that is enabling for
  technologists
• Simplify the development/integrate/debug/test
  cycle for current and next generation NASA rovers
• Have people other than the developers using and
  like the system