Lessons Learned from Testing the STEREO G

1
Lessons Learned from Testing the STEREO GC Truth
Model

• Gary M. Heiligman
• JHU/APL

2
Outline

• STEREO guidance and control
• Context of the truth model
• Verification of the truth model
• Gold standards
• Results of testing
• Sample defect types
• Lessons learned, assessment, and recommendations

3
The STEREO Spacecraft

• Two nearly identical spacecraft
• Perform orbital maneuvers to achieve
  Earth-leading and Earth-trailing heliocentric
  orbits
• Sun-pointed, 3-axis stabilized solar observatory
• Maintain 7 arcsec or better pointing at Sun
• Maintain high-gain antenna pointing at Earth
• 2 year prime mission life
• Control buildup of angular momentum due to
  torques from solar radiation pressure

4
STEREO Guidance and Control
Trq Commands
Reaction Wheels (4)
Attitude Interface Electronics
Sun Angles
Sun Sensor
Wheel Rates
Thr Commands
Thrusters(12)
Inertial Measurement Unit
Status
Gyro ?q Accel ?V
Step Commands
HGA Gimbal
Gimbal ang.
CDH Processor
Commands
1553 Bus
GC Data
Pointing Error Signals
Star Tracker q
Inputs
Star Tracker
G C Processor
Guide Telescope
5
STEREO GC Subsystem Components

• Actuators
• 4 Reaction wheels
• 12 Thrusters
• 3 sets of 4 each
• Canted
• High gain antenna rotary actuator (HGARA) stepper
  (redundant windings)
• Onboard processor
• BAE 25 MHz RAD6000
• Algorithms implemented in Simulink, compiled with
  Real Time Workshop

• Sensors
• Digital sun sensors
• 5 heads
• Inertial measurement units
• 2 redundant units
• 3 ring laser gyros and 3 accelerometers per unit
• Star tracker
• Fine guidance telescope
• Part of SECCHI instrument suite
• Potentiometers for HGARA angle measurement

6
Complete Standalone Model in Simulink
7
Hardware in the Loop (HIL) Simulator
STEREO Test Support System (STSS)
Flight Avionics
8
STSS Truth Model Components
9
Truth Model in Simulink
10
Methods for Verifying the Truth Model

• Inspection
• Conducted by independent GC analyst
• Standalone Testing
• Compile full Simulink model to run on PC or Unix
• Able to run up 100 real time
• Run and analyzed by developers and independent
  analyst
• Truth Model Acceptance Testing
• Compile Truth Model to embed into STSS
• Runs at real time with or without flight avionics
• Run and analyzed by independent tester
• GC Flight Software Acceptance Testing
• Exposes Truth Model defects triggered by flight
  algorithm
• Run and analyzed by independent tester

11
Truth Model Acceptance Test Plan

• Test dynamics kinematics
• Simplest physics model
• --then--
• Test actuator models
• Add to physics model
• --then--
• Test sensor models

• Rotational Dynamics
• High-Gain Antenna (HGA) Motion and Solar Torque
• Orbit Generator
• Thrusters
• Reaction Wheels and Tachometers
• Inertial Measurement Units
• Star Tracker
• Digital Sun Sensors
• Guide Telescope

12
Functional Areas for Acceptance Testing

• Rotational Dynamics
• High-Gain Antenna (HGA) Motion and Solar Torque
• Orbit Generator
• Thrusters
• Reaction Wheels and Tachometers
• Inertial Measurement Units
• Star Tracker
• Digital Sun Sensors
• Guide Telescope

13
Gold Standards for Truth Model Testing(1 of 2)

• Gold Standard a trusted source for acceptable
  outputs to which the outputs from the software
  under test are compared.
• Analytical equations (expressed in MATLAB)
• Express the physics in textbook equation form
• Begin with solid-body rotation
• Add frictionless wheels
• MATLAB software libraries
• ode45 Runge-Kutta solver
• chebvand gallery function for evaluating
  Chebyshev polynomials

14
Gold Standards for Truth Model Testing(2 of 2)

• JPL HORIZONS ephemeris terminal server
• Provides Earth X, Y, Z in J2000.0 heliocentric
  coordinates
• MIL-STD-1553 bus messages recorded from
  engineering model hardware
• Reaction wheel
• IMU (Earth rotation is a good gyro stimulator)
• Star tracker (but star tracker requires a
  stimulator)
• Guide telescope

15
Gold Standard for Basic Rotational Dynamics

• function dy PropagateAttitude(t, y, torque,
  inertia_inv, inertia, whl_torque, whl2b, jb2whl)
• Elements 13 are angular rate in the
  non-rotating body frame
• From Wertz eq. 16-54 on p. 523
• Isc dw/dt N - dh/dt - w x (Isc w h)
• dy(13, 1) inertia_inv (torque - whl2b
  whl_torque - ...
• cross(y(13,1), (inertia y(13,1) whl2b
  y(811,1)))')
• Wertz Eq. 17-3 p.559
• capital_omega ...
• 0 y(3) -y(2) y(1) ...
• -y(3) 0 y(1) y(2) ...
• y(2) -y(1) 0 y(3) ...
• -y(1) -y(2) -y(3) 0
• Elements 47 are the inertial-to-body
  quaternion
• Wertz Eq. 17-2 p. 559
• dy(47,1) 0.5 capital_omega y(47,1)

16
Functional Areas vs. Gold Standards
17
Results of Truth Model TestingNumber of Defects
Reported
18 Algorithmic defects found by truth model
acceptance testing
18
Sample Defect TypeRotational Dynamics (1 of 3)
0.004 N m s
19
Sample Defect TypeRotational Dynamics (2 of 3)

• First noted in standalone testing by independent
  GC analyst
• Angular momentum not conserved in torque-free
  motion
• Dependent on integration step size
• Error was modest in magnitude (1 part in 105)
• Adequate agreement for basic rotational dynamics
  tests
• GC analysts did not pursue it vigorously
• Not acceptable for solar torque and HGA tests
  because the error was comparable to the torques
  on the body
• Multiple root causes
• Missing minor term in inertia tensor inversion
• Sign error in higher-order term of integrator
  block
• Isolated by subsequent inspection and standalone
  testing
• Eventual solution reduced error to 1 part in 1010

20
Sample Defect TypeRotational Dynamics (3 of 3)
0.00000002 N m s
21
Side TopicWhich is easier to inspect?
if (k 0) for (j 0 j lt 3 j) y0,j
ICj elseif (k 1) for (j 0 j lt 3
j) y1,j ICj dydt0,j dt else
for (j 0 j lt 3 j) yk1,j yk,j
(1.5 dydtk,j 0.5
dydtk-1,j) dt endif
22
Sample Defect TypeGuide Telescope Message
Problems

• Guide Telescope messages defined by Interface
  Control Document (ICD)
• GT development team supplied a file containing
  MIL-STD-1553B bus messages generated by bench
  test
• Tester captured bus messages generated by
  analogous TM test
• Comparison revealed many differences
• Order of Sun presence bits
• ICD and TM agreed but GT EDM data capture process
  was flawed
• Sun presence bits contained false positives
• TM algorithm error
• 2s complement checksum errors
• Unclear wording of ICD led to incorrect TM
  implementation

23
MIL-STD-1553 Bus Monitor
24
Sample Defect TypeOrbit Generator
Time (s relative to JD) - 0.0090613136
1.0004675 t
Its a defect but so what?
25
One Measure of TM Testing EfficacyDefects in TM
found by Other Testing
26
Lessons Learned (1 of 2)

• The first test is the hardest (large investment
  before payoff begins)
• Make the gold standard analytic model reusable
• Independence of testing / analysis team is
  valuable
• Developers may dismiss problems as unimportant
• TM testing requires much more analysis than GC
  testing
• The TM tester must be able to isolate the error
  amongst
• TM algorithm
• Gold standard algorithm
• Interface code
• CDH, front end, ground system
• Analysis software and tools
• Sensors

27
Lessons Learned (2 of 2)

• Data capture from actual sensors found defects
  that no other method could find (before IT)
• The line between TM acceptance testing and
  performance testing is blurry
• Err on the safe side when you can
• But some bugs really are unimportant

28
Assessing the Value of TM Testing

• TM acceptance testing is expensive
• Approximately 10 SM was spent on acceptance
  testing
• TM testing found a significant number of valid
  defects
• Other methods also found TM defects
• Inspection of Simulink model
• Analysis of standalone test results
• GC testing
• None of the defects found by TM testing were
  mission-ending, but most needed to be fixed
• Testing informs inspection, inspection informs
  testing

29
Recommendations

• Inspect the model first
• Perform standalone testing and analysis
• Test Truth Model dynamics just enough to support
  subsequent tests
• Test the Truth Model sensors against data capture
  from actual sensors (EM or flight) to find
  format, polarity, etc. errors
• Design GC tests to exercise Truth Model as well
• Examine all test results for Truth Model defects

30
Backups
31
Adams Method for ODE solution

• Is this sign correct?