Consistent Terminology, Consistent Results

1
Consistent Terminology, Consistent Results

• Introduction and Definitions

2
Agenda Introductions and Definitions

• Design integrity A working definition
• Why is this important?
• A tale of two designs
• Has anything changed?
• Why should I care?
• The miracle cloud
• The alternative
• Overview
• Implications
• Additional Definitions
• Summary

3
Definition and Goal

• Design The invention and disposition of the
  forms, parts, or details of something according
  to a plan. (AH dictionary online)
• Integrity The state of being unimpaired
  Soundness The quality or condition of being
  whole or undivided completeness (AH)
• This seminar is intended to talk about techniques
  and issues that ensure the soundness and
  completeness of both the end product and the
  means used to produce it.

4
Design 1 Radarsat 1 ACP
5
Radarsat 1 ACP Overview

• Program dates late 1990 late 1992
• Specifications
• Processor 8 MHz 80386/80387
• Memory128 k x 8 SRAM, 128kx8 EEPROM, 16k x 8
  PROM
• Interfaces A/D (16), D/A (4-12), Synchronous
  serial (3 input, 3 output), RS-232 GSE
• Function Attitude control processor for the
  RADARSAT1 satellite
• Logic Implementation MSI logic PALs (16L8,
  16R6, 16R8)
• Additional functions (cross-strap, control)
  external

6
PAL Reminder
7
Design 2 - Command Telemetry Board (CTB)
8
CTB Overview

• Program Dates early 2001 late 2003
• Specifications
• Processor RTX2010 (16 MHz)
• Memory 4M x 8 (random), various FIFOs (16k x8)
  as necessary
• Interfaces
• MIL-STD-1553B
• Synchronous Serial (command / telemetry)
• Analog input
• High-level discrete (output)
• Low-level discrete (input / output)
• Functionality S/C command/telemetry (level 0 and
  active)
• Logic Implementation 4 54SX32S FPGAs
• Additional resources (in same box) RAD 750, Mass
  Memory, Instrument Interface Card

9
Whats Changed?

• Capability / Complexity
• Logic Density
• Specificity
• RADARSAT (1 small specification with interface
  focus)
• CTB (1 large specification with interface, s/w,
  operations focus)
• Software Centricity
• Initial Errors
• RADARSAT 3 jumpers 1 PAL design change
• CTB 14 FPGA revisions
• 2 spec change
• 5-6 mistakes
• 6-7 data dependency

10
Whats Not Changed?

• Overall program schedules
• Proportional budget
• Expectation of correctness
• Pain from mistakes

11
What Explains the Difference?

• Engineers arent as capable? Insulting!
• Everything is just more complex? Copout!
• Methodology?
• Methodology hasnt changed
• Always inadequate, we just got lucky
• Adequate for old designs, no longer adequate
• Methodology has changed
• Used to be adequate, but we lost the recipe
• Design philosophy of systems has changed?
• Predicated on maximum flexibility
• Expectation of extreme complexity
• Over-specification almost impossible to meet

12
What Do These Examples Illustrate?

• The incidence of initial correctness for designs
  seems to be decreasing
• Design changes seem to be more common
• Problems late in the verification/validation
  cycle seem to be more frequent
• Perhaps a combination of the factors presented
  explains this, but
• Desired complexity is not going to decrease
• Budgets are not going to get bigger
• The expectation of excellence isnt going to go
  away
• The only solution is to develop and improve a
  consistent methodology for implementing robustly
  designed products
• Based on basic principles
• Applicable to a variety of conditions

13
Why Should I Care?

• Why do I work?
• Self-actualization (fun, monetary reward,
  interest)
• Why do people want us to work for them?
• They need what we produce
• What do people want engineers (especially in
  Aerospace) to produce?
• A quality product that satisfies the customers
  needs
• How do they want us to produce such a product?
• Consistently and efficiently

14
The Laymans View The Miracle Cloud
15
The Miracle Cloud Method

• Note that too many engineering schools teach this
  approach without meaning to
• Advantages to the miracle cloud method
• Total creative freedom
• Disadvantages to the miracle cloud method
• Product quality is variable
• Team makeup dependent
• Team mood/morale dependent (Monday morning car)
• Luck dependent
• Product is not produced in a repeatable manner
• Product is not produced in an efficient manner
• Result
• Quality low
• Customer Satisfaction Low

16
How Do We Replace the Miracle Cloud?

• Provide structure to the development effort
• Evaluate the effort and the product produced
• Improve the effort and the product
• Repeat

17
Definitions of Importance

• From Q9000-2000
• Process A set of interrelated and interacting
  activities which transforms inputs to outputs in
  our case ideas to devices
• Product The result of a process

18
Implications From These Definitions

• If we want a consistent product, we must have a
  consistent process
• If we want to improve a product, we must improve
  the process
• If our company has no (or inadequate) process and
  we must produce a quality product, then we must
  establish a process personal responsibility
• Developing, imposing, and improving a process is
  not an end (in and of itself) it is only a means
  to an end

19
A Model for Discussing the Design Process
20
Notes on the Model

• Feedback / iteration are not shown for clarity
• Model may be recursive
• Board development process includes FPGA
  requirement definition, FPGA development,
  instantiation, etc.
• Board development process includes the FPGA
  validation product
• Successes and failures are cumulative
• Good requirements successful development gt
  successful instantiation
• Bad requirements failed development gt failed
  instantiation
• Complexity multiplies
• Complex requirements increase design complexity
  which, in turn, increases verification complexity
• Processes are absolute gates to the next stage of
  development

21
Implications From the Model

• All processes must be addressed at all levels of
  design there are no shortcuts!
• Does not imply same formality at all levels
• Does imply same rigor at all levels
• Up front work on requirements is essential!
• Must provide adequate time and money
• Must gain team buy-in to the process
• Benefits compound throughout the rest of the
  activities
• Simplicity is an essential virtue
• Complexity inevitably multiplies
• A product is not qualified until both
  verification and validation are complete

22
Additional Useful Definitions (courtesy of
Q9000-2000)

• Specification A document stating requirements,
  needs, or expectations that are obligatory
• Quality The degree to which a set of inherent
  characteristics fulfill requirements
• Customer satisfaction Customers perception of
  the degree to which the customers requirements
  have been fulfilled
• Verification Confirmation, through the
  provision of objective evidence, that specified
  requirements have been fulfilled
• Validation Confirmation, through the provision
  of objective evidence, that the requirements for
  a specific intended use or application have been
  fulfilled
• Objective evidence Data supporting the
  existence or verity of something
• Continual Improvement recurring activity to
  increase the ability to fulfill requirements
• Note the importance of Requirements

23
Summary

• I have no assurance that my product will have
  consistent quality without
• Well-defined requirements
• A well planned approach to implementing the
  requirements
• A clearly defined plan for verification and
  validation of the requirements
• The ability to improve the process that produces
  the product
• Without quality product, customer satisfaction is
  impossible
• Without customer satisfaction, I wont work!
• Therefore, I must care about ensuring design
  integrity