Designing User Interfaces Spring 1999

1
SE 746-NT Embedded Software Systems
Development Robert Oshana Lecture
34 For more information, please
contact NTU Tape Orders NTU Media
Services [phone]
oshana_at_airmail.net
tapeorders_at_ntu.edu
2
Embedded Systems Testing Part 1
From Beatty ESC 2002
3
Agenda

• Introduction
• Types of software errors
• Finding errors methods and tools
• Embedded systems and RT issues
• Risk management and process

4
Introduction

• Testing is expensive
• Testing progress can be hard to predict
• Embedded systems have different needs
• Desire for best practices

5
Method

• Know what you are looking for
• Learn how to effectively locate problems
• Plan to succeed manage risk
• Customize and optimize the process

6
Entomology

• What are we looking for ?
• How are bugs introduced?
• What are their consequences?

7
Entomology Bug Frequency

• Rare
• Less common
• More common
• Common

8
Entomology Bug severity

• Non functional doesnt affect object code
• Low correct problem when convenient
• High correct as soon as possible
• Critical change MUST be made
• Safety related or legal issue
• Domain Specific !

9
Entomology - Sources

• Non-implementation error sources
• Specifications
• Design
• Hardware
• Compiler errors
• Frequency common 45 to 65
• Severity Non-functional to critical

10
Entomology - Sources

• Poor specifications and designs are often
• Missing
• Ambiguous
• Wrong
• Needlessly complex
• Contradictory

Testing can fix these problems !
11
Entomology - Sources

• Implementation error sources
• Algorithmic/processing bugs
• Data bugs
• Real-time bugs
• System bugs
• Other bugs

Bugs may fit in more than one category !
12
Entomology Algorithm Bugs

• Parameter passing
• Common only in complex invocations
• Severity varies
• Return codes
• Common only in complex functions or libraries
• Reentrance problem
• Less common
• Critical

13
Entomology Algorithm Bugs

• Incorrect control flow
• Common
• Severity varies
• Logic/math/processing error
• Common
• High
• Off by 1
• Common
• Varies, but typically high

14
Example of logic error
If (( this AND that ) OR ( that AND other ) AND
NOT ( this AND other ) AND NOT ( other OR NOT
another ))
Boolean operations and mathematical calculations
can be easily misunderstood In complicated
algorithms!
15
Example of off by 1
for ( x 0, x lt 10 x)
This will execute 11 times, not 10!
for ( x array_min x lt array_max x)
If the intention is to set x to array_max on the
last pass through the loop, then this is in error!
Be careful when switching between 1
based language (Pascal, Fortran) to zero based (C)
16
Entomology Algorithm bugs

• Math underflow/overflow
• Common with integer or fixed point math
• High severity
• Be careful when switching between floating point
  and fixed point processors

17
Entomology Data bugs

• Improper variable initialization
• Less common
• Varies typically low
• Variable scope error
• Less common
• Low to high

18
Example - Uninitialized data
int some_function ( int some_param ) int
j if (some_param gt 0) for ( j0 jlt3
j) / iterate through some process
/ else if (some_param lt
-10) some_param j / j is
uninitialized / return some_param
return 0
19
Entomology Data bugs

• Data synchronization error
• Less common
• Varies typically high

20
Example synchronized data
struct state / an interrupt will trigger
/ GEAR_TYPE gear / sending snapshot in a
message / U16 speed U16 speed_limit U8
last_error_code snapshot snapshot.speed
new_speed / somewhere in code
/ snapshot.gear new gear / somewhere
else / snapshot.speed_limit speed_limit_tb
gear
Interrupt splitting these two would be bad
21
Entomology Data bugs

• Improper data usage
• Common
• Varies
• Incorrect flag usage
• Common when hard-coded constants used
• varies

22
Example mixed math error
unsigned int a 5 int b -10 / somewhere
in code / if ( a b gt 0 )
ab is not evaluated as 5 ! the signed int b is
converted to an unsigned int
23
Entomology Data bugs

• Data/range overflow/underflow
• Common in asm and 16 bit micro
• Low to critical
• Signed/unsigned data error
• Common in asm and fixed point math
• High to critical
• Incorrect conversion/type cast/scaling
• Common in complex programs
• Low to critical

24
Entomology Data bugs

• Pointer error
• Common
• High to critical
• Indexing problem
• Common
• High to critical

25
Entomology Real-time bugs

• Task synchronization
• Waiting, sequencing, scheduling, race conditions,
  priority inversion
• Less common
• Varies
• Interrupt handling
• Unexpected interrupts
• Improper return from interrupt
• Rare
• critical

26
Entomology Real-time bugs

• Interrupt suppression
• Critical sections
• Corruption of shared data
• Interrupt latency
• Less common
• critical

27
Entomology System bugs

• Stack overflow/underflow
• Pushing, pulling and nesting
• More common in asm and complex designs
• Critical
• Resource sharing problem
• Less common
• High to critical
• Mars pathfinder

28
Entomology System bugs

• Resource mapping
• Variable maps, register banks, development maps
• Less common
• Critical
• Instrumentation problem
• Less common
• low

29
Entomology System bugs

• Version control error
• Common in complex or mismanaged projects
• High to critical

30
Entomology other bugs

• Syntax/typing
• if (ptrNULL) Cutpaste errors
• More common
• Varies
• Interface
• Common
• High to critical
• Missing functionality
• Common
• high

31
Entomology other bugs

• Peripheral register initialization
• Less common
• Critical
• Watchdog servicing
• Less common
• Critical
• Memory allocation/de-allocation
• Common when using malloc(), free()
• Low to critical

32
Entomology Review

• What are you looking for ?
• How are bugs being introduced ?
• What are their consequences ?
• Form your own target list!

33
Finding the hidden errors

• All methods use these basic techniques
• Review checking
• Tests demonstrating
• Analysis proving

These are all referred to as testing !
34
SE 746-NT Embedded Software Systems
Development Robert Oshana 10 minute
break For more information, please
contact NTU Tape Orders NTU Media
Services [phone]
oshana_at_airmail.net
tapeorders_at_ntu.edu