FIT for Embedded System Test Development

1

• FIT for Embedded System Test Development
• 
  
• 
  PRESENTER Jingwen Chen

2
Main topics

• Agile methodology from the desktop world
• XP Inspired embedded system life-cycle
• Adapting FIT into embedded system development
• What is FIT/FitNesse
• EmbeddedFitNesse testing framework
• MatLab/C FitNesse testing framework
• Issues with FIT in embedded design
• Conclusion

3
What we want to avoid ?

• System defects
• Software errors cost 59 billion / year
• Baxter Healthcare has to recall all its 256,000
  Colleague Volumetric Infusion Pumps because the
  hardware / software problems that already cause 3
  deaths and 6 serious injuries.
• We want to find an appropriate and efficient
  defect free development process for embedded
  devices

4
Agile methodology from the desktop business world

• Minimize risk by developing software in short
  iterations, ech iteration is like a miniature
  software project of its own
• Extreme Programming (XP) http//www.extremeprogram
  ming.org/map/project.html

5
(No Transcript)
6
Agile methodology from the desktop business world

• Explore design space
• Test Driven Development (TDD)
• Improved defect reduction
• Enhanced communication between the customers and
  developers
• Extremes of knowledge within a multi-disciplinary
  team
• Customer product end-users and researchers
• Developer Engineers

7
XP inspired (XPI) product lifecycle for
biomedical embedded system

• Stage 1 XPI Product envisagement
• Customers together with developers produce
  executable system specification
• System acceptance tests
• Stage 2 XPI Proof of concept
• Algorithms developed in MATLAB and validated by
  the acceptance tests written in first stage
• Additional unit test in MATLAB using MUnit

8
XP inspired (XPI) product lifecycle for
biomedical embedded system

• Stage 3 XPI Prototyping of initial product
• Forming the initial product using host machine
  simulation / embedded system
• System moved into the C/C production language
  and validated via tests written in stage 1 and
  stage 2
• Additional embedded unit tests added for new
  environment.
• Stage 4 XPI Full production of system
• The production of the full system on real
  embedded system target board
• Satisfy the acceptance tests and unit tests
  developed

9
Build an appropriate Agile development
environment for embedded system

• A mechanism to allow customers without embedded
  system knowledge to form a part of the
  development team
• FitNesse/FIT is one possible solution

10
What is FIT/FitNesse ?

• A software testing tool designed for customers
  with limited IT knowledge
• A wiki-based system for creating, organizing, and
  running FIT test tables
• Fit test tables are easy to be read and written
  by customer

11
An example of FIT test table
TestFixtures. CheckLifeSign() TestFixtures. CheckLifeSign() TestFixtures. CheckLifeSign()
items inputValue testResults?
bloodPressure 95 normal
heartRate 80 normal
temperature 38 high expected
normal actual
12
Starting from a simple temperature measurement
embedded system

• Test software function
• CalculateTemperature ( ) running on the target
  board convert pulse width to actual temperature

Target Platform
Application Layer

Voltage pulse
CalculateTemperature( )
Time High
System Software Layer
VDK real-time operating system
Time Low
Input device TMP03 thermal sensor
Hardware Layer
Analog Device ADSP-BF533 processor
13
Stage 1. XPI Product envisagement

• Customers together with the developers write the
  Fit test table

DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest
VoltHigh VoltLow outputTemperature(F)
10 100 195
20 100 155
30 100 115
14
Stage 2. XP Inspired Proof of concepts

• MatLabFitNesse testing framework
• Developing the function in MatLab validated by
  the fit test in stage 1

FitNesse Web server Wiki editor and storage
DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest
VoltHigh VoltLow outputTemperature(F)
10 100 195
20 100 155
30 100 115 expected
30 100 135 actual
Runner
Fit Server
Fixture for Test table
MATLAB API
MATLAB ENGINE
15
Stage 3. XPI initial production system

• Migrate the MatLab code into embedded system
  language validated via tests for stage 1 and
  stage 2 using host machine simulation

HOST MACHINE
Runner
Fit Server
FitNesse Web server Wiki editor and storage
DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest
VoltHigh VoltLow outputTemperature(F)
10 100 195
20 100 155
30 100 115 expected
30 100 135 actual
Fixture for Test table
MATLAB API
C Simulation Environment
MATLAB ENGINE
EmbeddedFitNesse
16
Stage 4. XPI full production system

• Running the function on real embedded target
  platform and satisfying the original test in
  stage 1

HOST MACHINE
FitNesse Web server Wiki editor and storage
DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest
VoltHigh VoltLow outputTemperature(F)
10 100 195
20 100 155
30 100 115 expected
30 100 135 actual
Runner
Fit Server
Fixture for Test table
MATLAB API
DSP Interface class
C Simulation Environment
VDSP API VDSP Environment
MATLAB ENGINE
EmbeddedFitNesse
S.M.I.L.E TARGET PLATFORM
17
Some tests we have done for embedded hardware
Lab2TestFixtures.SetCoreTimerAssembly Lab2TestFixtures.SetCoreTimerAssembly Lab2TestFixtures.SetCoreTimerAssembly Lab2TestFixtures.SetCoreTimerAssembly Lab2TestFixtures.SetCoreTimerAssembly Lab2TestFixtures.SetCoreTimerAssembly
setCOUNT setPERIOD setSCALE readTCOUNT() readTPERIOD() readTSCALE()
2000 2000 1 2000 2000 1
2000 3000 3 2000 expected 3000 3
2000 3000 3 3000 actual 3000 3
1000 1000 2 1000 1000 2

• An error on the ADSP-BF533 processor was found by
  undergraduate students taking our embedded
  systems course.
• They were using TDD to develop the register
  setting function SetCoreTimer ( ) running on the
  embedded target board.

18
Issues with the full EmbeddedFitNesse testing
framework
Long communication path !

HOST MACHINE
FitNesse Web server Wiki editor and storage
DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest DSPFixture.VDSPColumnFixtureTest
VoltHigh VoltLow outputTemperature(F)
10 100 195
20 100 155
30 100 115 expected
30 100 135 actual
Runner
Fit Server
Fixture for Test table
MATLAB API
DSP Interface class
C Simulation Environment
VDSP API VDSP Environment
MATLAB ENGINE
EmbeddedFitNesse
S.M.I.L.E TARGET PLATFORM
19
FitNesse in embedded design
Is FitNesse a good solution for embedded system
Agile development ?

• How to use FIT tables to specify an embedded
  system properly ?
• Embedded system acceptance tests includes
• system operations
• external and internal stimulus responses
• interrupt latency
• system reliability
• Extend the fit test table to handle diverse test
  cases.
• timing diagrams
• signal comparison figures
• Efficiency and reliability concerns
• Make the EmbeddedFitNesse testing frame work more
  efficiently

20
Conclusion

• XPI Lifecycle embedded system development
• EmbeddedFitNesse testing framework
• MatLab/C FitNesse testing framework
• Issues with FIT in embedded design

21

• Thank you !