Component Interface Testing

1
Component Interface Testing

• SciDAC Quarterly Report
• Argonne, IL
• William McLendon
• Erik DeBenedictis
• Ron Oldfield
• Sandia National Laboratories

Sandia is a multiprogram laboratory operated by
Sandia Corporation, a Lockheed Martin
Company,for the United States Department of
Energy under contract DE-AC04-94AL85000.
2
Overview

• SC2003 Demonstration of APItest v.1
• APItest Development
• Lessons learned
• v.2
• Ron Oldfield

3
SC2003 (APItest v.1)

• Presented APItest at SC2003 in the ASCI booth.
• HTTP interface developed for demo after September
  meeting.
• Demonstrated APItest working with some basic
  tests.
• Demo

4
APItest v.2

• Built on Twisted Framework
• URL http//www.twistedmatrix.com
• Better development environment for
• HTTPD mode of operation
• Distributed component testing
• DB interfacing

5
APItest v.2

• Some new Test file formats
• Collaboration with Scott Jackson
• Separates tests from groupings to allow test
  reuse.
• Test file (.apt) -- contains 1 test
• Batch file (.apb) -- lists groups of test w/
  ordering
• Dependencies specified in batch files similar to
  GraphVIZ file formatting.

6
New File Formats Why?

• Why split Test file from its dependencies?
• Organization
• Can organize tests more readily
• Reusability
• APItest v.1 requires you to cut and paste a LOT
• Readability
• Smaller files
• Each file more targeted

7
APItest v.2 Test File Example
lt?xml version"1.0" encoding"utf-8" ?gt
lttestDefgt ltinfogtSome information about the
this testlt/infogt lttest name"test_name"
minPctMatch"100" maxPctMatch"100"
numReps"1" onMismatch"CONTINUE"gt  
ltinput name"sd"gtsome sss commandlt/inputgt  
ltinput name"comp2"gtanother sss commandlt/inputgt
  ltoutput name"sd" format"regexp"gtexpected
reply from sdlt/outputgt   ltoutput name"comp2"
format"literal"gtexpected reply from
comp2lt/outputgt   lt/testgt lt/testDefgt
8
Standard Test Metadata Attributes
Attribute Type Reqd Default Desc
name string YES name of test
numReps integer NO 1 number of times to execute test
minPctMatch float NO 0.0 min of repetitions that must match for test to pass
maxPctMatch float NO 100.0 max of repetitions that must match for test to pass
preDelay float NO 0.0 delay in seconds prior to executing test
postDelay float NO 0.0 delay in seconds after executing test before continuing to next test.
iterDelay float NO 0.0 delay in seconds between tests during loop (only used if numReps gt 1)
onMismatch string NO CONTINUE If a test fails to match, what do we do? CONTINUE,BREAK,HALT
9
APItest v.2 Batch File Example
lt?xml version"1.0" encoding"utf-8" ?gt
lttestBatchgt ltinfogtDescription of the test
filelt/infogt lttest name"research.foo.foobar"
/gt lttest name"research.foo.bar" /gt ltdep
parent"research.a" child"research.b"
status"PASS" /gt ltdep parent"research.a"
child"research.c" /gt ltdep parent"research.b"
child"research.e" /gt ltdep parent"research.c"
child"research.d" /gt ltdep parent"research.d"
child"research.e" status"ANY" /gt lt/testBatchgt
10
APItest v.2 Batch File Example (Graphically)
research/foo/foobar.apt
research/foo/bar.apt
research/a.apt
research/b.apt
research/c.apt
research/d.apt
research/e.apt
11
APItest v.2 Tests

• APItest v.2 is the release version.
• Learned a LOT from development of v.1
• Thanks for all the great feedback from those
  whove tried it out.
• v.2 is mostly an integration problem with
  Twisted.
• Twisted will greatly improve the ease for
  distributed testing.
• Would like a copy of the latest API documents for
  all components if not posted on SSS Web Notebook.
• Feedback is still greatly appreciated
• We have a telecon number - can set one up
  anytime.
• Hope to get some real test suites going this
  quarter.

12
Twisted Python

• Framework, written in Python for writing
  networked applications.
• Some Features
• web server
• RDDB interface
• Object broker
• Asynchronous event handling
• Distributed Applications

13
New Member at Sandia Ron Oldfield
14
(No Transcript)
15
APItest Component Architecture Concept