ACCEPTANCE TESTS - PowerPoint PPT Presentation

1 / 10
About This Presentation
Title:

ACCEPTANCE TESTS

Description:

SDD/DFS A. Modigliani. VLT 2nd Generation Instrumentation ... Are all recipes documented, including In/Out tags? ( esorex man-page) SDD/DFS A. Modigliani ... – PowerPoint PPT presentation

Number of Views:139
Avg rating:3.0/5.0
Slides: 11
Provided by: mpe1
Category:

less

Transcript and Presenter's Notes

Title: ACCEPTANCE TESTS


1
ACCEPTANCE TESTS
Andrea Modigliani
2
OUTLINE
  • Why/When
  • Priorities
  • Acceptance tests
  • Experience feedback

3
Why/When
  • Why
  • Need to define a clear and compact list of
    requirements and tests to verify the compliance
    of DRL deliveries to PAE/COM1.
  • Standardise and make transparent the review
    process.
  • Possibly distribute the review across a long
    period.
  • Ensure that the DRL not only meets scientific
    specs but also is easy to maintain, portable,
    well documented, efficient, robust.
  • When
  • Every three months to have at least 4-6
    iterations in the period FDR-PAE.
  • We encourage the consortium to apply the tests
    during development.
  • Tests are performed by ESO after a given DRL
    version has been certified/provided on an
    appropriate data set.

4
Priorities
  • The next slides list several requirements
    organized by priority
  • Priority 1 tests will be checked on every
    delivery,
  • Priority 2 tests are performed from time to time,
  • Priority 3 tests are verified at the end (PAE
    end of commissioning)together with priority 1
    and priority 2
  • Each test is listed and a possible verification
    criteria is indicated in parenthesis,
  • references to other presentations are indicated
    as Author

5
Compliance and initial verifications
  • The acceptance test should be part of the PAE
    process (executed on DRL 0.5).
  • Any action should to be completed before the last
    commissioning run (in DRL 1.0).
  • See also R. Palsa, Y. Jung.
  • Does the pipeline follow the recipe template Y.
    Jung? (static checks).
  • Is the documentation in English? (manual
    inspection)
  • Does the software conform to ESO coding standards
    R. Palsa? (manual inspection, compilation with
    no warning with strict compiler options).
  • Does the pipeline use only CPL and agreed upon
    libraries C. Izzo, S. Castro ? (static checks).
  • Does the code contain only CPL functions/objects?
    If something is missing, ask cpl-help_at_eso.org
    (compute ratio CPL/SLOC).
  • Are exported function namespace protected?
    (static checks).

6
Execution tests to verify the delivery is
complete and installable/executable (1)
  • Do the recipes exist in the expected version?
    (esorex recipes, esorex man-page)
  • Is there a test package covering each combination
    of recipes and major instrument setting? (manual
    inspection, compare with DRL doc)
  • Are test data appropriate to verify each recipe
    and DRL relevant function? (manual inspection,
    compare with DRL doc)
  • Are recipes executable? (verify on provided test
    package)
  • Do the recipes create the expected products S.
    Castro? (compare output versus DRL doc)
  • Do the recipes create the QC they should P.
    Ballester, S. Castro? (compare output versus DRL
    doc)

7
Execution tests (2)
  • Do the expected DRL functions exist? (nm
    libiiinstrument.so)
  • Is there a unit test for each DRL function
    covering each instrument setting and parameter
    space L. Lundin? (manual inspection, compare
    with DRL doc)
  • Are there memory leaks J. M. Larsen? (esorex
    mem-check recipe_name recipe_name.sof)
  • Are there memory errors J.M. Larsen? (valgrind
    toolmemcheck)
  • Is the recipe interface implemented in a standard
    way Y. Jung? (runtest.pl, test with Gasgano)
  • Is the doxygen doc complete? (make html, check
    results)
  • Do all source files contain addtogroup/defgroup?
    (static checks, make html)
  • Are all recipes documented, including In/Out
    tags? (esorex man-page)

8
Detailed validation are results correct?
  • Do recipes give correct results (FITS products
    QC parameters)? (run recipe on test data and
    check In/Out, compare with DRL doc)
  • Do the interfaces fit together (is it possible to
    run a recipe cascade?)
  • Are the final results (after running the cascade)
    correct? (manual inspection, compare with DRL
    doc)
  • Does each recipe terminate in a reasonable time ?
    (exposure time/execution time gt1). To profile
    see J.M. Larsen
  • Does the pipeline work and give results as
    defined in the DRL design doc? (manual
    inspection, compare results with DRL doc)

9
Detailed validation (2)
  • Does the unit test implement the quality
    assessment defined in the DRL doc? (manual
    inspection, compare with DRL doc) L. Lundin
  • Does the unit test check the basic validity of
    the function results (including QC parameters)
    and that the accuracy is as required? (manual
    inspection, compare with DRL doc)
  • Does the unit test cover relevant parts of input
    parameter space, such as true/false for Boolean
    parameters? (manual inspection)
  • Do the unit tests pass? (manual inspection, make
    check)
  • Does the recipe verify it receives proper input
    (check for input TAGs)?
  • Is the FITS format as described in the DRL doc
    (extensions/keywords)?
  • Are the product valid FITS files? (fitsverify)
  • Is the documentation and code understandable?
    (manual inspection)

10
Experience and feedback
  • Communication is a crucial factor
  • Acceptance tests should be part of the
    implementation schedule presented at FDR.
  • Acceptance tests may involve additional work in
    the initial implementation phase but will save a
    lot of time for PAE and commissioning and help to
    provide a better product to the users.
  • Acceptance tests are in addition to other tests
    which may be applied by the instrument SV team.
Write a Comment
User Comments (0)
About PowerShow.com