Towards%20an%20Effective%20Software%20Component%20Certification%20Process

1
Towards an Effective Software Component
Certification Process

• Advisor
• Silvio Lemos Meira
• srlm_at_cin.ufpe.br

Student Alexandre Alvaro aa2_at_cin.ufpe.br
2
Agenda

• Introduction
• Reuse
• RiSE Reuse in Software Engineering
• CBD Component-Based Development
• Component
• Quality
• Component Certification
• Component Certification History
• Future

3
Introduction Reuse

• Frakes, 1995
• Software reuse is the use of existing software
  knowledge or artifacts to build new software
  artifacts.
• Everything that was done, should not be done
  again

4
Introduction Reuse

• Advantages
• Increase
• Quality
• Productivity
• Decrease
• Time to market
• Manutenability
• Reduce life-cycle of development
• Software Reuse
• Not simple
• Systematic Software Reuse

5
Introduction RiSE

• Almeida et al., 2004
• The IEEE International Conference on Information
  Reuse and Integration.

http//www.cin.ufpe.br/rise
6
Introduction CBD

• Software reuse ideas is not new
• McIlroy, 1969
• Mass Produced Software Components
• To (re)use, instead of to develop
• To keep a set of reused components
• Repository systems
• Mili, 1998
• 50 proposed solutions to this problem

7
Introduction Component

• The exactly concept of component in CBD is not
  yet a consensus...

A software component is a unit of composition
with contractually specified interfaces and
explicit context dependencies only. A software
component can be independently deployed and is
subject to third-party composition.
Szyperski, 2002
8
Introduction Component

• Bass et al., 2000
• CMU/SEIs report
• Inhibitors
• Lack of available components
• Lack of stable standards for component
  technology
• Lack of certified components
• Lack of an engineering method to consistently
  produce quality systems from components.

9
Introduction Component

• Besides CMU/SEI...
• Heineman, 2000
• Councill, 2001
• Crnkovic, 2001
• Wallnau, 2003

10
Introduction Quality
ISO 9000
CMM

• Weber Nascimento, 2002

11
Introduction Component Certification

• Concept...

Third-party certification is a method to ensure
that software components conform to well-defined
standards based on this certification, trusted
assemblies of components can be constructed.
Councill, 2001
12
Introduction Component Certification
13
Introduction Component Certification

• Frakes et al., 1996

14
History of Component Certification

• History of the Software Component Certification
• Decade of 90
• Mathematical models
• Test-Based models

15
History of Component Certification

• Poore et al., 1993
• Planning and certifying software system
  reliability
• Three mathematics model
• Test cases
• Report the failures
• Data are analyzed to achieve a reliability index
• Reliability of system
• Considering how the components affects this
  reliability

16
History of Component Certification

• Wohlin Runeson, 1994
• Certification of Software Components
• Method that consist
• Usage model
• Usage profile
• Test cases based on this models
• Collection the failure data
• Certification of reliability
• Hypothesis certification
• Certify a specific reliability level -gt given
  degree of confidence
• Reutilization degree of the models

17
History of Component Certification

• Rohde et al., 1996
• Certification of Reusable Software Components
• Rome Laboratory of the Air Force, NY

18
History of Component Certification

• Rohde et al., 1996
• Certification of Reusable Software Components
• Certification process
• Readiness Assessment
• Compile without error and execute the code
• Static Analysis
• Automatic tool
• Code Inspection
• Manual technique
• Testing
• Other tests
• Analysis of the certification process

19
History of Component Certification

• TCI Initiative, 19981
• Affiliation of researchers
• Formal interface specification
• Supports compositional reasoning
• A restricted set of behavioral properties of
  assemblies
• Difficult to find real contributions

1 http//www.trusted-components.org
20
History of Component Certification

• Voas, 1998
• Certifying Off-the-Shelf Software Components
• Automated technologies
• Black-Box testing and fault injection
• Methodology
• Black-box component testing
• System-level fault injection
• Operational system testing
• Certify components to a determined environment...

21
History of Component Certification

• Wohlin Regnell, 1998
• Reliability Certification of Software
  Components
• Extend Wohlin Runeson, 1994 work
• Certification process
• Usage specification (usage model and usage
  profile)
• Certification procedure
• Three approaches
• Certification Process
• Reliability Certification of Component and
  Systems
• Certify or Derive System Reliability

22
History of Component Certification

• Wohlin Regnell, 1998
• Reliability Certification of Software
  Components
• Extend Wohlin Runeson, 1994 work

23
History of Component Certification

• Wohlin Regnell, 1998
• Reliability Certification of Software
  Components
• Extend Wohlin Runeson, 1994 work

?
24
History of Component Certification

• Voas Payne, 2000
• Dependability Certification of Software
  Components
• Metrics framework
• Create a tests methodology
• Component testability score
• Mathematical models
• Statistics approaches
• Estimates the number of test cases necessary
• Consider
• The number of tests that a component received
• The fault revealing ability of those test cases.

25
History of Component Certification

• Morris et al., 2001
• Software Component Certification
• Four steps
• Tests Specification
• Specification Document
• Specified Results
• Test-Pattern Verificator
• Limitations

26
History of Component Certification

• However...
• Testing is not enough...
• Sametinger, 1997
• Component certification levels
• Level 1 A component is described with keywords
  and a summary and is stored for automatic search.
  No tests are performed the degree of
  completeness is unknown
• Level 2 A source code component must be compiled
  and metrics are determined
• Level 3 Testing, test data, and test results are
  added
• Level 4 A reuse manual is added.

27
History of Component Certification

• Heineman et al., 2000
• Panel presented in ICSE2000
• Discuss the necessity of trust assurance in
  component
• Considerable CBD researchers participate
• Heineman organizations
• Councill software development
• Flynt benefits to the customers
• Shaw reutilization of the components

28
History of Component Certification

• Workshops
• 4th ICSE Workshop on Component-Based Software
  Engineering (CBSE) Component Certification and
  System Prediction, 2001.
• 5th ICSE Workshop on Component- Based Software
  Engineering (CBSE) Benchmarks for Predictable
  Assembly, 2002.

29
History of Component Certification

• Long time considering just test...
• Stafford Wallnau, 2001
• Is Third Party Certification Necessary?
• Define a process model
• Support prediction of system properties prior to
  component selection
• Introduce credentials concept
• ltproperty,value,credibilitygt
• Active component dossier
• A dossier is an abstract component that defines
  certain credentials

30
History of Component Certification

• Long time considering just test...
• Stafford Wallnau, 2001
• Is Third Party Certification Necessary?

31
History of Component Certification

• Stafford Wallnau, 2001
• Is Third Party Certification Necessary?
• Some open questions
• What level of trust is required?
• Are there other mechanisms that might be used to
  support trust?
• How to certify measurement techniques?

32
History of Component Certification

• Other authors
• How certification should be carried out? (Goulao
  Abreu, 2002)
• What does it mean to trust a component? (Hissam
  et al., 2003)
• What characteristics of a component make it
  certifiable, and what kinds of component
  properties can be certified? (Wallnau, 2003)

33
History of Component Certification

• Councill, 2001
• Third-Party Certification and Its Required
  Elements
• Other aspect of component certification
• Human
• Industrial
• Business
• Certification is the components future

34
History of Component Certification

• Woodman et al., 2001
• Issues of CBD Product Quality and Process
  Quality
• Analyze some process in various CBD approaches
• Examine 11 potential CBD quality attributes

Reusability Maintainability Accuracy Clarity
Replaceability Interoperability Scalability Performance
Flexibility Adaptability Reliability
Reusability Maintainability Accuracy Clarity
Replaceability Interoperability Scalability Performance
Flexibility Adaptability Reliability
35
History of Component Certification

• Hissam Wallnau, 2003
• Enabling Predictable Assembly
• Extends the Stafford Wallnau, 2001work
• Introduced Prediction-Enabled Component
  Technology (PECT)
• Component technology with analysis technology
• Prediction of assembly properties
• Identify required component properties
• Certifiable properties

36
History of Component Certification

• Hissam Wallnau, 2003
• Enabling Predictable Assembly
• Extends the Stafford Wallnau, 2001work
• Component technology and analysis technology
• Component model
• Component runtime environment
• Assembly environment

• Defines a property theory
• Parameters of this theory
• Component properties

More abstract, less acurate
Increased accuracy prediction
Increased accuracy prediction both
37
History of Component Certification

• Hissam Wallnau, 2003
• Enabling Predictable Assembly
• Extends the Stafford Wallnau, 2001work
• Validation
• Empirical
• Predictions made, conform to observations
• Limitations
• Two prediction technology may be incompatible
• How are non-resource attributes, such as
  security, to be empirically validated?
• Industrial component certification ?

38
History of Component Certification

• Meyer, 2003
• The Grand Challenge of Trusted Components
• Two complementary roads
• Low Road
• Qualification of existing components
• High Road
• Production of components with fully proved
  correctness properties.

39
History of Component Certification

• Meyer, 2003
• The Grand Challenge of Trusted Components

40
History of Component Certification

• McGregor, 2003
• Measuring Component Reliability
• Support prediction of assemblies reliabilities
  based on properties of the components
• Method to measuring and communicating the
  reliability of the component
• Components services
• Components documentation
• Test plan is created, based on components
  services
• Provide the reliability of each service
• This method is a fundamental element of PECT

41
History of Component Certification

• Wallnau, 2003
• Volume III A Technology for Predictable
  Assembly from Certifiable Components
• CMU/SEIs report
• How component technology can be extended in order
  to achieve Predictable Assembly from Certifiable
  Components (PACC).
• Runtime behavior of software components
  assemblies
• Components property
• Components proprieties need rigorously defined
  and trusted and
• It can be certified by independent third-party
  developers

42
History of Component Certification

• Wallnau, 2003
• Volume III A Technology for Predictable
  Assembly from Certifiable Components
• CMU/SEIs report
• SEIs approach to PACC is PECT.

43
History of Component Certification

• Wallnau, 2003
• Volume III A Technology for Predictable
  Assembly from Certifiable Components
• CMU/SEIs report
• Status
• On going work
• PECT is relatively immature
• One or more certification properties
• Tools are being developed
• Functional certification complements the PECT
  Meyer, 2003
• Precondition to PECT
• Non-functional properties

44
History of Component Certification

• Two failures case....
• National Information Assurance Partnership (NIAP)
  
• Together with NIST and NSA
• From 1993 until 1996
• Defines criteria for certifying security features
  of components
• Restricted set of behavioral assembly properties.
• IEEE
• 1997
• The initiative was suspended, in this same year

45
Summary
46
Future RiSE Context

• Almeida et al., 2004
• The IEEE International Conference on Information
  Reuse and Integration.

http//www.cin.ufpe.br/rise
47
Future work

• Key CBD Requirements
• What are the requirements for a certification
  process?
• Woodman et al., 2001
• 11 CBD requirements
• Simao,2003
• 124 CBD requirements
• Larson, 2004
• 72 CBD requirements
• Component Quality Model
• What requirements are more important ?
• Meyer, 2003

48
Future work

• Certification Method
• How certify components ?
• A Metrics Framework
• How to measure the component certification
  processes ?

Sofware Component Certification Process
49
Future work

• Write a Paper
• On the Software Component Certification Process
• The history
• The proposal

50
Future work
51
References

• Frakes, 1995 Frakes, W., B., Fox, C., J.
  Sixteen Questions about Software Reuse.
  Communications of the ACM, June, 1995.
• Szyperski, 2002 Szyperski, C., 2002. Component
  Software Beyond Object-Oriented Programming.
  Addison-Wesley, USA. ISBN 0-201-74572-0.
• Mcllroy, 1968 Mcllroy, M. D., 1968. Mass
  Produced Software Components. NATO Software
  Engineering Conference Report, October, pp.
  79-85.
• Mili et al., 1998 Mili, A., Mili, R.,
  Mittermeir, R., 1998. A Survey of Software Reuse
  Libraries. Annals Software Engineering, Vol. 05,
  pp. 349414.
• Heineman Councill, 2001 Heineman, G. T.,
  Councill, W. T., 2001. Component-Based Software
  Engineering Putting the Pieces Together.
  Addison-Wesley, USA. ISBN 0-201-70485-4.
• Heineman et al., 2000 Heineman, G. T.,
  Councill, W. T., Flynt, J. S., Mehta, A., Speed,
  J. R., Shaw, M., 2000. Component-Based Software
  Engineering and the Issue of Trust. The IEEE
  Proceedings of the 22nd International Conference
  on Software Engineering (ICSE), Canada, pp.
  661-664.
• Crnkovic, 2001 Crnkovic, I., 2001.
  Component-based software engineering - new
  challenges in software development. Software
  Focus, Vol. 2, No. 4, pp. 27-133.

52
References

• Wallnau, 2003 Wallnau, K. C., 2003. Volume III
  A Technology for Predictable Assembly from
  Certifiable Components. Software Engineering
  Institute (SEI), Technical Report, Vol. III,
  April.
• Frakes Terry, 1996 Frakes, W., Terry, C.,
  1996. Software Reuse Metrics and Models. ACM
  Computing Survey, Vol. 28, No. 2, June, pp.
  415-435.
• Poore et al., 1993 Poore, J., Mills, H.,
  Mutchler, D., 1993. Planning and certifying
  software system reliability. IEEE Computer, Vol.
  10, No. 1, January, pp. 88-99.
• Wohlin Runeson, 1994 Wohlin, C., Runeson, P.,
  1994. Certification of Software Components. IEEE
  Transactions on Software Engineering, Vol. 20,
  No. 6, June, pp. 494-499.
• Rohde et al., 1996 Rohde, S. L., Dyson, K. A.,
  Geriner, P. T., Cerino, D. A., 1996.
  Certification of Reusable Software Components
  Summary of Work in Progress. The IEEE Proceedings
  of the 2nd International Conference on
  Engineering of Complex Computer Systems (ICECCS),
  Canada, pp. 120-123.

53
References

• Voas, 1998 Voas, J. M., 1998. Certifying
  Off-the-Shelf Software Components. IEEE Computer,
  Vol. 31, No. 6, June, pp. 53-59.
• Wohlin Regnell, 1998 Wohlin, C., Regnell, B.,
  1998. Reliability Certification of Software
  Components. The IEEE Proceedings of the 5th
  International Conference on Software Reuse
  (ICSR), Canada, pp 56-65.
• Voas Payne, 2000 Voas, J. M., Payne, J.,
  2000. Dependability Certification of Software
  Components. Journal of Systems and Software, Vol.
  52, No.2-3, June, pp. 165-172.
• Morris et al., 2001 Morris, J., Lee, G.,
  Parker, K., Bundell, G. A., Lam, C. P., 2001.
  Software Component Certification. IEEE Computer,
  Vol. 34, No. 9, September, pp. 30-36.
• Sametinger, 1997 Sametinger, J., 1997. Software
  Engineering with Reusable Components. Springer
  Verlag, USA. ISBN 3-540-62695-6.

54
References

• Stafford Wallnau, 2001 Stafford, J., Wallnau,
  K. C., 2001. Is Third Party Certification
  Necessary?. The IEEE Proceedings of the 4th ICSE
  Workshop on Component-Based Software Engineering
  (CBSE), Canada, May, pp. 1317.
• Councill, 2001 Councill, B., 2001. Third-Party
  Certification and Its Required Elements. The IEEE
  Proceedings of the 4th ICSE Workshop on
  Component-Based Software Engineering (CBSE),
  Canada, May.
• Woodman et al., 2001 Woodman, M., Benebiktsson,
  O., Lefever, B., Stallinger, F., 2001. Issues of
  CBD Product Quality and Process Quality. The IEEE
  Proceedings of the 4th ICSE Workshop on
  Component-Based Software Engineering (CBSE),
  Canada, May.
• Hissam et al., 2003 Hissam, S. A., Moreno, G.
  A., Stafford, J., Wallnau, K. C., 2003. Enabling
  Predictable Assembly. Journal of Systems and
  Software, Vol. 65, No. 3, March, pp. 185-198.
• Meyer, 2003 Meyer, B., 2003. The Grand
  Challenge of Trusted Components. The IEEE
  Proceedings of 25th International Conference on
  Software Engineering (ICSE), USA, pp. 660667.

55
References

• McGregor et al., 2003 McGregor, J. D.,
  Stafford, J. A., Cho, I. H., 2003. Measuring
  Component Reliability. The IEEE Proceedings of
  the 6th ICSE Workshop on Component-Based Software
  Engineering (CBSE), USA, May, pp. 13-24.
• Schmidt, 2003 Schmidt, H., 2003. Trustworthy
  components compositionality and prediction.
  Journal of Systems and Software, Vol. 65, No. 3,
  March, pp. 215-225.
• Simão, 2003 R. Simao, A. Belchior, Quality
  Characteristics for Software Components
  Hierarchy and Quality Guides. Lecture Notes in
  Computer Science, pp. 188-211, June.
  Springer-Verlag. 2003.
• Larson, 2004 M. Larson, Predicting Quality
  Attributes in Component-based Software Systems,
  PhD Thesis, Malardalen University, 2004.