Testing AspectOriented Programs with State Models

1
Testing Aspect-Oriented Programs with State Models

• Dianxiang Xu
• North Dakota State University

2
Agenda

• Introduction
• Aspect-Oriented State Models
• Automated Test Generation
• Empirical Studies
• Open Issues
• Research Plan
• Software Engineering Curriculum

3
Aspect-Oriented Programming (AOP)
superimposition
Aspect
class
Aspect
class
class
Crosscutting concerns
Primary (base) concerns
4
AOP Basic Constructs

• Join Point
• Well-defined point in the flow of a program
• Pointcut
• A collection of join points
• Advice
• Additional code to be executed at (before, after,
  around) the join points
• Intertype Declarations
• Introduce new members to base classes
• Aspect Precedence

5
Is AOP More Immune to Errors?

• No Compelling Reason Exists!
• A Sample Fault Model
• Incorrect pointcut
• Incorrect advice
• Incorrect intertype declaration
• Incorrect aspect or advice precedence

6
Research Goal Aspect Assurance

• Verifying Conformance to Requirements
• Define rigorous state models
• Generate tests from state models
• Detecting Aspect Faults Incremental Testing
• Test base classes first, and then aspects
• If base classes pass all tests, but the
  aspect-oriented program as a whole fail some
  tests, the failure would have to do with aspects.
  

7
MACT Automated Test Generation
Coverage Criterion
AO State Model
builds
chooses
Test Generation
Tester
edits
Transition Tree
chooses
Target Language
Code Generation
Test Code
Test Harness
Test Results
Test Execution
AOP under Test
8
Research Issues

• How to specify the expected impact of aspects on
  object states for testing purposes?
• AO extension to state models with testability
• How to generate tests from AO state models?
• Various coverage criteria
• AOP as runtime code instrumentation
• To what extent can base class test parameters be
  reused for testing aspects?
• Base class tests may not be valid for aspects
• How to ensure correctness of AO models?
• Model checking with LTSA

9
Related Work

• Testing of AOP
• Data flow based (Zhao et al. 03, Lemos 07)
• Control flow based (Zhou et al. 04, Lemos 07)
• Test input generation (Xie et al. 05, 06)
• Fault model (Alexander et al. 04)
• Regression testing (Xu et al., 07)
• Our work model-based test generation
• State models, UML models, Petri nets

10
Related Work contd

• Aspect-Oriented Modeling Notations
• Existing formalisms
• UML dominant approach (AOM Workshop series)
• Petri Nets (Xu et al., 2005-06)
• Statecharts (Elrad, et al. 2002-05)
• Our work testability explicit weaving
  mechanism
• Different purposes of modeling
• Specification, code generation, reverse
  engineering.
• Our work test generation

11
State Models for Classes

• UML Protocol State Machines
• S a set of states
• E a set of events
• Constructors and methods
• T a set of transitions
• (si, ep, q, sj)

12
Class Model CarSimulator
(OFF00, !getIgnition() getThrottle()0
getBrakepedal()0") (ON00, "getIgnition()
getThrottle()0 getBrakepedal()0") (ON01,
"getIgnition() getThrottle()0
getBrakepedal()!0") (ON10, "getIgnition()
getThrottle()!0 getBrakepedal()0")
13
State-Based Tests

• Abstract test a sequence of transitions
• (?, newp0, q0, S0), (S0, e1p1, q1, S1),,
• (Sn-1, enpn, qn, Sn)
• Test input new, e1,, en
• Expected result S0, q0, S1, q1,, Sn, qn
• Properties of expected states that a correct
  implementation should produce.
• Concrete test
• Determine actual arguments of new, e1,, en
• Must satisfy preconds - undecidable problem!

14
State Models for Aspects

• Join Points
• States, events, or transitions in a class model
• Pointcuts
• Each pointcut picks out a group of join points
• Advice
• State models
• Inter-Model Declarations
• Introduce new states, events, and transitions
• Aspects
• Encapsulated entity of inter-type declarations,
  pointcuts, and advice.

15
Aspect ModelCruiseControlIntegrator
16
Impacts of Aspects on Base Models

• Adding new transitions among existing states
• Removing transitions from the base models
• Modifying guard conditions of transitions
• Introducing new states and new transitions
• Introducing new events and new transitions
• Incorporating states and events of other classes
  for object interactions

17
Weaving Aspect Models Semantics

• Inter-model declarations
• Introduce new transitions, states, and events.
• Pointcuts
• Pointcuts are a naming mechanism for mapping
  states/events/transitions in advice models to the
  counterparts in base models.
• Advice
• Transitions selected by the pointcuts are
  replaced with corresponding transitions in advice
  models.

18
Coverage-Oriented Test Generation

• State Coverage
• Transition Coverage
• Transition Pair Coverage
• State Round-Trip Binders
• Effect Round-Trip
• Transition Round-Trip

19
Transition Trees
20
Test Parameters
21
From Transition Tree to Test Code

• private static void modelTest1_1_2_5_1()
• System.out.println("Test case 1_1_2_5_1")
• CarSimulator carsimulator new CarSimulator()
• assertState(carsimulator, CarSimulator.OFF00)
• assertState(carsimulator.getCruiseController(),
  CruiseController.INACTIVE)
• carsimulator.engineOn()
• assertState(carsimulator, CarSimulator.ON00)
• assertState(carsimulator.getCruiseController(),
  CruiseController.ACTIVE)
• carsimulator.accelerate()
• assertState(carsimulator, CarSimulator.ON10)
• carSimulator.setSpeed(30)
• carsimulator.on() //constraint
  isMinSpeedReached()

22
State Wrapper Aspects

• public aspect CarSimulatorModelState
• declare parents CarSimulator implements
  StateBasedClass
• public static final String CarSimulator.OFF00"OF
  F00"
• public static final String CarSimulator.ON00"ON0
  0"
• public String CarSimulator.getModelState()
• if(!getIgnition() getThrottle()0
  getBrakepedal()0)
• return OFF00
• else if(getIgnition() getThrottle()0
  getBrakepedal()0)
• return ON00
• else return "Wrong state"

23
Incremental Testing Process

• Test Base Classes
• Build the state models
• Generate abstract test cases from models
• Derive concrete test suites
• Test Aspects
• Build aspect models
• Wave them into the base models
• Generate abstract tests from woven models
• Generate test suites by reusing, modifying, and
  extending concrete base class tests
• Instantiate new abstract tests

24
Class Tests vs Aspect Tests

• Positive base class tests become negative aspect
  tests.
• Negative base class test can become positive
  aspect tests.

25
Class Tests vs Aspect Tests

• Rule 1 Full reuse of positive tests
• A positive class test remains as a positive test.
  
• Rule 2 Full reuse of negative tests
• A negative class test remains as a negative test.
• Rule 3 Modified reuse of positive base class
  tests as negative aspect tests.
• Rule 4 Reuse of negative tests for expansion.
• A negative base class test can be expanded to
  either positive or negative tests

26
Mutation Analysis Fault Model

• Missing pointcut or incorrect pointcut missing
  join points
• Pointcut picking out extra join points
• Pointcut with incorrect focus of control flow
• Incorrect advice type
• Incorrect advice that fails to enforce
  precondition
• Incorrect advice that fails to establish
  postcondition or intra-class state invariant
• Incorrect advice that fails to preserve
  inter-class state invariant
• Incorrect effects of inter-type declarations
• Incorrect precedence of interfering aspects

27
Empirical Studies
28
Empirical Studies CruiseControl
29
Empirical Studies Telecom
30
Summary Issues Addressed

• Aspect-oriented state models with testability
• Automated test generation for various structural
  coverage criteria
• Reuse of class tests for aspects
• Correctness of AO models via model-checking

31
Summary Open Issues

• What kind of base class tests are less likely
  helpful for revealing aspect faults?
• How to prioritize the test cases to be reused?
• How to generate tests via model-checking?
• How to generate tests with respect to coverage
  criteria of transition preconditions?

32
Research Plan

• Testing for Software Safety
• NASA grant
• Aspect Assurance
• Higher-Order Software Architectures for Pervasive
  Computing
• Verification and Testing for Software Security

33
Undergraduate SE Curriculum

• General Education Courses
• Science, Math, Engineering, etc.
• Foundation
• CS1, CS2, Algorithms, OS, Database,
• Core
• SE Principles, Requirements, Design, HCI,
  Construction, Testing, Planning Management
• Special Tracks
• Game, Web/E-Commerce, healthcare,
• Capstone Projects

34
SE Curriculum - cont

• External Funding Programs
• NSF S-STEM
• Scholarships in Science, Technology, Engineering,
  and Mathematics
• NSF REU
• Research Experiences for Undergraduates
• NSF CPATH
• Pathways to Revitalized Undergraduate Computing
  Education
• DOE GAANN (graduate)

35
Integrating Research Education

• Incorporating research results into courses
• MACT
• Involving students in research
• Tool development
• Empirical studies

36
Questions?