BFTPR

1
Check and Simulate A Case for Incorporating
Model Checkingin Network Simulation
Ahmed Sobeih, Mahesh Viswanathan and Jennifer C.
Hou
Submitted to ACM-IEEE International Conference
onFormal Methods and Models for Codesign
(ACM-IEEE MEMOCODE'2004)
2
Outline

• Problem Definition
• Motivation
• Background
• Suggested Solution
• Overview
• Implementation Details
• A proof-of-concept case study
• Experimental Results
• Comparison with Existing Work
• Conclusions
• Future Work

3
Problem Definition

• Existing Network Simulators (e.g., ns-2, J-Sim)
• Performance Evaluation
• End-to-end Delay
• Packet Delivery Ratio
• Throughput, Bandwidth Consumption, etc.
• Existing Network Simulators do NOT usually
  exhaust all possible scenarios.
• Protocol Validation (e.g., error-finding)
• Could a routing protocol suffer from routing
  loops?
• Is a security protocol vulnerable to attacks?
• Will the receiver in a reliable unicast protocol
  always deliver all the packets sent by the sender?

4
Motivation

• If an error goes undetected, serious damages may
  happen after implementation
• Packets may loop indefinitely!
• An attack may break the system!
• The earlier an error is found the better
• Building another prototype for validation
  purposes is time- and effort-consuming and
  error-prone.

Can we have a single integrated tool providing
both Performance Evaluation and Protocol
Validation?
5
Background
Formal Reasoning of S/W and H/W Systems
Model Checking (State-Space Exploration)
Theorem Proving

• Explore the state space until either the entire
  state space is explored or an error state is
  found
• Does not require deep understanding of
  mathematical concepts
• Provides an error trace. Why the error
  happened? How can the error be fixed?

• Use a formal technique (e.g., Hoare logic,
  induction) to prove that the implementation of
  the system meets its specification
• Requires experience in mathematical concepts
• May need to prove auxiliary lemmas first

6
Background Model Checking
s4
s1
s5
s2
s0
s6
s3
s7

• Initial State
• Transitions (a.k.a events)
• When each transition is enabled?
• How each transition is handled?
• Validation Properties (a.k.a. assertions)
• J-Sim explores only one execution path!

7
Suggested Solution

• Incorporate Model Checking in J-Sim
• Question How can we make J-Sim explore the
  entire state space, rather than just exploring
  one single execution path?
• Design Goals
• The core implementation of the network simulator
  must not be modified.
• Modifications (if any) to the simulation code
  that implements a network protocol should be
  minimal.
• The role of the network protocol designer should
  be acceptable.

8
Suggested Solution Overview

• On-the-fly Model Checking
• Execute the simulation code directly to find
  errors in the network protocol

J-Sim
Model Checker
True / Error Trace
Pn
P1
P2
9
Suggested Solution Implementation

• Handling State Space Explosion
• Conventional (Application-independent)
  Techniques
• Stateful Searches (e.g., hashing)
• Application/Protocol-specific Heuristics
• Prune the state space
• E.g., packet loss, packet reorder
• Handling Time Overhead
• Best-First Search
• Guide the search towards buggy paths (i.e.,
  paths that may lead to an error state faster)
• Protocol-dependent metrics

10
Suggested Solution Implementation (cont.)
Stateful On-the-fly Model Checking Procedure

• Procedure modelCheck()
• 1. AlreadyVisitedStates
• 2. NonVisitedStates initialState
• 3. while ( NonVisitedStates gt 0 )
• currentState NonVisitedStates.remove()
• 5. if ( currentState does not exist in
  AlreadyVisitedStates )
• 6. AlreadyVisitedStates AlreadyVisitedStates
  U currentState
• 7. for ( all possible events e )
• 8. if (e.EnablingFunction() true)
• 9. CopyFromModelToEntities(currentState)
• 10. nextState currentState
• 11. e.EventHandler()
• 12. CopyFromEntitiesToModel(nextState)

11
Suggested Solution Implementation (cont.)
13. if (nextState does not exist in
AlreadyVisitedStates) 14. if (
nextState.verify() false ) 15.
Print("modelCheck FOUND ERROR STATE
") 16. nextState.printState() 17.
Print("ERROR TRACE ") 18.
printErrorTrace(nextState) 19.
exit(-1) 20. else
21. NonVisitedStates NonVisitedStates U
nextState
22. exit(1)
12
Suggested Solution Implementation (cont.)

• Implementation Problems
• Ports (drcl.comp.Port)
• Use the function-call execution model instead of
  the default independence execution model
• ACA Timers (drcl.comp.ACATimer)
• Suggested Solution
• Make the modification in a subclass of the J-Sim
  class of a network protocol entity

13
A Proof-of-Concept Case Study ARQ
Data Lost
ACK Lost
Normal Case
14
Case Study ARQ (cont.)

• Protocol-specific Heuristics
• ACKs are not lost
• Network Cloud (i.e., without specifying the
  structure of the network between the two
  end-points)
• Consider packet loss as a one single event
  regardless of the cause of the loss (e.g.,
  router/link failure, congestion, packet
  corruption, etc.)
• Data packets are not reordered
• Best-first Search

15
Comparison with Maude

• Maude ( http//maude.cs.uiuc.edu/ )
• Network Protocol expressed in Full Maude as an
  OO module in Rewriting Logic
• Assertions expressed in Linear Temporal Logic
  (LTL)
• Why Maude?
• Very powerful both theorem proving (for
  equational logic) and model checking (for
  rewriting logic)
• Maude LTL Model Checker has performance
  comparable to SPIN
• Maude LTL Model Checker can handle only
  finite-state systems

16
Performance Evaluation
Time needed to find the error (in msec.)

• Best-first search reduces the time needed to
  find the error
• Protocol-specific heuristic reduces time even
  more
• Almost half the time needed by Maude LTL model
  checker

17
Performance Evaluation (cont.)
Length of the error trace (in number of states)

• Shorter error trace
• () Use of J-Sim saves the network protocol
  designer from having to build another prototype
  in Maude

18
Conclusions

• J-Sim with Error-Finding Capabilities
• Reduce Time and Effort
• A proof-of-concept case-study
• Finding violations of safety property
• Protocol-specific Heuristics, Best-First Search
• Reduce time needed to find errors
• Comparison with Maude LTL model checker
• Less time and shorter error traces than Maude
  LTL
• Need to build only one prototype

19
Future Work

• Model Checking complex/standard network
  protocols
• AODV
• DSR
• Checking general LTL formulae
• Comparison with other model checkers
• Reducing the network protocol designers effort
• Automatic specification of the state
• Automatic determination of protocol-specific
  heuristics