Verisim: Formal Analysis of Network Simulations - PowerPoint PPT Presentation

About This Presentation
Title:

Verisim: Formal Analysis of Network Simulations

Description:

We instrumented simulation code for AODVv0 supplied by the CMU Monarch Project. ... Simulation from Monarch uses 50 mobile nodes on 1500x300m grid moving 20 m/s. ... – PowerPoint PPT presentation

Number of Views:23
Avg rating:3.0/5.0
Slides: 32
Provided by: bhargavano
Category:

less

Transcript and Presenter's Notes

Title: Verisim: Formal Analysis of Network Simulations


1
Verisim Formal Analysis of Network Simulations
  • Karthikeyan Bhargavan, Carl A. Gunter, Moonjoo
    Kim, Insup Lee, Davor Obradovic, Oleg Sokolsky,
    Mahesh Viswanathan
  • University of Pennsylvania

2
Internetwork Routing Protocols
  • Internetwork routing protocols enable
    interoperability between physical networks.
  • Routing protocols for the Internet RIP, OSPF,
    and BGP.
  • Routing protocols for packet radio AODV and DSR.
  • Routing protocols and software have growing
    importance and complexity.

3
Current Analysis Techniques
  • Rigorous mathematical models and proofs limited
    by complexity.
  • Testing.
  • Testbed expensive.
  • Operational risky and inconvenient.
  • Simulation.
  • Performance attributes throughput, latency,
    reliability, etc.

4
Limitations of Performance Analysis of Simulations
  • Flaws may not be detected if no expected
    performance can be used for comparison.
  • When flaws are suspected, finer means of analysis
    are useful.
  • Some flaws do not manifest themselves as
    performance problems (e.g. most security gaps).

5
Verisim
  • Verisim provides support for logical analysis
    of network simulations.
  • This talk describes its architecture and logic.
  • We provide a series of experiments aimed at
    assessing the approach.

6
Monitoring and Checking (MaC) Framework
NS
MEDL
7
Meta Event Description Language (MEDL)
  • Expresses properties of traces.
  • Extension of Linear Temporal Logic (LTL) with
    auxiliary variables.
  • More expressive than LTL.
  • Properties classified into (instantaneous) events
    and (enduring) conditions.

S Kannan, M Kim, I Lee, O Sokolsky, M Viswanathan
98
8
NS Network Simulator
Instrumented Protocol Code P C
Protocol Agents
P
P
P
Scenario
Configuration Parameters OTcl
Network Model
N
N
N
Topology OTcl
Traffic Agents
src/sink
src/sink
src/sink
Traffic Model OTcl
NS Trace
VINT Project
9
Verisim
Properties MEDL
Instrumented Protocol C
Checker
NS
Trace
Metatrace
Scenario OTcl
10
Ad Hoc Networks
  • Routing for a wireless internetwork without the
    aid of a central base station.
  • Connections are low-bandwidth, lossy, and highly
    transient.
  • Unique routing assumptions
  • Most routes are seldom used.
  • Bandwidth must be protected.

11
Illustration Part 1 of 2
Movement
Routing
12
Illustration Part 2 of 2
New Routing
13
AODV Protocol
  • If a node S needs a route to a destination D and
    does not have one, it floods a route-request
    (RREQ) packet through the network.
  • Each recipient R of this RREQ keeps a return
    pointer.
  • R broadcasts the request to its neighbors if it
    is not D and does not have a route to D.
  • If R is D, or has a route to D, it responds with
    a route-reply (RREP) packet using the return
    pointers for S.

Perkins and Royer 99
14
Loop Freedom in AODV
  • Routing loops are undesirable.
  • AODV uses sequence numbers to indicate freshness
    of link information.
  • Key Invariant If next(n) n, then
  • seqno(n) ? seqno(n), and
  • if seqno(n) seqno(n), then hops(n) gt hops(n).
  • The invariant ensures that there are no loops.

15
Outline of Experiment
  • Run a scenario of modest complexity.
  • Analyze it in Verisim using a list of 9
    properties of AODV expressed in MEDL.
  • First pass Repair First Bug (RFB).
  • Second pass tune the MEDL formulas to avoid
    rerunning the simulation discovering bugs in the
    metatrace.
  • We instrumented simulation code for AODVv0
    supplied by the CMU Monarch Project.

16
Experiment Scenario
1
2
3
4
5
17
Experiment Scenario
1
2
3
4
5
18
Experiment Scenario
1
2
3
4
5
19
Experiment Scenario
1
2
3
5
4
20
Experiment Scenario
1
5
2
3
4
21
Sample MEDL Alarm
alarm LoopInvatnxtdst sendroutatdst
when ((at?nxt) ? (at?dst) ? (nxt?dst) ?
(obs_nexthopatdst nxt) ?
((obs_seqnoatdst gt obs_seqnonxtdst) ?
((obs_seqnoatdst obs_seqnonxtdst)
? (obs_hopcontatdst lt
obs_hopcntatdst))))
This is the negation of the fundamental invariant
ensuring no loops in AODV If the next hop from
node at toward destination dst is node nxt then
the sequence number (for dst) of at is less than
or equal to that of nxt or they are equal and the
hop count (for dest) of at is less than or equal
to that of nxt.
22
Repair First Bug Experiment
  • Let ? be the MEDL property set.
  • Run the simulation to get a trace T.
  • Run the checker to get a metatrace T?.
  • Repair the first bug in the metatrace to get new
    protocol code.
  • Rerun the simulation with the new protocol code
    to get a new trace U.
  • Rerun the checker to get a new metatrace U?.
  • Continue until an empty metatrace is obtained.

23
RFB Experiment Statistics
Meta- trace Dest Rep Detect RErr Node Rep Loop Env Total Alarms
T? 4 54 38 43 220
U? 0 54 38 43 216
V? 0 48 39 44 206
W? 0 0 0 0 1
24
Tuning Experiment Statistics
Meta- trace Dest Rep Detect RErr Node Rep Loop Env Total Alarms
T? 4 54 38 43 220
T? 0 54 38 43 216
T? 0 0 38 50 166
T? 0 0 21 0 21
No new simulation traces.
25
Analysis of Off-The-Shelf (OTS) Simulations
  • The prior experiment involved fewer than 10,000
    events, and we designed it to exercise key
    scenarios.
  • Can we do useful analysis with OTS performance
    simulations?
  • Aim Verisim can be added with small
    modifications, run alongside OTS simulations, and
    find logical failures in a practical period of
    time.

26
Experiment
  • Took largest available OTS simulation for AODV
    and ran it to create a trace.
  • Simulation from Monarch uses 50 mobile nodes on
    1500x300m grid moving 20 m/s.
  • 5220 seconds (1.5 hours) to complete simulation.
  • 6,446,316 events.
  • NaĂŻve effort ran MEDL with MonSeqNo test on all
    nodes (2500 relations) using 550Mhz dual
    processor machine with 1GB of memory.
  • Aborted the experiment after 4 days estimate 100
    days to complete this analysis.

27
Optimizations For Network Routing Simulations
  • Population Abstraction test the property for a
    subset of the routers.
  • Packet Type Abstraction prune the trace to
    include only relevant events.

28
Sample Experiment Results for MonSeqNo
  • Population Abstraction for 5 nodes.
  • Trace size 6,446,316 events
  • Property size 14,543 bytes
  • Time 51,054 seconds
  • Rate 0.54 micro seconds per event per property.
  • Population and Packet Type Abstractions.
  • Trace size 6812 events
  • Property size 14,543 bytes
  • Time 51 seconds (found failures)
  • Rate 0.51 micro seconds per event per property.

29
Sample Experiment Results for LoopInv
  • Population Abstraction for 5 nodes.
  • Trace size 69,411 events
  • Property size 75,508 bytes
  • Time 8064 seconds
  • Rate 1.54 micro seconds per event per property.
  • Population and Packet Type Abstractions.
  • Trace size 48,735 events
  • Property size 75,508 bytes
  • Time 5912 seconds (found failures)
  • Rate 1.61 micro seconds per event per property.

30
Related Work
  • Communication-based test generation systems.
    STRESS and Verisoft.
  • Test oracles based on formulas or formal
    operational specifications. GIL and TETRA.
  • Formal instance verification of routing
    protocols. SPIN/PITHIA for PNNI.
  • Simulations based on logical specifications.
    MTSim and Maude.
  • Network monitoring systems. Emerald.

31
Summary
  • Verisim integrates simulation and logical trace
    analysis.
  • The combination provides a more flexible approach
    to analyzing network simulations for safety
    properties.
  • It is able to find bugs in existing simulator
    studies.
  • Its flexibility can be exploited to improve
    debugging turn-around time.
Write a Comment
User Comments (0)
About PowerShow.com