Network Simulation and Testing

1
Network Simulation and Testing

• Polly Huang
• EE NTU
• http//cc.ee.ntu.edu.tw/phuang
• phuang_at_cc.ee.ntu.edu.tw

2
Case Study Papers

• J. Padhye, V. Firoiu, D. Towsley, and J. Kurose.
  Modeling TCP throughput A simple model and its
  empirical validation. In Proceedings of the ACM
  SIGCOMM Conference, pages 303-314, Vancouver,
  Canada, September 1998. ACM
• J. Broch, D. Maltz, D. Johnson, Y. Hu, J.
  Jetcheva, A Performance Comparison of Multi-Hop
  Wireless Ad Hoc Network Routing Protocols. In the
  Proceedings of the Fourth Annual ACM/IEEE
  International Conference on Mobile Computing and
  Networking (MobiCom'98)
• M. Christiansen, K. Jeffay, D. Ott, and F.
  Donelson Smith. Tuning RED for web traffic. In
  ACM SIGCOMM2000, August 2000

3
Doing Your Own Analysis

• Having a problem
• Need to simulate or to test
• Define experiments
• Base scenarios
• Scaling factors
• Metrics of investigation

4
Base Scenarios

• The source models
• To generate realistic traffic
• The topology models
• To generate realistic networks
• The routing policy models
• To compute realistic paths
• The packet-drop/route-failure models
• To generate realistic failures

5
Then?
6
AgainDoing Your Own Analysis

• Having a problem
• Need to simulate or to test
• Define experiments
• Base scenarios
• Scaling factors
• Metrics of investigation

7
Case StudiesInternet Performance Analysis

• Model Validation
• System Comparison
• Parameter Tuning

8
Types of Analysis Paper

• Model validation
• System comparison
• Parameter tuning

9
1. Model Validation

• An analytical model of a system
• Based on understanding of the system and/or
  physical observations
• Could be in various forms
• Ex. differential equations, markov chains, etc
• Need to validate it IS whats going on in the
  real world
• The experiments
• Varying a parameter
• To compare results from
• The model, simulations, or live tests

10
2. System Comparison

• A number of competing design of the same system
• Based on the common wisdom no pay, no gain
• No one works the best all the time
• Need to know which works better in what
  circumstances
• The experiments
• Varying a parameter
• To compare results from
• The various designs

11
3. Parameter Tuning

• A set of parameter tunable about a design
• Different parameter settings could give quite
  different system performance
• Due to the complexity of a design
• Need to find out what would be the optimal
  settings in what circumstances
• The experiments
• For a set of performance metric of interest
• To compare system performance from
• The various parameter settings

12
Case StudiesInternet Performance Analysis

• Model Validation
• System Comparison

13
Modeling TCP

• A simple model and its empirical validation
• J. Padhye, V. Firoiu, D. Towsley, and J. Kurose.
  Modeling TCP throughput
• In Proceedings of the ACM SIGCOMM Conference,
  pages 303-314, Vancouver, Canada, September 1998.
  ACM

14
The Motivation

• A part of the telephone network success is due to
  the understanding of voice traffic
• i.e., throughput of a call is 64kbps
• The rationale
• Want the Internet to be at least as successful
• TCP connections is comparable to the voice calls
• Main traffic source
• Also in the form of flows
• Need to know throughput of a TCP connection

15
Voice Call is Simple

• Constant 64 kbps
• 8 samples per millisecond
• 8 bits per sample

16
TCP not quite as simple
17
Reno TCP

• Increase congestion window size
• slow start (cwnd lt ssh) cwnd 1
• steady state (cwnd ? ssh) cwnd 1/cwnd
• Decrease congestion window size
• duplicated acknowledges cwnd cwnd/2
• timeout cwnd 1
• ssh cwnd/2

18
TCP With Different ssh
cwnd1
cwnd2
cwnd4
ssh 20
source
sink
cwnd1
cwnd2
cwnd3
ssh 2
source
sink
21/2 5/2 5/2 2/5 29/10
19
10-second Quiz
Both curves represent some TCP attributes
behavior. What are they?
ssh
ssh
20
Queuing Approach
Assume fixed packet size MSS BW AMSS/(Te-Ts)
cwnd
ssh
A
Time
Ts
Te
21
The Magical (1/p)1/2

• Show in a simplified analysis
• infinitely long TCP connections
• only in the steady state
• cwnd 1 per RTT
• no timeouts
• only duplicated acknowledges
• cwnd / 2 per drop
• Average Bandwidth
• MSS/RTT (3/2p)1/2

22
Saw Tooth Behavior
23
Deriving BW
W average tooth tip W/2 average tooth dip
Total number of packets sent between two packet
drops is (W/2 W) (W/2) /2 (3/8)W2
W
W/2
p probability of packet loss (3/8) W2 1/p
W (8/3p)1/2 BW MSS (3/8) W2 / (RTT W/2)
MSS/RTT (3/2p)1/2
RTT W/2
24
A Brief History
Steady Connections
Short Connections
25
Terminology

• TCP in rounds
• A round is a flight of packets until their ACKs
• I.e., a round trip time (RTT)
• TDP
• Triple duplicate ACK period
• A series of rounds followed by a packet drop due
  totriple dup ACKs
• I.e., a packet-loss free period

26
Notation

• Wi
• Window size at round i
• I.e., number of packets sent within round I
• b
• Number of data packets to trigger an ACK
• In average TCP implementations, the TCP receiver
  sends back an ACK every other data packets
• I.e., 2 rounds to increase the window size by 1

27
TCP in Rounds
28
TCP in TDPs
Wi now window size at the end of TDPi Ai
number of rounds in TDPi
29
Deriving BW
Y number of pkts sent in TDP A duration of TDP
in seconds Solve Bandwidth W via solving Y and
A EY EW EA
30
Solving EY, EA
r round trip time RTT Er
31
Basically

• Make assumptions when necessary
• TD losses are i.i.d.
• Xi and Wi are mutually i.i.d.
• Solve for mean values using probability

32
Result for Triple-Dup-ACKs
conclusion for triple-dup-ACK steady state (in
segments)
p pkt loss prob
Padhye98a, eqn 20
compare to Mathis et als earlier result (in
bytes)
Confidence check!
33
Real Contribution

• Extending the model to include timeout losses

34
Extending the Period of Consideration

• TD losses interleaving with TIMEOUT losses
• A random number of TD losses between 2 TIMEOUT
  losses
• Si
• The time period between 2 TIMEOUT losses
• ZiTD part of Si containing consecutive TDPs
• ZiTO part of Si containing recovering from the
  TIMEOUT loss

35
Deriving New BW
EnEYER EW
EnEAEZTO n number of TDPs R number of
retransmissions due to TO
36
Solving for the New Terms

• Easier
• ER 1/(1-p)
• Complicated
• En 1/Q

37
BW with Timeout Loss
Padhye98a, eqn 29
38
Modeling a Limited Window

• Define Wmax as the window limit
• What is performance if window limited?
• Data sent per round Wmax
• Rounds until TDP EX
• EY ? Wmax EX
• EA (EX1)RTT ?EXRTT
• EW ? Wmax/RTT

39
BW with Limited Window
Padhye98a, eqn 32
B( Wmax, RTT, p, b, To)
40
OK. Nice math but does it work?
41
Validation Approach

• Set up bulk transfers
• Infinite source TCP flows
• Compare measured data vs. old model vs. their
  model
• Actual vs. no timeout model vs. with timeout model

42
Data Collection

• 2 set of measurement
• Between 18 hosts
• tcpdump at the sender
• N1 1997-1998 beginning
• 24 connections
• 1 hour long each ? 1x1hr
• N2 1998
• 13 connections
• 100 transfers each connection
• 100 second long each transfer ? 100x100s

43
Getting Actual BW

• Known B(Wmax, RTT, p, b, To)
• Wmax from TCP sync packet
• b 2
• Run their own analysis tool to get
• p
• Total of packets sent
• Total of losses TD losses and TO losses
• Significant amount of TO losses
• RTT average over the entire connection
• To average over the entire connections single
  To observations

44
Qualitative Comparisons

• Given a connections RTT, To, and Wmax
• Draw estimations from old and new model
• Old model line
• New model line
• Plot for every 100sec interval the p at x-axis
  and BW at y-axis
• 36 points for 1st measurement set
• 100 points for 2nd measurement set
• Points aligning to the new model estimation
• 6 such figures for each data set

45
Validation
Padhye98a, figs 7 9
(window limited)
(non-window limited)
46
Quantitative Comparison

• Calculate the estimation error
• Take the difference of the estimated and observed
  BW
• Divided by the observed BW
• Average for all points
• New model is better
• Except for a couple cases there are mostly TD
  losses
• The approximation is kind of OK

47
Reality Check

• The model not considering
• Fast recovery rare
• Slow start negligible
• Packet loss
• Losses within a round correlated drop-tail queue
• Losses across rounds are independent RTT away OK
• RTT independent of W
• True for most cases
• Not true for a modem case

48
Strange Stuff

• TCP-Reno implementations vary
• Linux
• 2 duplicate ACKS (3 same ACKs)
• Exponential backoff isnt quite as stated in the
  spec
• Irix
• Exponential backoff limited to 25, not 26
• SunOS
• More likely TCP-Tahoe

49
Quick Summary

• Extend TCP BW model to include
• Timeout
• Maximum window size
• Validate by live measurements
• 2 data sets
• Qualitative and quantitative comparisons
• Check on the applicability of assumptions

50
Questions?
51
Case StudiesInternet Performance Analysis

• Model Validation
• System Comparison

52
Comparing 4 Ad-Hoc Routing Protocols

• A Performance Comparison of Multi-Hop Protocols
• J. Broch, D. A. Maltz, D. B. Johnson, Y. C. Hu,
  and J. Jetcheva.
• In Proc. of the ACM/IEEE MobiCom, October 1998

53
Ad Hoc Network

• A collection of wireless mobile nodes
• Dynamically forming a temporary network

54
Features

• Without the use of any existing network
  infrastructure or centralized administration
• Infrastructure-less networking
• Little or no communication infrastructure
• Expensive or inconvenient to establish/use
  infrastructure
• No central administration
• Some overlay network
• Some peer-to-peer networks

55
Examples

• Students using laptops for interactive lectures
• Business associates sharing information during a
  meeting
• Soldiers relaying information for situational
  awareness on the battlefield
• Disaster relief personnel coordinating efforts
  after an earthquake

56
Ad Hoc Routing

• Finding a path from the source to the destination
  in ad hoc networks
• Multi-hop exchange
• Each host is also a router

57
These Four

• DSDV
• Destination-Sequenced Distance Vector
• TORA
• Temporally-Ordered Routing Algorithm
• DSR
• Dynamic Source Routing
• AODV
• Ad Hoc On-Demand Distance Vector

58
Destination-Sequenced Distance Vector (DSDV)

• Presented SIGCOMM 94 by Perkins and Bhagwat
• Each node contains a routing table for each hop
  with sequence number and metric
• Each node advertises a monotonically increasing
  even sequence number
• Greatest sequence number is the more favorable
  route.
• Guaranteed Loop-freedom
• With the use of sequence number

59
DSDV Example
Updated Forwarding Table
60
Temporally-Ordered Routing Algorithm (TORA)

• Presented INFOCOM 97 by Park and Carson
• Designed to Minimize overhead and discover routes
  on demand
• Think about it as water flowing through tubes on
  its way to a destination
• Node broadcasts a QUERY packet, recipient
  broadcasts an UPDATE packet
• Uses IMEP as transport
• Reliable, in-order transmission

61
Route Creation Example
62
Dynamic Source Routing (DSR)

• Published in Mobile Computing, 96 by Johnson and
  Maltz
• Uses source rather then hop-by-hop routing, each
  packet contains list of nodes for packet to pass
  through.
• No need for up-to-date routing information, more
  importantly eliminates need for periodic route
  advertisement

63
DSR (cont)

• Route Discovery
• Flood route request message
• Request answered with route reply by
• Destination
• Optimized if some other node that knows the way
• Route Maintenance
• If 2 nodes listed next to each other in route
  move out of range
• Return route error message to sender
• Sender can either use another route in its cache
  or invoke Route Discovery Again.

64
Ad Hoc On-Demand Distance Vector (AODV)

• Presented as Internet-Draft (Currently on Version
  12), Perkins and Royer, 1997
• Takes the basic on-demand mechanism of Route
  Discovery and Maintenance from DSR, plus
  hop-by-hop routing, etc from DSDV
• Hello messages are passed between routes every
  second, Failure to receive 3 consecutive means
  link is taken down

65
AODV Example
Route Request
Route Reply
66
Test Methodology

• All tests based on
• 50 wireless nodes
• Rectangular flat place, 1500m x 300m
• 900 seconds of simulated run time
• 7 Different Pause Times, which is how long each
  node remains stationary
• 0,30,60,120,300,600, and 900 (no motion)
• 10 movement patterns for each pause time, 70
  total
• 20 m/s Max node speed (10 avg.), also used 1m/s

67
Metrics

• Packet delivery ratio Application layer packets
  originated at source to received packets
• Characterizes completeness and correctness of the
  routing protocol
• Routing overhead Total of packets sent during
  transmission
• Scalability
• Path optimality Difference between number of
  hops a packet took and the shortest path measured
• Measures the ability to efficiently use network
  resources.

68
Packet Delivery vs. Pause time (20 sessions)
69
Routing OH vs. pause time (20 sessions)
70
(No Transcript)
71
(No Transcript)
72
Path Optimality
73
Change of Speed (20m/s -gt 1ms)
74
(No Transcript)
75
Quick Summary

• Detailed packet-level simulation of 4 recent
  routing protocols
• DSDV performs predictably, not good when mobility
  increases
• TORA uses large amounts of OH, delivered packets
  well
• DSR was good at all speeds and rates!
• AODV does almost as good, but more OH makes it
  more expensive then DSR

76
Questions?