Multilevel Simulation of Discrete Network Models

1
Multilevel Simulation of Discrete Network Models

• John A. Drew Hamilton, Jr., Ph.D.Lieutenant
  Colonel, United States ArmyDirector, Joint
  Forces Program Office

2
Research Objectives

• Specify discrete event network simulation
  components at different levels of abstraction
  with transparent consistency.
• Examine extracted run-time components of a
  simulation.
• Alter simulation model components at various
  levels of abstraction.
• Validate the developed network simulation results
  against those obtained by using classical
  simulation procedures.

3
Why Study Networks?

• Computer networks provide the underlying support
  for the information explosion that is
  revolutionizing academia, industry and
  government.
• This increase in capability brings a concomitant
  increase in complexity.
• The current lack of easy to use , large scale
  network monitoring and modeling tools makes
  systematic study of networks difficult.

4
Research Prerequisites

• The network components must be decomposable into
  reusable generic objects.
• Varying levels of detail are possible.
• Accurate traffic can be represented.

5
Abstraction in Simulation

• Abstraction is the selective examination of
  certain aspects of a problem. The goal of
  abstraction is to isolate those aspects that are
  important for some purpose and suppress those
  aspects that are unimportant.Rumbaugh, et al,
  Object-Oriented Modeling and Design,
  Prentice-Hall.
• Model abstraction is the identification of
  relationships between models described at
  different levels of detail and with deriving more
  abstract relationships from more detailed ones.
  Sevinc, Theories of Discrete Event Model
  Abstraction, Proceedings of the 1991 Winter
  Simulation Conference.

6
Relationships
Tradeoffs detail vs. abstractiondecomposition
vs. aggregation
Wall, Multilevel Abstraction of Discrete Models
in an Interactive Object-Oriented Simulation
Environment, Ph.D. Dissertation, Texas AM
University, 1993.
7
Network Complexity Factors

• The number and variety of managed resources.
• The distribution of devices and physical
  structure of the network, as well as subnetting.
• The number and variety of communications services
  and distributed applications.
• The degree to which services are integrated and
  the associated quality of service (QoS).
• The number of organizational and administrative
  units.
• Mission of the organization.
• Hegering, Abeck Wies, A Corporation Operation
  Framework for Network Service Management, IEEE
  Communications, Jan. 1996.

8
Network Monitoring Tools

• There are a variety of commercial network
  analysis tools on the market. Law and
  McComas,Simulation Software for Communications
  Networks The State of the Art, IEEE
  Communications, Vol. 32, No. 3, Mar. 1994, pp. 44
  - 50.
• Many of these products are very expensive,
  putting them out of bounds for many researchers.
  
• Fortunately, there are some free alternatives.
• Network monitoring provides the measurements that
  ultimately validate the accuracy of a network
  simulation.
• The monitoring process is neither easy nor
  inexpensive and is limited to the configuration
  already installed and operational.

9
Network Monitoring

• An important step in simulating a complex system
  is to observe the system (if possible).

• Systematic observation also yields data for
  validating the model.
• Additionally, monitoring provides the means to
  em-ploy historical validation.
• Sargent, Simulation model verification and
  validation, Proceedings of the 1991 Winter
  Simulation Conference, p. 40.

10
Network Monitoring

• Drawbacks
• This is not predictive, just lessons learned by
  the hard taskmaster of experience.
• Experimenting on an operational network is
• risky
• costly
• interfering
• Meaningful measurement efforts must be made over
  time and will produce very large amounts of data
  to be interpreted.
• Theoretical systems cannot be directly observed.

11
Network Simulators

• NETSIM
• M.I.Ts Network Simulator
• MaRS
• Maryland Routing Simulator (University of
  Maryland)
• LANSF
• Local Area Network Simulation Facility
  (University of Alberta)
• SMURPH
• System for Modeling Unslotted Real-Time Phenomena
  (University of Alberta)

12
Commercial Simulators

• General-purpose simulation languages such as
  MODSIM, BONeS DESIGNER, GPSS/H, SIMSCRIPT II.5,
  SLAMSYSTEM, SES/workbench.
• Communications-oriented simulators such as BONeS
  PlanNet, COMNET III, LNET II.5 and NETWORK II.5.
  
• Communications-oriented simulation language such
  as OPNET Modeler.

13
Simulation Granularity
14
Simulation Granularity

• The major component of resolution is the level of
  detail that the simulation can receive as input
  and return as output.
• If the resolution of a network model is at the
  packet level, then there is no information
  provided at the bit level.
• Bits submitted as input to the model cannot be
  processed unless the bits are collected and
  formed as packets prior to processing.
• The resolution of the output will be at the
  packet level unless an external synthesizing
  function of some sort is used to manipulate the
  output.
• The output will be produced based on the
  granularity of the simulation, but the output may
  be manipulated by an external function which may
  change the granularity.

15
Variable Resolution

• Davis outlines methods of varying simulation
  resolution
• Using high resolution to provide a picture when
  the lower-resolution depiction seems too
  abstract.
• Invoking high resolution for special processes
  within the course of an otherwise low resolution
  simulation.
• Using high resolution to establish bounds for
  parametric analysis using lower-resolution
  models. (e.g., the number of retries to deliver
  a packet.)
• Using high resolution to calibrate
  lower-resolution recognizing that knowledge of
  the world comes at all levels of detail.
• Using low resolution for decision support,
  including rapid analysis of alternative courses
  of action.
• Davis, An Introduction to Variable-Resolution
  Modeling and Cross-Resolution Model Connection,
  RAND Report R-4252-DARPA, The RAND Corporation,
  Santa Monica, Calif., 1992.

16
Network Simulation

• The complexity of network simulation requires
  multiple forms of abstraction control.
• Three methods of interest are
• Multilevel Simulation
• Hierarchical Abstraction
• Aggregation

17
Multilevel Simulation
18
Hierarchical Abstraction
19
Aggregation

• Representational aggregation
• Absolute consistency

20
Resolution Summary

• Model resolution is a critical component in
  determining the utility of a simulation. Model
  fidelity is a closely related concept but one
  does not imply the other.
• Simulation models are under development which
  allow the analyst to dynamically alter the
  resolution of the various components.
• There is a fundamental need for variable
  resolution models in which there is true
  consistency across resolution levels and for
  concepts and methods making it easier to do
  cross-resolution work, including models not
  originally designed to be compatible. Davis
  Blumenthal, The Base of Sand Problem, A White
  Paper on the State of Military Combat Modeling,
  RAND Report N-3148-OSD/DARPA, 1991.
• An understanding of abstraction, resolution
  multimodeling provides the basis for an open
  simulation architecture OSA. Hamilton Pooch,
  An Open Simulation Architecture for Force XXI,
  Proceedings of the 1995 Winter Simulation
  Conference, Washington, D.C., Dec. 3 - 6, 1995,
  pp. 1296 - 1303.

21
OPNET

• Communications-oriented scripting language.
• Provides access to source code.
• Uses a multilevel modeling structure.

State Transition Diagram
22
Network Model (Upper Level)
23
Network Model (Lower Level)
24
Node Model
25
Process Model
26
Validation Structure
Does the model implementation correctly reflect
the network?
Do the predicted approximate the actual results?
A modification to Knepell and Arangnos
validation framework
Does the software perform correctly?
27
Conceptual Model Validity

• Three subnets 133, 134, 135 were monitored and
  250,000 packets were collected from each subnet.
• Data used as simulator input and three
  performance measures computed
• packet throughput in packets per second
• mean packet length
• utilization in terms of throughput mean packet
  length divided by bandwidth

28
Observed versus Expected
Expected
Observed
29
Conceptual Validation Results

• Used the Smith-Satterthwaite procedure
• for comparing means with unequal variances.
• computer test statistic for a t test.
• Packet lengths had identical means and variances,
  no further analysis needed.
• Observed throughput and utilization data had high
  variances, expected data had low variances.
• Throughput and utilization data for all subnets
  passed t-tests using the Smith-Satterthwaite
  (i.e. paired) procedure.

30
Steady State Computation

• Three stations with varying loads were selected.
  
• The packet throughput for each station was
  collected during three sixty second runs each
  using a different random number seed.
• After discarding the first ten seconds of
  transient observations, the means of the
  remaining observations were computed.
• From Udo W. Pooch. Ph.D., if the number of
  observations in which the output is greater than
  the average is about the same as the number in
  which it was less, then steady state conditions
  are likely to exist.

31
Steady State on Guru
32
Experimentation Plan

• High resolution script or table driven
• Low resolution distribution driven
• Exponentially distributed synthetic workloads
  were generated for high resolution nodes.
• Nodes set to either low or high resolution.

33
Operational Validity

• Objective Approach
• Utilization data statistically tested and found
  to be valid.
• Subjective Approach
• Throughput data found to have similar means but
  significantly different variances.

34
Summary

• Multilevel, mixed-resolution simulation can
  effectively expand the problem domains studied
  through simulation in the following ways
• improved cost efficiency
• ability to model notional network components
• ability to improve simulation run times by
  eliminating unnecessary detail
• Representational aggregation of nodes into
  subnets and subnets into collections of subnets
  provides abstraction mechanisms that allow the
  analyst to focus on areas of specific interest.
• Lack of data may dictate simulating entities at
  low resolution.

35
Original Recommendations

• Operational use of the MMRNS requires an
  automated topology builder.
• The OPNET environment provides a powerful GUI,
  but individually modeling hundreds of individual
  nodes is tedious.
• Gateways between the MMRNS and other open
  architecture tools should be built.
• Mixed-resolution simulation should be used to
  model communications networks still under
  development.

36
Results over Time

• Army DISC4 adopted OPNET as a standard due to its
  open architecture tactical version under
  development.
• Joint Staff initiates NETWARs program.
• Army SIGCEN has implemented a network modeling
  and simulation block of instruction in their FA
  24 course.
• Army ISEC has been using mixed resolution
  simulation to support base ops network design in
  their Technology Integration Center.
• ISEC in partnership with University of Arizona
  form Arizona Center for Integrative Modeling and
  Simulation.

37
Conclusion

• The Defense Department has demonstrated multiple
  interests in the extension of this research.
• This research demonstrates the efficacy of mixed
  resolution simulation. The flexibility provided
  by mixed resolution simulation may be safely
  utilized by following the methodology described
  herein.
• It was specifically demonstrated that utilization
  could be accurately simulated with as many as 75
  of the stations operating in low resolution mode.