CPE 619 Queueing Networks

1
CPE 619Queueing Networks

• Aleksandar Milenkovic
• The LaCASA Laboratory
• Electrical and Computer Engineering Department
• The University of Alabama in Huntsville
• http//www.ece.uah.edu/milenka
• http//www.ece.uah.edu/lacasa

2
Overview

• Queueing Network model in which jobs departing
  from one queue arrive at another queue (or
  possibly the same queue)
• Open and Closed Queueing Networks
• Product Form Networks
• Queueing Network Models of Computer Systems

3
Open Queueing Networks

• Open queueing network external arrivals and
  departures
• Number of jobs in the system varies with time
• Throughput arrival rate
• Goal To characterize the distribution of
  number of jobs in the system

4
Closed Queueing Networks

• Closed queueing network No external arrivals or
  departures
• Total number of jobs in the system is constant
• OUT is connected back to IN
• Throughput flow of jobs in the OUT-to-IN link
• Number of jobs is given, determine the throughput

5
Mixed Queueing Networks

• Mixed queueing networks Open for some workloads
  and closed for others Þ Two classes of jobs.
  Class types of jobs
• All jobs of a single class have the same service
  demands and transition probabilities. Within each
  class, the jobs are indistinguishable

6
Series Networks

• k M/M/1 queues in series
• Each individual queue can be analyzed
  independently of other queues
• Arrival rate l. If mi is the service rate for
  ith server

7
Series Networks (contd)

• Joint probability of queue lengths
• ? product form network

8
Product-Form Network

• Any queueing network in which
• When fi(ni) is some function of the number of
  jobs at the ith facility, G(N) is a normalizing
  constant and is a function of the total number
  of jobs in the system

9
Example 32.1

• Consider a closed system with two queues and N
  jobs circulating among the queues
• Both servers have an exponentially distributed
  service time. The mean service times are 2 and 3,
  respectively. The probability of having n1 jobs
  in the first queue and n2N-n1 jobs in the second
  queue can be shown to be
• In this case, the normalizing constant G(N) is
  3N1-2N1.
• The state probabilities are products of
  functions of the number of jobs in the queues.
  Thus, this is a product form network.

10
General Open Network of Queues

• Product form networks are easier to analyze
• Jackson (1963) showed that any arbitrary open
  network of m-server queues with exponentially
  distributed service times has a product form

11
General Open Network of Queues (contd)

• If all queues are single-server queues, the
  queue length distribution is
• Note Queues are not independent M/M/1 queues
  with a Poisson arrival process
• In general, the internal flow in such networks is
  not Poisson. Particularly, if there is any
  feedback in the network, so that jobs can return
  to previously visited service centers, the
  internal flows are not Poisson

12
Closed Product-Form Networks

• Gordon and Newell (1967) showed that any
  arbitrary closed networks of m-server queues
  with exponentially distributed service times
  also have a product form solution
• Baskett, Chandy, Muntz, and Palacios (1975)
  showed that product form solutions exist for an
  even broader class of networks

13
BCMP Networks

• 1. Service Disciplines
• First-come-first-served (FCFS)
• Processor sharing (PS)
• Infinite servers (IS or delay centers) and
• Last-come-first-served-preemptive-resume
  (LCFS-PR)
• 2. Job Classes The jobs belong to a single
  class while awaiting or receiving service at a
  service center, but may change classes and
  service centers according to fixed probabilities
  at the completion of a service request

14
BCMP Networks (contd)

• 3. Service Time Distributions
• At FCFS service centers, the service time
  distributions must be identical and exponential
  for all classes of jobs
• At other service centers, where the service times
  should have probability distributions with
  rational Laplace transforms
• Different classes of jobs may have different
  distributions
• 4. State Dependent Service
• The service time at a FCFS service center can
  depend only on the total queue length of the
  center
• The service time for a class at PS, LCFS-PR, and
  IS center can also depend on the queue length
  for that class, but not on the queue length of
  other classes
• Moreover, the overall service rate of a
  subnetwork can depend on the total number of jobs
  in the subnetwork

15
BCMP Networks (contd)

• 5. Arrival Processes
• In open networks, the time between successive
  arrivals of a class should be exponentially
  distributed
• No bulk arrivals are permitted
• The arrival rates may be state dependent
• A network may be open with respect to some
  classes of jobs and closed with respect to other
  classes of jobs

16
Non-Markovian Product Form Networks

• By Denning and Buzen (1978)
• 1. Job Flow Balance For each class, the number
  of arrivals to a device must equal the number of
  departures from the device
• 2. One Step Behavior A state change can result
  only from single jobs either entering the system,
  or moving between pairs of devices in the
  system, or exiting from the system. This
  assumption asserts that simultaneous job-moves
  will not be observed.
• 3. Device Homogeneity A device's service rate
  for a particular class does not depend on the
  state of the system in any way except for the
  total device queue length and the designated
  class's queue length. This assumption implies
  the following

17
Non-Markovian PFNs (contd)

• a. Single Resource Possession A job may not be
  present (waiting for service or receiving
  service) at two or more devices at the same time
• b. No Blocking A device renders service whenever
  jobs are present its ability to render service
  is not controlled by any other device
• c. Independent Job Behavior Interaction among
  jobs is limited to queueing for physical devices,
  for example, there should not be any
  synchronization requirements
• d. Local Information A device's service rate
  depends only on local queue length and not on
  the state of the rest of the system

18
Non-Markovian PFNs (contd)

• e. Fair Service If service rates differ by
  class, the service rate for a class depends only
  on the queue length of that class at the device
  and not on the queue lengths of other classes.
  This means that the servers do not discriminate
  against jobs in a class depending on the queue
  lengths of other classes
• 4. Routing Homogeneity The job routing should be
  state independent. The routing homogeneity
  condition implies that the probability of a job
  going from one device to another device does not
  depend upon the number of jobs at various devices

19
Machine Repairman Model

• Originally for machine repair shops
• A number of working machines with a repair
  facility with one or more servers (repairmen)
• Whenever a machine breaks down, it is put in the
  queue for repair and serviced as soon as a
  repairman is available

• Scherr (1967) used this model to represent a
  timesharing system with n terminals
• Users sitting at the terminals generate requests
  (jobs) that are serviced by the system which
  serves as a repairman
• After a job is done, it waits at the
  user-terminal for a random think-time''
  interval before cycling again

20
Central Server Model

• Introduced by Buzen (1973)
• The CPU is the central server that schedules
  visits to other devices
• After service at the I/O devices the jobs return
  to the CPU

21
Types of Service Centers

• Three kinds of devices
• 1. Fixed-capacity service centers Service time
  does not depend upon the number of jobs in the
  device
• For example, the CPU in a system may be modeled
  as a fixed-capacity service center.
• 2. Delay centers or infinite server No queueing.
  Jobs spend the same amount of time in the device
  regardless of the number of jobs in it. A group
  of dedicated terminals is usually modeled as a
  delay center.
• 3. Load-dependent service centers Service rates
  may depend upon the load or the number of jobs in
  the device., e.g., M/M/m queue (with m gt 2 )
• A group of parallel links between two nodes in a
  computer network is another example

22
Summary

• Product form networks Any network in which the
  system state probability is a product of device
  state probabilities
• Jackson Network of M/M/m queues
• BCMP More general conditions
• Denning and Buzen Even more general conditions

23
Operational Laws
24
Overview

• What is an Operational Law?
• Utilization Law
• Forced Flow Law
• Littles Law
• General Response Time Law
• Interactive Response Time Law
• Bottleneck Analysis

25
Operational Laws

• Relationships that do not require any assumptions
  about the distribution of service times or
  inter-arrival times
• Identified originally by Buzen (1976) and later
  extended by Denning and Buzen (1978)
• Operational ? Directly measured
• Operationally testable assumptions ? assumptions
  that can be verified by measurements
• For example, whether number of arrivals is equal
  to the number of completions?
• This assumption, called job flow balance, is
  operationally testable
• Statement a set of observed service times is or
  is not a sequence of independent random
  variables is not operationally testable

26
Operational Quantities

• Quantities that can be directly measured during
  a finite observation period
• T Observation interval Ai number of
  arrivals
• Ci number of completions Bi busy time Bi

27
Utilization Law

• This is one of the operational laws
• Operational laws are similar to the elementary
  laws of motion For example,
• Notice that distance d, acceleration a, and time
  t are operational quantities. No need to consider
  them as expected values of random variables or to
  assume a distribution

28
Example 33.1

• Consider a network gateway at which the packets
  arrive at a rate of 125 packets per second and
  the gateway takes an average of two milliseconds
  to forward them
• Throughput Xi Exit rate Arrival rate 125
  packets/second
• Service time Si 0.002 second
• Utilization Ui Xi Si 125 ? 0.002 0.25 25
  
• This result is valid for any arrival or service
  process. Even if inter-arrival times and
  service times to are not IID random variables
  with exponential distribution

29
Forced Flow Law

• Relates the system throughput to individual
  device throughputs
• In an open model, System throughput of jobs
  leaving the system per unit time
• In a closed model, System throughput of jobs
  traversing OUT to IN link per unit time
• If observation period T is such that Ai Ci?
  Device satisfies the assumption of job flow
  balance
• Each job makes Vi requests for i-th device in the
  system
• If the job flow is balanced and C0 is of jobs
  traversing the outside link gt Ci of jobs
  visiting the i-th deviceCi C0 Vi or Vi
  Ci/C0 Vi is called visit ratio

30
Forced Flow Law (contd)

• System throughput

31
Forced Flow Law (contd)

• Throughput of ith device
• In other words
• This is the forced flow law

32
Bottleneck Device

• Combining the forced flow law and the utilization
  law, we get
• Here DiVi Si is the total service demand on the
  device for all visits of a job
• The device with the highest Di has the highest
  utilization and is the bottleneck device

33
Example 33.2

• In a timesharing system, accounting log data
  produced the following profile for user programs
• Each program requires five seconds of CPU time,
  makes 80 I/O requests to the disk A and 100 I/O
  requests to disk B
• Average think-time of the users was 18 seconds
• From the device specifications, it was determined
  that disk A takes 50 milliseconds to satisfy an
  I/O request and the disk B takes 30 milliseconds
  per request
• With 17 active terminals, disk A throughput was
  observed to be 15.70 I/O requests per second
• We want to find the system throughput and device
  utilizations

34
Example 33.2 (contd)

• Since the jobs must visit the CPU before going to
  the disks or terminals, the CPU visit ratio is

35
Example 33.2 (contd)

• Using the forced flow law, the throughputs
  are
• Using the utilization law, the device
  utilizations are

36
Transition Probabilities

• pij Probability of a job moving to jth queue
  after service completion at ith queue
• Visit ratios and transition probabilities are
  equivalent in the sense that given one we can
  always find the other
• In a system with job flow balance
• i 0 ? visits to the outside link
• pi0 Probability of a job exiting from the
  system after completion of service at ith device
• Dividing by C0 we get

37
Transition Probabilities (contd)

• Since each visit to the outside link is defined
  as the completion of the job, we have
• These are called visit ratio equations
• In central server models, after completion of
  service at every queue, the jobs always move back
  to the CPU queue

38
Transition Probabilities (contd)

• The above probabilities apply to exit and
  entrances from the system (i0), also. Therefore,
  the visit ratio equations become
• Thus, we can find the visit ratios by dividing
  the probability p1j of moving to jth queue from
  CPU by the exit probability p10

39
Example 33.3

• Consider the queueing network
• The visit ratios are VA80, VB100, and VCPU181.
• After completion of service at the CPU the
  probabilities of the job moving to disk A, disk
  B, or terminals are 80/181, 100/181, and 1/181,
  respectively. Thus, the transition probabilities
  are 0.4420, 0.5525, and 0.005525

40
Example 33.3 (contd)

• Given the transition probabilities, we can find
  the visit ratios by dividing these probabilities
  by the exit probability (0.005525)

41
Little's Law

• If the job flow is balanced, the arrival rate is
  equal to the throughput and we can write

42
Example 33.4

• The average queue length in the computer system
  of Example 33.2 was observed to be 8.88, 3.19,
  and 1.40 jobs at the CPU, disk A, and disk B,
  respectively. What were the response times of
  these devices?
• In Example 33.2, the device throughputs were
  determined to be
• The new information given in this example is

43
Example 33.4 (contd)

• Using Little's law, the device response times are

44
General Response Time Law

• There is one terminal per user and the rest of
  the system is shared by all users.
• Applying Little's law to the central subsystem
• Q X R
• Here,
• Q Total number of jobs in the system
• R system response time
• X system throughput

45
General Response Time Law (contd)

• Dividing both sides by X and using forced flow
  law
• or,
• This is called the general response time law
• This law holds even if the job flow is not
  balanced

46
Example 33.5

• Let us compute the response time for the
  timesharing system of Examples 33.2 and 33.4
• For this system
• The system response time is
• The system response time is 68.6 seconds

47
Interactive Response Time Law

• If Z think-time, R Response time
• The total cycle time of requests is RZ
• Each user generates about T/(RZ) requests in T
• If there are N users
• or
• R (N/X) - Z
• This is the interactive response time law

48
Example 33.6

• For the timesharing system of Example 33.2, we
  can compute the response time using the
  interactive response time law as follows
• Therefore
• This is the same as that obtained earlier in
  Example 33.5.

49
Bottleneck Analysis

• From forced flow law
• The device with the highest total service demand
  Di has the highest utilization and is called the
  bottleneck device
• Note Delay centers can have utilizations more
  than one without any stability problems.
  Therefore, delay centers cannot be a bottleneck
  device
• Only queueing centers used in computing Dmax
• The bottleneck device is the key limiting factor
  in achieving higher throughput

50
Bottleneck Analysis (contd)

• Improving the bottleneck device will provide the
  highest payoff in terms of system throughput
• Improving other devices will have little effect
  on the system performance
• Identifying the bottleneck device should be the
  first step in any performance improvement project

51
Bottleneck Analysis (contd)

• Throughput and response times of the system are
  bound as follows
• and
• Here, is the sum of total
  service demands on all devices except terminals
• These are known as asymptotic bounds

52
Bottleneck Analysis Proof

• The asymptotic bounds are based on the following
  observations
• 1. The utilization of any device cannot exceed
  one. This puts a limit on the maximum obtainable
  throughput
• 2. The response time of the system with N users
  cannot be less than a system with just one user.
  This puts a limit on the minimum response time
• 3. The interactive response time formula can be
  used to convert the bound on throughput to that
  on response time and vice versa

53
Proof (contd)

• For the bottleneck device b we have
• Since Ub cannot be more than one, we have

54
Proof (contd)

• With just one job in the system, there is no
  queueing and the system response time is simply
  the sum of the service demands
• Here, D is defined as the sum of all service
  demands
• With more than one user there may be some
  queueing and so the response time will be higher.
  That is

55
Proof (contd)

• Combining these bounds we get the asymptotic
  bounds.

56
Typical Asymptotic Bounds
57
Typical Asymptotic Bounds (contd)

• The number of jobs N at the knee is given by
• If the number of jobs is more than N, then we
  can say with certainty that there is queueing
  somewhere in the system
• The asymptotic bounds can be easily explained to
  people who do not have any background in queueing
  theory or performance analysis

58
Example 33.7

• For the timesharing system considered in Example
  33.2
• The asymptotic bounds are

59
Example 33.7 Asymptotic Bounds
60
Example 33.7 (contd)

• The knee occurs at
• or
• Thus, if there are more than 6 users on the
  system, there will certainly be queueing in the
  system.

61
Example 33.8

• How many terminals can be supported on the
  timesharing system of Example 33.2 if the
  response time has to be kept below 100 seconds?
• Using the asymptotic bounds on the response time
  we get
• The response time will be more than 100, if
• That is, if
• the response time is bound to be more than 100.
  Thus, the system cannot support more than 23
  users if a response time of less than 100 is
  required.

62
Summary

• Symbols