Greg Wilson

1
CSC407 Software ArchitectureWinter
2007Performance (II)

• Greg Wilson
• gvwilson_at_cs.utoronto.ca

2
Model Revisited
Service Rate µ
FCFS
Arrival Rate ?
Server
Queue

• Parameters
• Service Order (FCFS)
• Average Arrival Rate (?)
• Service Requirement (S)
• Mean Service Time (ES)
• Average Service Rate (µ 1/ES)
• For single server model, we require ?µ

3
Performance Metrics

• Response Time, Time in System (Ts)
• Waiting Time (Tw)
• Number of jobs in System (Ns)
• Number of jobs in Queue (Nq)
• Utilization (Ui) Ui Bi/Ti
• Throughput (Xi) Xi Ci/Ti µi Ui
• Utilization Law Ui Xi . ES

4
Exponential Distribution

• Let q(t) be the probability that nothing happens
  in time interval t
• If x represents the waiting time to the first
  event, then by definition Pxgtt q(t)
• If the system is memoryless, then
• q(t1)q(t2) q(t1 t2)
• Only solution is q(t) e-?t
• So Fx(t) Pxt 1-q(t) 1-e-?t

5
Another View

• Let s and t be time intervals 0

Pxgtts x gt s Px gt ts / Px gt s
e-(ts) / e-s
e-t
Pxgtt

• Probability of something happening is independent
  of whatever has (or hasnt) happened before)
• Note mean and standard deviation are both 1/?
• ? is often called the rate

6
How Many?

• If interarrival times are exponentially
  distributed, then Pk events in 0,t
  ?te-?t/k!
• A Poisson distribution with parameter ?
• So if the arrival and service times are both
  memoryless, what is the systems performance?
• I.e., if jobs arrive independently, and the times
  required to process jobs are independent, how
  quickly can we do them?
• And how long will the queue get?

7
M/M/1 Queue

• Memoryless property If the last job arrived t0
  time ago, no job will arrive for time t with the
  probability P0(t) e-?t.
• For small ?t,
• P0(?t) 1 - ? ?t
• P1(?t) ? ?t
• Pk(?t) 0 for k gt 1
• The arrival process can be viewed as an infinite
  state machine where each state is the number of
  jobs in the queue

8
M/M/1 Queue

• Service time also follows exponential
  distribution.
• So the departure process can be viewed as

2
1
3
4
5
6
1
0
µ
µ
µ
µ
µ
µ
9
M/M/1 Queue

• The M/M/1 queue can be viewed as

• The M means Markovian
• Or memoryless
• The 1 means a single server
• So whats the probabiliyt of being in state N?

10
Markov Models

• A way to translate transition probabilities into
  occupation times
• Among other things

0.4
0.6
A
B
0.2
0.1
0.8
0.3
0.2
C
D
0.2
0.7
0.5
11
Markov Models
0.6PA 0.2PB 0.1PC 0.3PD
PB 0.6PA
0.2PC 0.2PD
0.5PD 0.8PB 0.2PC

• Under-determined
• But PA PB PC PD 1.0

PA 0.2644 PB 0.1586
PC 0.2308 PD 0.3462
12
Using the Model

• Data center has M machines
• ? fail per hour
• Takes h hours to repair, so ? 1/h
• How often are all machines running?
• How often are fewer than half machines running?
• Often written into Service Level Agreements
  (SLAs)
• If halving the failure rate costs F/machine, and
  halving the repair time costs R, which is the
  better investment?

13
Generalized Birth-Death Models

• A restricted class of Markov models

?1
?0
?2
?3
0
1
2
3

?1
?0
?2
?3
flow in flow out
?1P1 ?0P0
?0P0 ?2P2 ?1P1 ?1P1

P0 P1 1
14
After a Little Algebra
? i-1 -1
P0 ? ? ?i/?i1 ?i/?i1
k0 i0

i-1
Pk ? ?i/?i1 ?i/?i1 P0
i0
utilization P1 P2 1 P0
throughput ?1P1 ?2P2 ? ?kPk
queue length 0P0 1P1 2P2 ? kPk
response time queue length / throughput ? kPk / ? ?kPk
15
M/M/1 System Steady State
16
M/M/1 System Steady State
17
M/M/1 System Steady State
18
M/M/1 System Performance Measures

• Utilization U 1 p0 ? ?/µ
• Conventional to use ? for utilization instead of
  U
• Number of customers in whole system

19
M/M/1 System Performance Measures

• Mean Response Time
• Littles Law ENs ? ETs

• Remember µ ?
• Hm what if µ ? ?

20
M/M/1 System Performance Measures

• Mean Wait Time in queue (Tw TQ)
• Number of jobs waiting in queue Nw
• Using Littles Law ENw ? ETw
• 
  ?2 / (1 - ?)

21
Example

• File server 30 requests/sec, 15 msec/request
• What is the average response time?
• What would it be if the arrival rate doubled?
• Assume requests are independent (must verify)
• Utilization ? ?ES 0.45
• So average response time is ES/(1 - ?) 0.027
  sec
• At 60 requests/sec, response time is 0.15 sec
• Doubling request rate increased response time by
  5.6!