Complexity measures

1
Complexity measures
2
Common measures

• Space complexity
• How much space is needed per process to run an
  algorithm?
• (measured in terms of N, the size of the network)
• Time complexity
• What is the max. time (number of steps) needed to
  complete the
• execution of the algorithm?
• Message complexity
• How many message are needed to complete the
  execution of the algorithm?

3
An example
Consider broadcasting in an n-cube (here n3) N
total number of processes 2n 8
Each process j gt 0 has a variable xj,
whose initial value is arbitrary
source
4
Broadcasting using messages

• Process 0 m.value x0
• send m to all neighbors
• Process i gt 0
• repeat
• receive m m contains the value
• if m is received for the first time
• then xi m.value
• send xi to each neighbor j gt
  I
• else discard m
• end if
• forever
• What is the (1) message complexity
• (2) space complexity per process?

m
m
m
Number of edges log2N
5
Broadcasting using shared memory

• Process 0 x0 v
• Process i gt 0
• repeat
• if ? a neighbor j lt i xi ? xj
• then xi xj (PULL DATA)
• this is a step
• else skip
• end if
• forever
• What is the time complexity?
• (i.e. how many steps are needed?)

Arbitrarily large!
6
Broadcasting using shared memory

• Now, use large atomicity, where
• in one step, a process j reads the states
• of ALL neighbors with smaller id, and
• updates xj only when these are equal,
• but different from xj.
• What is the time complexity?
• How many steps are needed?
• The time complexity is now O(n2)

7
Time complexity in rounds

• Rounds are truly defined for synchronous
• systems. An asynchronous round consists
• of a number of steps where every process
• (including the slowest one) takes at least
• one step. How many rounds will you need
• to complete the broadcast using the large
• atomicity model?

An easier concept is that of synchronous
processes executing their steps in lock-step
synchrony
8
Representing distributed algorithms
Why do we need these? Dont we already know a
lot about programming? Well, you need to
capture the notions of atomicity,
non-determinism, fairness etc. These concepts are
not built into languages like JAVA, C, python
etc!
9
Syntax semantics guarded actions

• ltguard Ggt ? ltaction Agt
• is equivalent to
• if G then A
• (Borrowed from E.W. Dijkstra A Discipline
  of Programming)

10
Syntax semantics guarded actions

• Sequential actions S0 S1 S2 . . . Sn
• Alternative constructs if . . . . . . . . . . fi
• Repetitive constructs do . . . . . . . . . od
• The specification is useful for representing
  abstract algorithms, not executable codes.

11
Syntax semantics

• Alternative construct
• if G1 ? S1
• G2 ? S2
• Gn ??Sn
• fi
• When no guard is true, skip (do nothing). When
  multiple guards are true, the choice of the
  action to be executed is completely arbitrary.

12
Syntax semantics

• Repetitive construct
• do G1 ??S1
• G2 ? S2
• .
• Gn ? Sn
• od
• Keep executing the actions until all guards are
  false and the program terminates. When multiple
  guards are true, the choice of the action is
  arbitrary.

13
Example graph coloring
There are four processes. The system has to
reach a configuration in which no two
neighboring processes have the same color.
1
0
0
program for process i do ?j ? neighbor(i) c(j)
c(i) ? c(i) 1-c(i) od
1
Will the above computation terminate?
14
Consider another example

• program uncertain
• define x integer
• initially x 0
• do x lt 4 ?? x x 1
• x 3 ? x 0
• od
• Question. Will the program terminate?
• (Our goal here is to understand fairness)

15
The adversary

• A distributed computation can be viewed as a
  game between the system and an adversary. The
  adversary may come up with feasible schedules to
  challenge the system (and cause bad things). A
  correct algorithm must be able to prevent those
  bad things from happening.

16
Non-determinism

• (Program for a token server - it has a single
  token
• repeat
• if (req1 ? token) then give the token to
  client1
• else if (req2 ? token) then give the token to
  client2
• else if (req3 ? token) then give the token to
  client3
• forever
• Now, assume that all three requests are sent
  simultaneously.
• Client 2 or 3 may never get the token! The
  outcome could
• have been different if the server makes a
• non-deterministic choice

Token server
3
1
2
17
Examples of non-determinism
If there are multiple processes ready to execute
actions, The who will execute the action first
is nondeterministic Message propagation delays
are arbitrary and the order of message reception
is non-deterministic
Determinism caters to a specific order and is a
special case of non-determinism.
18
Atomicity (or granularity)

• Atomic all or nothing
• Atomic actions indivisible actions
• do red message ? x 0 red action
• blue message ? x7 blue action
• od
• Regardless of how nondeterminism is
• handled, we would expect that the value of
• x will be an arbitrary sequence of 0's and 7's.
• Right or wrong?

x
19
Atomicity (continued)

• do red message ? x 0 red action
• blue message ? x7 blue action
• od
• Let x be a 3-bit integer x2 x1 x0, so
• x7 means (x21, x1 1, x21), and
• x0 means (x20, x1 0, x20)
• If the assignment is not atomic, then many
• interleavings are possible, leading to
• any possible value of x between 0 and 7

x
So, the answer depends on the atomicity of the
assignment
20
Atomicity (continued)

• Does hardware guarantee any form of atomicity?
  Yes!
• Transactions are atomic by definition (in spite
  of process failures). Also, critical section
  codes are atomic.
• We will assume that G ? A is an atomic
  operation. Does it make a difference if it is
  not so?

if x ? y ? y x fi
x
y
if x ? y ? y x fi
21
Atomicity (continued)

• Program for P
• define b boolean
• initially b true
• do b ? send msg m to Q
• empty(R,P)? receive msg
• b false
• od
• Suppose it takes 15 seconds to
• send the message. After 5 seconds,
• P receives a message from R. Will it
• stop sending the remainder of the
• message? NO.

P
Q
R
b
22
Fairness

• Defines the choices or restrictions on the
  scheduling of actions. No such restriction
  implies an unfair scheduler. For fair schedulers,
  the following types of fairness have received
  attention
• Unconditional fairness
• Weak fairness
• Strong fairness

Scheduler / demon / adversary
23
Fairness

• Program test
• define x integer
• initial value unknown
• do true ? x 0
• x 0 ? x 1
• x 1 ? x 2
• od

• An unfair scheduler may never schedule the second
  (or the third actions). So, x may always be equal
  to zero.
• An unconditionally fair scheduler will eventually
  give every statement a chance to execute without
  checking their eligibility. (Example process
  scheduler in a multiprogrammed OS.)

24
Weak fairness

• A scheduler is weakly fair, when it eventually
  executes every guarded action whose guard becomes
  true, and remains true thereafter
• A weakly fair scheduler will eventually execute
  the second action, but may never execute the
  third action. Why?

• Program test
• define x integer
• initial value unknown
• do true ? x 0
• x 0 ? x 1
• x 1 ? x 2
• od

25
Strong fairness

• A scheduler is strongly fair, when it eventually
  executes every guarded action whose guard is true
  infinitely often.
• The third statement will be executed under a
  strongly fair scheduler. Why?

• Program test
• define x integer
• initial value unknown
• do true ? x 0
• x 0 ? x 1
• x 1 ? x 2
• od

Study more examples to reinforce these concepts