Algorithms Lecture 10

1
AlgorithmsLecture 10

• Lecturer Moni Naor

2
Linear Programming in Small Dimension

• Canonical form of linear programming
• Maximize c1 x1 c2 x2 cd xd
• Subject to a1,1 x1 a1,2 x2 a1,d xd
  b1
• a2,1 x1 a2,2 x2
  a2,d xd b1
• ...
• an,1 x1 an,2 x2 an,d xd
  bn
• n number of constraints and
• d number of variables or dimension

3
Linear Programming in Two Dimensions
Feasible region
Optimal vertex
4
What is special in low dimension

• Only d constraints determine the solution
• The optimal value on those d constraints
  determine the global one
• Problem is reduced to finding those constraints
  that matter
• We know that equality hold in those constraints
• Generic algorithms
• Fourier-Motzkin (n/2)2d
• Worst case of Simplex number of bfs/vertices (nd
  )

5
Key Observation

• If we know that an inequality constraints is
  defining
• we can reduce the number variables
• Projection
• Substitution

4X1 - 6x 2 4
Feasible region
Optimal vertex
6
Incremental Algorithm B

• Input A set of n constraints H on d variables
  
• Output the set of defining constraints
• 0. If Hd output B(H)H
• Pick a random constraint If h 2 H
• recursively find B(H \ h)
• 2. If B(H \ h) does not violate h output B(H \
  h)
• else project all the constraints onto h
  and recursively solve this (n-1,d-1) lp program

7
Correctness, Termination and Analysis

• Correctness by induction
• Termination if the non defining constraints
  chosen
• No need to rerun
• Analysis probability that h is one the defining
  constraints is d/n

• 0. If Hd output B(H)H
• Pick a random constraint If h 2 H
• recursively find B(H \ h)
• 2. If B(H \ h) does not violate h output B(H \
  h)
• else project all the constraints onto h
  and recursively solve this (n-1,d-1) lp
  program

8
Analysis

• Analysis probability that h is one the defining
  constraints is d/n
• T(d,n)
• d/n T(d-1,n-1) T(d,n-1)
• by induction
• d/n (d-1)! (n-1) d! (n-1)
• (n-1)d!(1/n 1)
• nd!

• 0. If Hd output B(H)H
• Pick a random constraint If h 2 H
• recursively find B(H \ h)
• 2. If B(H \ h) does not violate h output B(H \
  h)
• else project all the constraints onto h
  and recursively solve this (n-1,d-1) lp
  program

9
How to improve

• The algorithm is wasteful
• When the solution does not fit the new a new is
  computed from scratch

• 0. If Hd output B(H)H
• Pick a random constraint If h 2 H
• recursively find B(H \ h)
• 2. If B(H \ h) does not violate h output B(H \
  h)
• else project all the constraints onto h
  and recursively solve this (n-1,d-1) lp
  program

10
Random Sampling idea

• Build the basis by adding the constraints in a
  manner related to history
• Input A set of n constraints H on d variables
  
• Output the set of defining constraints
• 0. If H c d2 return simplex on H
• S Ã ?
• Repeat
• Pick random R ½ H of size r
• Solve recursively on S R solution is u
• V set of constraints in H violated by u
• If V t, then S Ã S V
• Until V ?

11
Correctness, Termination and Analysis

• Claim Each time we augment S (S Ã S V), we add
  to S a new constraint from the real basis B of
  H
• If u did not violate any constraint in B it would
  be optimal
• So V must contain an element from B which was not
  in S before
• Since Bd, we can augment S at most d times
• Therefore the number of constraints in the
  recursive call is RS r dt
• Important factor for analysis what is the
  probability of successful augmentation

12
Sampling Lemma

• For any H and S ½ H The expected (over R) number
  of constraints V that violate u (optimum on S
  R) is at most nd/r
• Proof
• Let X(R,h) be 1 iff h violated h(S R)
• Need to bound
• ER ?h X(R,h) 1 / R ?Rr ?h X(R,h)
• instead consider all subsets Q R h of size
  r1
• 1 / R ?Qr1 ?h2 Q X(Q\h,h)
• (Q/R) (r1) ProbQ,h 2 Q X(Q\h,h)
• n d / (r1)

13
Analysis

• Setting t 2 nd /r implies (from Markovs
  inequality)
• number of recursive call until a successful (V
  t) augmentation is constant
• Number of constraints in recursive call bounded
  by rO(d2n/r)
• Setting rd n1/2 means that this is O(r)
• Total expected running time
• T(n) 2 d T(d n1/2 ) O(d2 n)
• Result
• O( (log n)log d (Simplex time) ) O(d2 n)
• Can be improved to O(dd d2n)
• Can be improved to O(dd1/2 d2n) using Kalai,
  Matousek-Sharir-Welzl

14
References

• Motwani and Raghavan, Randomized Algorithms
  Chapter 9.10
• Michael Goldwasser, A Survey of Linear
  Programming in Randomized Subexponential Time
  http//euler.slu.edu/goldwasser/publications/SIGA
  CT1995_Abstract.html
• Pioter Indyks course at MIT, Geometric
  Computation
• http//theory.lcs.mit.edu/indyk/6.838/
• Applet http//web.mit.edu/ardonite/6.838/linear-p
  rogramming.htm