ESI 6912: Dynamic Programming

1
ESI 6912Dynamic Programming

• Resource Allocation problems - 3

2
Resource allocation with multiple resources

• Recall the resource allocation problem with 2
  resource constraints.

3
Multiple Resources
4
Using Lagrange relaxation

• Relax the 2nd capacity constraint and penalize
  the violation in the objective function (with
  penalty parameter ??0)

5
Using Lagrange relaxation

• Note that we can rewrite this problem as

6
Using Lagrange relaxation

• So, for fixed ?, we obtain a single resource
  allocation problem with profit functions
• So we can solve this problem using e.g. recursive
  fixing

7
Property 1

• The optimal allocation for a given ? (say x ?) is
  the best solution to the original problem among
  all allocations that use no more of the 2nd
  resource.

8
Proof of Property 1

• Since x ? is the optimal solution to the relaxed
  problem, we have
• This implies
• Therefore, if then

9
Property 1

• This property implies that if, for some ?, we
  have d (x ?)L, then x ? is the optimal solution
  to the original problem.

10
Property 2

• The amount used of the 2nd resource is
  non-increasing in ?.

11
Proof of Property 2

• Let 0??1??2. Then
• and
• Thus
• and therefore

12
Property 2

• This property implies that, if d (x
  0)?L, then this solution is optimal and the 2nd
  resource constraint is redundant.
• Otherwise, we can use property 2 to search for a
  value of ? for which d (x ?)L.

13
Algorithm bisection

• Find two values of ? (0??1??2) whose
  corresponding consumption of the 2nd resource
  bracket the value L
• Solve the relaxed problem with
• If d (x ?)L, STOP.
• If d (x ?)gtL, set ?1? otherwise set ?2?.
  REPEAT.

14
Algorithm graphical
p(x)
5
p(x)-?d(x)c
p(x)c
4
3
2
1
d(x)
1
2
3
4
5
6
7
15
Algorithm bisection

• The bisection algorithm works if the profit as
  function of the amount of resource 2 exhibits
  strictly decreasing marginal return.
• In this case, there is a continuum of values for
  ? that correspond with each value for the
  resource consumption.
• Bisection will find one of the values
  corresponding to a consumption of L, and
  therefore the optimal solution, in finite time.

16
Bisection Potential problems

• There are no solutions for which exactly L units
  of resource 2 are consumed

p(x)
5
p(x)-?d(x)c
p(x)c
4
3
2
1
d(x)
1
2
3
4
5
6
7
17
Bisection Potential problems

• The marginal returns are decreasing, but not
  strictly at L

p(x)
p(x)-?d(x)c
p(x)-?d(x)c
5
p(x)-?d(x)c
4
3
2
1
d(x)
1
2
3
4
5
6
7
18
Bisection Potential problems

• The marginal returns are not everywhere decreasing

p(x)-?d(x)c
p(x)
p(x)-?d(x)c
5
4
3
2
1
d(x)
1
2
3
4
5
6
7
19
Another resource allocation problem (1)

• Consider the problem of allocating two types of
  resources to N activities.
• There are K units of resource 1 available, and L
  units of resource 2.
• The profit associated with allocating xn units of
  resource type 1 and yn units of resource type 2
  to activity n is pn(xn,yn).

20
NLP formulation

• Formulate this problem as a dynamic programming
  problem.

21
Dynamic Programming formulation

• State
• (n,x,y) (first activity to be considered, total
  resources remaining)
• Initial states (N1,x,y)
• Ending state (1,K,L)
• Decision
• Amount of resources to allocate to activity n

22
Another resource allocation problem (1)

• Optimal value function
• f(n,x,y) the maximum profit obtainable from
  activities n,,N using x units of resource 1 and
  y units of resource 2
• We wish to find f(1,K,L)
• Boundary conditions
• f(N1,x,y) 0, x0,,K, y0,,L

23
Another resource allocation problem (1)

• Recurrence relation
• for x0,,K y0,,L n1,,N
• Running time
• The network has O(NKL) nodes and O(N(KL)2) arcs
• The costs of each arc can be determined in
  constant time
• The running time is O(N(KL)2)

24
Another resource allocation problem (2)

• Reconsider the previous problem.
• However, now assume that we have Z dollars
  available, that we can spend on resources.
• One unit of resource i costs ri (i 1,2).
• How many units of each resource should we buy?

25
NLP formulation

• Solve this problem using dynamic programming.

26
Another resource allocation problem (2)

• Approach 1
• The DP formulation of the problem with fixed
  values of K and L in fact solves all problems
  with smaller resource capacities
• Solve the original problem with KZ/r1 and
  LZ/r2.
• Choose the combination that yields the highest
  profit
• Running time O(NZ4/(r1r2))

27
Another resource allocation problem (2)

• Approach 2
• First, find the optimal profit for a given budget
  assigned to activity n
• Then solve a dynamic programming problem find the
  optimal amount of money to spend on resources for
  each activity

28
Dynamic Programming formulation

• State
• (n,z) (first activity to be considered, total
  budget available)
• Initial states (N1,z)
• Ending state (1,Z)
• Decision
• Budget to allocate to resources for activity n

29
Another resource allocation problem (2)

• Optimal value function
• f(n,z) the maximum profit obtainable from
  activities n,,N with a budget of z
• We wish to find f(1,Z)
• Boundary conditions
• f(N1,z) 0, z0,,Z

30
Another resource allocation problem (2)

• Recurrence relation
• for z0,,Z n1,,N
• Running time
• The network has O(NZ) nodes and O(NZ2) arcs
• The costs of all arcs can be determined in O(NZ2)
• The running time is O(NZ2)