Column Generation

1
Column Generation

• ENGG 6070
• Instructor Shawki M Areibi
• Students Limin Ma, Hao Qin

2
What is Column Generation ?
Column generation is a technique that is widely
used to solve large scale integer programs
3
Prototype of Linear Programs of Large-scale
Systems
z Max z c1 x1 c2 x2 cn xn ,
Subject to ai1x1 ai2 x2 ainxn
bi ( i 1, 2, , m) (1)
xn ? 0 ( j 1, 2, ,
n) where n is very large , may be more
than millions.
4
Example Cutting stock problem A paper
company must produce various sizes of its paper
products from its raw rolls to meet customer
demand.
5
Example Cutting stock problem If customers
need bi rolls of sizes Li , how the company meet
the demand using the fewest number of rolls ?
6
Example Cutting stock problem A key obstacle
In general, there are exponentially many
patterns xn to cut raw rolls
x1
x2
xn
7
Structure of Large-Scale Problems
8
The Decomposition Algorithm
Decompose the large scale system in many
subsystem for computation. For Independent
subsystem , it can be written as Minimize
subject to
bi (i 1, 2, ,t)
bi (i t1, t2, ,u)
bi (i
u1, u2, ,m)
xj gt0 (j 1, 2, , n)
9

• Why Decomposition ?
• Provide significant computational savings,
  since the computations for linear programs
  are quite sensitive to m, the number of
  constraints, in practice proportionally to m3.
  So it can save 1/m2 times.
• Each of the independent subproblem can be
  treated separately. Data can be gathered,
  analyzed and stored separately.

10
Why Column generation ?

• Because of the large number of variables and
  columns, directly solution by the simplex method
  may be inappropriate. Simply generating all the
  coefficients data aij usually prohibits this
  approach.
• Column generation extends the technique
  introduced in the decomposition algorithm, of
  using the simplex method (SM), but generating the
  coefficient data only as needed. Consequently,
  new data could be generated by solving this
  linear program with an appropriate objective
  function.

11
The Approach of Column Generation
As in decomposition, assume a priori that certain
variables, say xJ1, xJ2, , xn are nonbasic and
restrict their values to zero. The problem
becomes zJ Max c1x1 c2x2
cJxJ , (1)
subject to ai1x1 ai2 x2
aiJxJ bi ( i 1, 2, , m) (2)
xj 0 (
j 1, 2, , J) The original problem (1) is
called a master problem (MP), whereas problem (2)
is called the restricted master problem (RMP).
12
Suppose that the RMP has been solved by SM
and that ?J1, ?J2, , ?Jm are the optimal shadow
prices. It is optimal if the simplex optimality
condition holds, that is, if
the original problem has been solved without
specifying all of the aij data or solving the
full MP.
If
then the simplex method, when applied to
MP, would introduce variable xs into the basis.
The new RMP can be solved by the simplex
method and the entire procedure can be repeated.
13
The Significance of Column Generation

• This procedure avoids solving the full MP
  instead it alternately solves a RMP . It is
  referred to as a subprobelm.
• The method is specified in flow-chart in
  Figure 2. Its efficiency is predicted upon
• Obtaining an optimal solution before many
  columns have been added to RMP.
• Being able to solve the subproblem effectively.

14
(No Transcript)
15
Remark
The column generation method might generate
a non-integer solution. For applications (with
large demands), rounding to nearest integers
produces satisfactory answers. If an exact
optimal is needed, one can apply the BB
technique.
16
Two Major Types of Column Generation
Type I Airline crew scheduling (ACS) Type II
The cutting stock (CS) problem
17
Two Types of Column Generation

• Type I column generation
• Uses an auxiliary model (AM) to
  identify an attractive set of columns,
  defining a RMP that optimizes over these
  explicitly defined columns.
• The RMP accepts these columns and does
  not interact further with the AM.

18
Type I Airline crew scheduling (ACS)

• The objective is to minimize total
  crew cost. Airline schedules give the
  type of aircraft and its departure and arrival
  times for each flight segment, a flight from one
  city to another.
• A crew works a rotation, which services
  a series of segments and may last several days.
  Each rotation must satisfy a number of
  constraints imposed by official agencies, union
  rules and airline restrictions.
• The cost of rotation includes lodging and
  other travel expenses, flight time and deadheading

19
The algorithm of ACS
Defining aij 1 if rotation j ( j?J ) serves
segment i (i ? I ), 0 otherwise and employing
decision variables x j 1 if rotation j is
prescribed, 0 otherwise ( j ? J ), Problem
ACS may be formulated as Min
ZACS (4.1) s.t.
i ? I
(4.2)
x j ?0, 1 j
? J (4.3)
20
Challenge there may be millions of
feasible rotations for a commercial airline
so that even generating them may require a
prohibitive amount of run time. Solution 1.
Invokes an AM to generate a set of feasible
rotations. Each generated column aj ( j ?
J ) consists of elements aij 1 (i ? Ij )
for each segment i served by
rotation j , and aij 0 (i ? Ij ). 2.
Solves the set partitioning problem with the
columns defined explicitly by the AM.
21

• Experience Result
• It is typically not possible to assure that
  a restricted set of rotations includes a subset
  that comprises an optimal solution, so
  the overall approach is a heuristic.
• Nevertheless, experience has shown that it
  is possible to prescribe good solutions if
  the set partitioning problem can be solved
  effectively for a large number of generated
  rotations.

22
Type II column generation
Type II column generation has two classic
applications i) The cutting stock (CS)
problem ii) The vehicle routing problem
23
The Cutting Stock (CS) Problem.
A company stocks rolls of sheet metal in
standard lengths Lm (m ? M). Assume that the
company has an unlimited number of each
standard length and that will cut these
standard lengths into shorter lengths
to satisfy customers orders. Customers
require bi pieces of length i (i ? I ), where
I is the set of cut lengths ordered. Note
that order I can be satisfied as long as
li ? Lm for some m. Standard length Lm costs
cm(m ? M) and the total cost of filling
orders includes the costs of all standard lengths
cut to fill the orders. The cutting stock
problem is to fill customer orders at minimum
total cost.
24
The Cutting Stock (CS) Problem.
Let aij denote the number of pieces of length i
(i ? I ) in cutting pattern j and c j , the
cost of the standard roll used in pattern j .
Thus, the index j implies m as well as I j ,
the subset of cut lengths included in the
cutting pattern. To be feasible, cutting
pattern j must observe the limitation imposed
by the standard length Lm(m ? M)
25
The algorithm
Decision variables x j number of cutting
patterns of type j prescribed ( j ? J ) to
present Problem CS as formulated
Min
(5.1) s.t.
i?I (5.2)
x j ? 0, Integer j ? J.
(5.3)
26
The algorithm
Linear relaxation for integer constraints The
requirement of the cutting stock problem is known
to be tight. This heuristic is not guaranteed to
prescribe the optimal integer solution. To
guarantee such a solution, the linear relaxation
may be used to obtain a lower bound at each
node in a branch-and-bound (BB) search tree.
27
The algorithm
The primary difficulty is that the number of
cutting patterns, J , may be extremely large,
so that solving the LP problem with all J
columns could take a prohibitive amount of
time.
28
The algorithm
Instead of listing all J columns explicitly
and pricing each out to identify the column
that should enter solution at each simplex
iteration, the RMP generates a limited set of
columns and must be optimized over these
columns to obtain dual variables that are used
to define an dual objective function .
Defining wi dual variable associated with
constraint i of type (5) (i . I ) and cm cost of
standard length Lm(m . M) and employing decision
variables yi number of length i in a cutting
pattern j .
29
The dual solution
It can be formulated as Min

(6.1) s.t.

(6.2)
yi ? 0, Integer i ? I
(6.3) If ZPOP ?0 (m ? M), the current RMP
solution is optimal. Otherwise, if ZPOP ?0, the
dual solution generates an improving column,
which is assigned a new column number j
( j ?j 1), an objective function
coefficient c j cm and coefficient in each row
i (i ? I )aij yi (i ? I ).
30
Comparison between Type I and Type II

• The primary limitation of the Type I method is
  that the AM generates columns without interacting
  with the RMP. It normally has to generating a
  large set of columns with the hope of including
  the subset that comprises the optimal solution
• Type II methods, which do involve interaction
  between the RMP and SP(s), allow columns to be
  identified and introduced as needed

31
Other Problem Can be Solved by Column Generation

• Generalized assignment problem
• The vehicle routing problem
• Single machine scheduling
• Assembly system design with tool changes.
• Problems that can be solved by the
  Dantzig-Wolfe
• decomposition.

32
Technical considerations
33
i) Formulation issues a) RMP formulation The
goal is to formulate a decomposition that will
allow the solution to the RMP to serve as a
tight bound on the value of the optimal integer
solution to facilitate the BB search
(1) They may provide tighter bounds than
more compact formulations can (2) They
may avoid symmetry that requires BB to
expend considerable run time to enumerate a
large number of equivalent solutions (3)
They are amenable to decomposition, perhaps
separating into a number of independent SPs that
may be solved easily and (4) They may provide
the only formulation possible.
34
b). SP formulation SP(s) also play a
crucial role through their structure,
symmetry, complexity and whether of not they
exhibit the Integrality Property. The integer
variables in an IP typically become decision
variables in one or more SPs. Formulations that
have the block diagonal structure are attractive
because they result in small, independent SPs
that are typically more effectively solved.
35
ii) Algorithm design issues a) Branching
strategies Branching must not destroy SP
structure and should result in siblings that
represent balanced sets of solutions.
Balancing is important because it can be expected
to result in a tighter bound at each sibling
node, facilitating solution. b) Subproblem
solution strategies SP solution strategies select
the column that will enter the RMP and a method
to solve each SP. One strategy would be to solve
all SPs and select the best improving column
to enter the RMP.
36
iii) Implementation issues a) Pool
management 1. Determine an initial BFS for the
RMP 2. Initialize the column pool to be empty 3.
Optimize the RMP over columns that are known
explicitly 4. Delete nonbasic generated columns
in the RMP that have positive reduced costs 5.
Select a subset of remaining pool columns with
negative reduced costs, add them to RMP and go to
step (3)
37
6. Eliminate any remaining columns from the
pool 7. Use a heuristic to solve the SP(s), If
improving columns are generated, add them to the
pool and go to step (5) 8. If no columns are
generated in step (7), use an optimizing
algorithm to solve the SP(s) If improving
columns are generated, add them to the pool
and go to step (5) 9. Stop
38

• b) Degeneracy
• Degeneracy in the RMP may be problematic
  for several reasons.
• First, a number of pivots ( and
  corresponding SP solutions to generate
  entering columns) may be required before the
  search can progress to a better, adjacent
  extreme point.
• Second, special steps may be required to avoid
  a large number of pivots (called stalling) and
  the worst-case, cycling.
• Thirdly, if the primal RMP has a degenerate
  optimum, the associated dual has
  alternative optimal solutions and the best
  set of optimal dual variables must be identified
  so that they can be passed into the SPs
  and the primal basic feasible solution that
  satisfies the LP optimality criterion may be
  identified.

39
Conclusions and recommendations for future
research

• Several issues must be resolved by future
  research to enhance existing capabilities.
• First, as in any LP problem, degeneracy causes
  inefficiency by requiring a number of pivot
  operations to advance beyond a degenerate
  extreme point and may lead to stalling and
  cycling.
• Second, SP solution strategies may require a
  large number of SPs to be solved for each
  column that is entered into the solution of the
  RMP.

40
Thank you !