MINLP

1
MINLP

• 6/17/05

2
References

• Engineering Optimization Methods and
  Applications. Reklaitis, Ravidran and
  Ragsdell.1983. Wiley Interscience. Chapter 11.
  (Welded Beam Lab Problem)
• Most 1.1 Manual. Technical Report No. AODL-96-01.
  Professor C.H. Tseng, Applied Optimal Design
  Laboratory (Word doc is better than Pdf with
  conversion errors.)
• Mixed Integer Discrete Continuous Optimization by
  Differential Evolution. (Coiled Spring Lab
  Problem)
• Applied Optimization with Matlab Programming.
  Ventataraman. 2002. Wiley Interscience.

3
Most Problem Formulation
4
Overview

• Real world is not always continuous.
• Cannot have 10.3 people.
• Cannot manufacturer all tolerances 1.324566
• Some parts have standard size for economy.
• Doors, windows, computer screens.
• Discrete variables kills our steepest ascent
• No partial derivatives (first or second)
• KKT cannot guarantee optimum

5
Plan

• Two approaches
• Solve a series of continuous problems to get
  discrete solution
• Standard Approach
• Enumeration Approach
• Branch and Bound Approach
• Solve discrete problem respecting design variable
  types and constraints
• Genetic algorithm (NSGA 2)
• Simulated Annealing (ASA)
• Both approaches require a lot more function
  evaluations
• Lab using branch and bound

6
Terminology

• Enumeration
• Branch and bound
• Incumbent
• Relaxation
• Candidate

7
Standard Approach

• Step 1 A Continuous relaxation and generate
  continuous solution.
• Step 2 Identify set of discrete solutions for
  further exploration.
• Step 3 For each set of discrete variable
  combination, a continuous optimization is
  performed.
• Step 4 A simple comparison of points from Step 3
  are made to identify optimal solution to discrete
  problem.

8
Sample MINLP Problem
2.24
9
Continuous Relaxation
10
Continuous Problem Formulation

• Could Solve with any iSIGHT Numerical
  optimization technique.
• Will use Most to get experience with it.
• Most implements SQP algorithm similar to NLPQL
  and Donlp2
• Most can be used for pure continuous problems.
• Will later use Most in Branch and Bound mode.

11
Most Continuous Problem Formulation
12
Most Problem Formulation
13
Most Optimizer Selection
14
Most Basic Parameters
15
Most Advanced Parameters
16
iSIGHT Output With Solution
17
Most Output Print Level 1
Lagrangian Multipliers
18
Most Output Level 2
Gradient Information
19
Identify Discrete Area of Interest
Optimal design x12.0, x22.0, x32.0
f0
20
Evaluate at 4 Points with DOE
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Solution Monitor
25
Function Evaluated at 4 Points
26
Better To Optimize At Each Point
Add a Parent Task with a DOE from Data
File Modify Child Task to only vary X1
27
Only Vary X1 for each Point
28
Results from DOE
29
Function Optimized from 4 Starting Points with 1
Design Variable
30
Observations

• More work than continuous optimization.
• No math guarantees of solution
• Two formal methods used for MINLP
• Enumeration
• Branch and bound

31
Exhaustive Enumeration
32
Joel Carpenter Club
Joel Carpenter Club has initiated fund raising
activities by selling pizza during the day. Over
the year, Joel has collected data to identify
a mathematical model that will reduce costs and
lead to higher profits. Only two types of pizza
will be sold pepperoni (x1) and cheese
(x2).The local dominos will deliver a maximum of
26 pizzas.
33
Solution Game Plan

• Solve using Relaxed Continuous Formulation for
  lower bound.
• Take all discrete value combinations and evaluate
  for lowest objective.

34
(No Transcript)
35
Most Continuous Evaluation
36
Most Continuous Evaluation
37
DOE Full Factorial For All Combinations
0 26 Pepperoni Pizzas 0 - 26 Cheese Pizzas
38
DOE Design Matrix
39
DOE Enumerative Solution
40
Branch and Bound

• A refined enumeration. Most of nonpromising
  integer points are discarded without testing.
• Integer problem is not directly solved. Solve
  series of constrained continuous problems
• Solve continuous problem by relaxing integer
  restrictions.
• If solution is a integer solution then it is
  optimum.
• If solution is not a integer solution
• Solution is a lower bound on best achievable.
• Find a non integer variable xi such that integer
  xi lt xi lt xi1
• Form two subproblems one with xi lt xiThe
  other with xi gt xi1

41
Branch and Bound

• Branch eliminates portion of continuous space
  that is not feasible for integer problem.
• Solve each of subproblems as a continuous
  problem.
• The solving of a continuous problem is a node.
• From each node may come two branches

42
Branch and Bound

• Process of branching and solving continues until
  a feasible integer solution is found.
• The objective value becomes upper bound on the
  minimum value of objective function. It is made
  the incumbent solution.
• All previous continuous solutions whose objective
  value gt incumbent are eliminated.
• All subsequent solutions whose objective value gt
  incumbent are eliminated.
• Eliminated nodes are fathomed since not possible
  to find a better solution.
• Depth first search or Breadth first search.

43
Types of Fathomed Nodes

• If continuous solution is an integer solution.
• The problem does not have a continuous feasible
  solution
• The optimal value of the continuous problem is
  larger than the incumbent solution.
• The integer solution objective is gt the current
  incumbent solution

44
Most Features

• Implements Branch and Bound
• Supports discrete, integer and continuous
  parameters.
• Continually adds constraints and solves
  continuous problems.
• Your code must support all continuous design
  variables. The integer and discrete is enforced
  outside the code.

45
Most Manual

• Very little diagnostic help or information.
• Describes problem formulation
• Describes tuning parameters.
• Recommends normalization of constraints

46
Sample MINLP Problem
47
Most Formulation
Most supports Real, Discrete and Integer. All
three can be in same formulation.
48
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(2,2,1.73)
(2,2,2.24)
49
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)
50
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(2,2,2.24)
(1.8,1.7,1.73) .04 F
X2gt2.5
X2lt1.5
(1.8,1.5,1.73)
(1.8,2.5,1.73)
51
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(2,2,2.24)
(1.8,1.7,1.73) .04 F
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
52
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.7,1.5,75)
(1.7,1.5,1.73)
53
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73)
54
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73)
X2gt1.5
X2lt.5
(1.2,.5.,75)
(1.2,1.5,.75)
55
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73)
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.2,1.5,.75)
56
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73)
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75) 1.35F
57
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73)
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75) 1.35F
58
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(1.8,2.5,1.73)
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) -.178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75) 1.35F
59
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2,2,2.24)
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) -.178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75) 1.35F
60
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24) .34F
X2gt2.5
X2lt1.5
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) -.178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75) 1.35F
61
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)-.034F
X2gt2.5
X2lt1.5
X2gt2.5
X2lt1.5
(2.1,1.5,2.24)
(2.1,2.5,2.24)
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) .178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75)1.35F
62
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)-.034F
X2gt2.5
X2lt1.5
X2gt2.5
X2lt1.5
(1.9,1.5,2.24).8F
(2.1,2.5,2.24)
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) .178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75)1.35F
63
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)-.034F
X2gt2.5
X2lt1.5
X2gt2.5
X2lt1.5
1.9,1.5,2.24).8F
(2.3,2.5,2.40)-.15F
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt2.24
X3gt2.78
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) .178F
(2.3,2.5,2.24)
(2.3,2.5,2.78)
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75)1.35F
64
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)-.034F
X2gt2.5
X2lt1.5
X2gt2.5
X2lt1.5
1.9,1.5,2.24).8F
(2.3,2.5,2.4)-.15F
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt2.24
X3gt2.78
X3lt.75
X3gt1.73
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) .178F
(2.2,2.5,2.24)-.19F
(2.3,2.5,2.78)
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75)1.35F
65
Solution Walk Through
(2,2,2) 0 F
X3lt1.73
X3gt2.24
(1.8,1.7,1.73) .04 F
(2.1,2.1,2.24)-.034F
X2gt2.5
X2lt1.5
X2gt2.5
X2lt1.5
1.9,1.5,2.24).8F
(2.3,2.5,2.4)-.15F
(1.7,1.5,1.6)-.15F
(2.0,2.5,1.73) .89F
X3lt2.24
X3gt2.78
X3lt.75
X3gt1.73
(2.2,2.5,2.24)-.19F
(2.4,2.6,2.78) .36F
(1.2,1,0.75)-.937F
(1.7,1.5,1.73) .178F
X2gt1.5
X2lt.5
(1.0,.5.,75)1.35F
(1.4,1.5,.75)1.35F
66
Most Solution
67
Joel Carpenter Club
Joel Carpenter Club has initiated fund raising
activities by selling pizza During the day. Over
the year, Joel has collected data to identify
a mathematical model that will reduce costs and
lead to higher profits. Only two types of pizza
will be sold pepperoni (x1) and cheese
(x2).The local dominos will deliver a maximum of
26 pizzas.
68
Solution Walkthrough
(10.86,9.65) 33.13
69
Solution Walkthrough
(12.86,9.65) 33.13
X1lt12
X1gt13
(13,9.65)
(12, 9.65)
70
Solution Walkthrough
(12.86,9.65) 33.13
X1lt12
X1gt13
(13,9.65)
(12, 10) 34F
71
Solution Walkthrough
(12.86,9.65) 33.13
X1lt12
X1gt13
(13,9.6)- 33.16
(12, 10) 34F
X2gt10
X2lt9
(13,10)
(13,9)
72
Solution Walkthrough
(12.86,9.65) 33.13
X1lt12
X1gt13
(13,9.6)- 33.16
(12, 10) 34F
X2gt10
X2lt9
(13,10)
(14.5,9) - 36.25F
73
Solution Walkthrough
(12.86,9.65) 33.13
X1lt12
X1gt13
(13,9.6)- 33.16
(12, 10) 34F
X2gt10
X2lt9
(13,10) - 29
(14.5,9) - 36.25F
74
Dave Most Solution Validation
75
Branch and Bound Most Solution
76
Number of Evaluations
Continuous Enumeration Branch and Bound
44 730 108
77
Guidelines

• Keep the number of integer and discrete variables
  as small as possible.
• Provide a good (tight) lower and upper bound on
  the integer variables and a tight set of discrete
  values for discrete variables.
• Keep in mind that the evaluation function must be
  smooth and continuous.

78
Lab

• Lab is in the file LabMINLP.doc

79
Questions