Nonlinear Programming

1
Nonlinear Programming

• Quadratic programs
• Unconstrained nonlinear programs
• Constrained nonlinear programs

2
Quadratic Programs

• Characteristics
• Quadratic objective function
• Linear equality inequality constraints
• Examples of quadratic objective functions
• Least-squares parameter estimation problems with
  linear model equations
• Multivariable control problems with linear model
  equations
• Quadratic program (QP)
• Very efficient solution methods available

3
Unconstrained Single Variable Optimization

• Examples of decision variables
• Optimal reflux rate in a distillation column
• Optimal air/fuel ratio in a furnace
• Optimization problem
• Analytical solution
• Assume function is unimodal with only one
  extremum
• Easily solved numerically in Excel, Matlab, etc.

4
Unconstrained Multivariable Optimization

• Examples of decision variables
• Optimal reflux rate boilup rate in a
  distillation column
• Optimal air/fuel ratio temperature in a furnace
• Optimization problem
• Analytical solution
• Requires more sophisticated software tools
• Optimization toolbox in Matlab

5
Method of Steepest Descent

• Extrema
• Global minima
• Local minima
• Method of steepest descent

6
Constrained Optimization

• General nonlinear program
• n decision variables
• m equality constraints
• n-m degrees of freedom available for optimization
• Constraints
• Active constraint limits optimal solution
• Inactive constraint does not limit optimal
  solution
• Optimal solution often located at intersection of
  active constraints

7
Exothermic CSTR

• Objective function conversion
• Decision variables x
• Volumetric flow rate coolant temperature
• Reactor concentration temperature
• Subject to lower and upper bounds
• Steady-state model equations
• Constrained optimization problem (2 degrees of
  freedom)

8
Convexity

• Convex function
• A function is convex if the function lies below a
  line connecting any two points in the domain
• Convex set
• A set is convex if a line connecting any two
  points in the set is completely contained within
  the set
• The intersection of convex sets yields a convex
  set
• Convex optimization problem
• Comprised of a convex objective function convex
  constraints
• Devoid of local optima
• Much easier to solve than non-convex optimization
  problems
• Linear quadratic programs are always convex
• Nonlinear programs are usually not convex due to
  nonlinear equality constraints

9
Solution of Nonlinear Programs

• Successive quadratic programming (SQP) methods
• Approximate NLP as sequence of QPs
• Require first-order derivatives (Jacobian matrix)
  second-order derivatives (Hessian matrix)
• Codes NPSOL, SNOPT, IPOPT
• Generalized-reduced gradient methods
• Solve sequence of simplified NLPs
• Require first-order second-order derivatives
• Codes Excel, GRG2, CONOPT
• Derivative-free methods
• Direct search, genetic simulated annealing
  algorithms
• Allow determination of global optima but very
  slow
• Practical issues
• Best code is highly problem dependent
• Most codes provide no guarantee of global
  optimality
• Usually necessary to rerun code with multiple
  initial guesses

10
CONOPT NLP Solver

• Developed and marketed by ARKI Consulting and
  Development A/S in Denmark (www.conopt.com)
• Feasible path solver based on the Generalized
  Reduced Gradient (GRG) method
• Multi-method capability with built-in logic for
  dynamic selection of the most appropriate method
• Extensions for Sequential Linear Programming
  Sequential Quadratic Programming (SQP)
• Preferred for highly nonlinear optimization
  models
• Very reliable relatively efficient for a broad
  range of optimization models
• Successfully applied to many parameter estimation
  dynamic optimization problems by the Henson
  Research Group

11
IPOPT NLP Solver

• Developed by freely available from Carnegie
  Mellon University (http//projects.coin-or.org/Ipo
  pt)
• Interior-point (barrier method) algorithm based
  on a filter line search
• Solves a sequence of barrier problems for
  decreasing values of the barrier parameter m
• The barrier problem solution converges to a local
  solution of the original NLP as m0
• Very efficient solver that is well suited for
  highly sparse large-scale problems
• Often faster than CONOPT for sparse, large
  dynamic optimization problems
• Probably less reliable than CONOPT across a
  diverse class of problems

12
Optimization Model Development

• Model Development
• Express all constraints as algebraic equations
  inequalities
• Identify decision variables
• Specify an objective function to be minimized or
  maximized in terms of the appropriate decision
  variables
• Provide first- and second-order derivatives
• Advanced Model Programming Language (AMPL)
• Originally developed by Bell Labs now available
  from AMPL Optimization LLC (www.ampl.com)
• Interfaces available for 35 solvers including
  CONOPT IPOPT
• Automatically provides first- and second-order
  derivatives
• General Algebraic Modeling System (GAMS)
• Originally developed by the World Bank now
  available from the GAMS Development Corporation
  (www.gams.com)
• Similar to AMPL