Computational Intelligence Based Methodologies for Modeling and Optimization

1
Computational Intelligence Based Methodologies
for Modeling and Optimization

• Somnath Nandi
• Asst. Professor
• Dept. of Petroleum and Petrochemical Engineering
• MIT - Pune

2
Contents

• Modeling
• Artificial Neural Networks
• Optimization
• Genetic Algorithms
• Differential Evolution
• Case Study
• Conclusion

3
Modeling

• Engineers and scientists required to analyze the
  complex processes and develop mathematical models
  which simulate their steady-state and / or
  dynamic behavior.
• The objective is to construct, from theoretical
  and empirical knowledge of the process, a
  mathematical description.
• A mathematical model provides information on the
  process behavior, over important ranges of
  operating variables, in terms of equations, which
  reflects at least the major features of the
  underlying mechanisms.

4
Modeling (contd )

• Phenomenological Approach
• - Process behavior described in terms of the
  appropriate mass, momentum and energy balance
  equations together with the pertinent chemical
  engineering principles.
• - Mathematical formulation describing the
  physico-chemical phenomena underlying in the
  process is formulated followed by model fitting.
• - Regression techniques based on the least
  squares minimization

5
Modeling (contd )

• Advantages
• - It provides a valuable insight into the
  process behavior
• - It possesses extrapolation ability
• Disadvantages
• - Owing to the complex nature of many processes,
  the underlying physico-chemical phenomenon is
  seldom fully understood
• - Collection of the requisite phenomenological
  information is costly, time-consuming and tedious
• - Nonlinear behavior common for many processes
  leads to complex nonlinear models, which in most
  cases are not amenable to analytical solutions
  thus, computationally intensive numerical methods
  must be utilized for obtaining solutions

6
Modeling (contd )

• Empirical Approach
• - Process behavior is modeled using
  appropriately chosen empirical equations, for
  instance, polynomial expressions.
• - Model can be constructed solely from the
  process input-output data without explicitly
  invoking the process phenomenology.
• - An appropriate functional form that possibly
  fits the process data is selected in advance
  following which the unknown model parameters are
  estimated using a suitable function fitting
  procedure.

7
Modeling (cond )

• Artificial Intelligence based Approach
• - AI is science and engineering of making
  intelligent systems, especially intelligent
  computer programs
• - Related to task of using computers to
  understand the human intelligence
• - Intelligence can be broadly defined as
  computational part of our ability to efficiently
  achieve goals in the world.

8
AI based Modeling Approaches

• Artificial Neural Networks (ANN)
• Support Vector Regression (SVR)
• Genetic Programming (GP)
• Fuzzy Logic (FL)

9
Artificial Neural Networks

• Efforts to develop computer models of the
  information processing of human nervous system
  (Rumelhart et. al., 1986).
• Simplified mathematical models describing the
  biological nervous system and functioning.
• A highly interconnected system of simple
  processing elements can learn complex
  interrelationships between the independent and
  the dependent variables in a data set.

10
Artificial Neural Networks
11
Artificial Neural Networks
x1
y1
x2
Output
Input
yK
xN
12
Artificial Neural Networks

• The distinct advantages of the ANN formalism are
• Can be developed solely from process
  input-output data.
• MIMO relationships can be approximated
• Possesses good generalization ability
• Can tolerate noisy data or incomplete
  information
• Can be developed even using qualitative data.
• Use a generic nonlinear function for function
  approximation and thus there is no need to
  specify system-specific data fitting function as
  done in traditional regression.

13
Artificial Neural Networks

• Principal applications of ANNs are
• (i) nonlinear function approximation (i.e.,
  process modeling),
• (ii) pattern recognition and classification,
• (iii) data reduction and compression,
• (iv) signal processing,
• (v) noise reduction.

14
What is Optimization ?

• Optimization is use of specific methods to
  determine the most cost-effective and efficient
  solution to a problem or design for a process
• A wide variety of problems in the design,
  construction, operation, and analysis of
  industrial processes can be resolved by
  optimization
• The field of statistics treats various principles
  termed "maximum likelihood," "minimum loss," and
  "least squares," and business makes use of
  "maximum profit," "minimum cost," "maximum use of
  resources," "minimum effort," in its efforts to
  increase profits

15
What is Optimization ?

• A typical engineering problem can be posed as
  follows a process can be represented by some
  equations or perhaps solely by experimental data.
  You have a single performance criterion in mind
  such as minimum cost
• The goal of optimization is to find the values of
  the variables in the process that yield the best
  value of the performance criterion
• A trade-off usually exists between capital and
  operating costs. The described factors-process or
  model and the performance criterion-constitute
  the optimization "problem."

16
What is Optimization ?

• Optimization is minimization or maximization of
  an objective function (also called a performance
  index or goal function) that may be subject to
  certain constraints
• min f (x) Goal function
• subject to,
• g (x) 0 Equality constraints
• h (x) lt 0 Inequality constraints

17
Need for Optimization

• Typical problems in engineering process design or
  plant operation have many (possibly an infinite
  number) solutions
• Optimization is concerned with selecting the best
  among the entire set by efficient quantitative
  methods
• Computers and associated software make the
  necessary computations feasible and cost
  effective
• To obtain useful information using computers,
  however, requires
• (1) critical analysis of the process or
  design,
• (2) insight about what appropriate performance
  objectives are (what is to be
  accomplished),
• (3) use of past experience, sometimes called
  engineering judgment.

18
Applications of Optimization

• Determining the best sites for plant location
• Routing tankers for the distribution of crude and
  refined products
• Sizing and layout of a pipeline
• Designing equipment and an entire plant
• Scheduling maintenance and equipment replacement
• Operating equipment, such as tubular reactors,
  columns, and absorbers
• Evaluating plant data to construct a model of a
  process
• Minimizing inventory charges
• Allocating resources or services among several
  processes
• Planning and scheduling construction

19
1-Dimensional Search
20
2-Dimensional Search
21
2-Dimensional Search
22
2-Dimensional Search
23
Unimodal Optimization
24
Multi-modal Optimization
A function exhibiting different types of
stationary points. a-inflection point
(scalar equivalent to a saddle point)
b-global maximum c-local minimum
d-local maximum
25
Global Methods of Optimization
26
Performance of Classical Techniques
27
Multiobjective Optimization

• A MOO problem will have two or more objectives
  involving many decision variables and constraints
• Consider an MOO problem with two objectives
  f1(x) and f2(x), and several decision variables
  (x)
• Minimize f1(x) (1)
• Minimize f2(x) (2)
• With respect to x
• Subject to xL x xU (3)
• h (x) 0 (4)
• g (x) 0 (5)

28
Multiobjective Optimization
29
Different Evolutionary Techniques

• Genetic Algorithms (GA)
• Simulated Annealing (SA)
• Ant Colony Optimization (ACO)
• Tabu Search (TS)
• Particle Swarm Optimization (PSO)
• Differential Evolution (DE)
• Memetic Algorithm (MA)
• Simultaneous Perturbation Stochastic
  Approximation (SPSA)

30
What is GA ?

• GAs are computer based search and optimization
  algorithms based on mechanics of natural genetics
  and natural selection
• A population of initial solution is generated
  within feasible region
• The main idea is
• - Survival of the fittest
• - Evolution of species with time
• Only best solution will survive till end

31
What is GA ?

• Genetic Algorithms (GAs) were invented by John
  Holland and developed by him and his students and
  colleagues. This lead to Holland's book
  "Adaptation in Natural and Artificial Systems"
  published in 1975.
• All living organisms consist of cells. In each
  cell there is the same set of chromosomes
• A chromosome consists of genes, blocks of DNA.
  Each gene encodes a particular protein
• Complete set of genetic material (all
  chromosomes) is called genome.
• Particular set of genes in genome is called
  genotype.

32
Working Principle

• Let us consider the maximization problem
• Coding
• - Variable xi are first coded into binary
  strings
• - Length of string is determined based on
  desired accuracy of solution

33
Working Principle

• Fitness function
• - GA are based on survival-of-the-fittest
• - Naturally suitable for solving
  maximization problems
• - Minimization are transformed to suitable
  maximization ones
• - Fitness function is a measure of goodness
  of the string
• - Our target is to keep on increasing the
  overall fitness functions of all the strings
• - Genetic operators perform duty to
  manipulate binary strings so that fitness
  function is keep on increasing on successive
  iterations

34
Working Principle

• GA Operators
• - Reproduction / Selection
• Selects good strings of a population
• Forms a mating pool
• Above average stings are picked from current
  population
• Multiple copies of selected strings are placed
  in mating pool in a probabilistic manner
• No new strings are formed in this phase
• Roulette Wheel or Stochastic Remainder
  Selection methods

35
Working Principle

• Crossover
• - New strings are created
• - It exchanges information among
• strings of mating pool
• - 2 strings are picked at random
• - Point of crossover is probabilistically
  chosen

Children Strings
Parent Strings
36
Working Principle

• Mutation
• - It changes 1 to 0 and vice versa
• - Small probability pm generally lt 0.1
• - Need is to create a point in the neighborhood
  of the current point
• - Performs local search around current solution
• - It maintains diversity of population

Mutation
37
Diversification

• Generate initial population covering entire
  range
• Visit new places
• Extract characteristics of each region
• Cover as much as possible
• Performing Global Search
• All are done by Crossover operator

38
Intensification

• Should be started once search space is well
  scanned
• Visit zones adjacent / nearby to already visited
  
• Check the performance
• Perform local search
• This is done by Mutation operator

39
Algorithm

• Step 1 Do coding, choose selection operator,
  crossover and mutation probability (pc and pm).
  Choose population size (n), string length (l),
  max. no. of iterations (Nmax)
• Step 2 Evaluate each string of population
• Step 3 Perform Reproduction on population
• Step 4 Perform crossover on random pairs of
  strings
• Step 5 Perform mutation on each string
• Step 6 Evaluate strings of new population
• Step 7 Set N N 1 and go to step 3
• Terminate if N gt Nmax or no further improvement
  on string performance

40
Advanced GA

• Multi Point Crossover
• Real Coded GA
• - Real variables are directly used
• - Optimal point of any desired accuracy
  obtained
• Non dominated Sorting
• - To keep versatility of population
• - Give more chance to a poor performer to
  enhance its skills
• Pareto GA
• - Population in a GA simulation is adaptively
  divided into separate subpopulation,
  corresponding to each optimum point by use of
  sharing functions
• - Can get all the solutions of Pareto Optimal
  front in one shot

41
GA - Applications

• Reactor Design Ammonia Synthesis
• Process Optimization
• Cumene Synthesis
• - Phenol Production
• Scheduling Refinery Operations
• Multiphase Trickle Bed Reactor
• Polymerization Processes
• MMA Synthesis
• - Polyethylene Plant
• - Nylon Manufacture
• Water Distribution

42
Differential Evolution

• Introduced by Storn and Price in 1996
• Algorithm works with a population of size N
• Algorithm iterates as follows
• - Generate new vector by adding weighted
  difference of two vectors to third
• - Mix new vector with target vector to yield
  trial vector
• - Replace target vector with trial vector if
  latter is strictly superior

43
Differential Evolution
44
(No Transcript)
45
Differential Evolution

• F and CR are DE control parameters
• F is a real-valued factor in the range (0.0,1.0
• Upper limit on F has been empirically determined.
• CR is a real-valued crossover factor in range
  0.0,1.0
• CR controls the probability that a trial vector
  parameter will come from the randomly chosen
  noise vector

46
Importance of Parameters

• Optimal values are dependent both on objective
  function characteristics and on the population
  size, NP
• Practical advice on how to select control
  parameters NP, F and CR can be found in the
  literature

47
Crossover in DE
48
DE - Applications

• Multiprocessor synthesis
• Power minimisation
• Neural network learning.
• Crystallographic characterization
• Design of Shell-and-Tube Heat Exchangers
• Heat transfer parameter estimation in a trickle
  bed reactor
• Gas Transmission Network
• Water Pumping and Distribution Systems
• Optimization of Ammonia Synthesis Reactor
• Design and Operation of Thermal Cracker

49
DE - Advantages

• Powerful algorithm- multidimensional functions
• Easy applicable to various problems.
• Widely used
• Literature and other materials available
• Generally good accuracy for real world problems
• Easy to implement as same parameter settings work
  fine for a wide range of problems
• Drawback
• Somewhat slow during initial iterations

50
Cumene Synthesis

• Main reaction
• Benzene Isopropyl Alcohol ? Cumene Water
  
• (benzene alkylation)
• Secondary reactions
• Cumene Isopropyl Alcohol ? p-Di-isopropyl
  Benzene Water
• (cumene alkylation)
• p-Di-isopropyl Benzene ? m-Di-isopropyl Benzene
  (isomerization)
• 2 Isopropyl alcohol ? Di-isopropyl ether
  Water
• (alcohol dehydration)

51
Cumene Synthesis
52
Catalyst

• Beta is a crystalline alumino-silicate catalyst
  with high silica content
• Important characteristic is that it is the only
  large pore zeolite with chiral pore intersections
  
• It consists of 12-membered rings interconnected
  by cages formed by intersecting channels
• The linear channels have pore opening dimensions
  of 5.7 ? 7.5 Å
• the tortuous channels with intersections of two
  linear channels have approximate dimensions of
  5.6 ? 6.5 Å
• The catalyst has pore volume of ? 0.2 cm3/g.
• Beta catalyst (1.5 mm extrudates with 20
  binder) in its active protonated form with Si to
  Al ratio of 15 was obtained from M/s UCIL, India

53
Reactor

• Vapor phase isopropylation of benzene was carried
  out in a pilot plant scale stainless steel
  reactor
• A preheater in its upstream and a condenser in
  the down-stream
• Material of construction SS 316,
• Internal diameter (ID) 25 mm
• Wall thickness 6 mm
• Reactor length 33 cm
• Catalyst bed height 10-15 cm
• Heating coils are wound around the reactor to
  provide proper heating and maintain temperature
• Reactor is also jacketed with insulation to
  minimize the heat loss

54
The Operation

• The liquid mixture of benzene and isopropyl
  alcohol was fed to the reactor by a positive
  displacement pump
• Hydrogen was used as the carrier gas
• The condensed products collected were analyzed
  with a Flame Ionization Detector (FID) using a
  Xylene Master capillary column fitted to a
  Shimadzu 15A Gas Chromatograph (GC)

55
Process Parametrs

• Important Operating Variables
• reaction temperature (x1)
• pressure (x2)
• benzene to isopropyl alcohol mole ratio (x3)
• weight hourly space velocity (WHSV) (x4)

• Outputs are Cumene yield and selectivity y1, y2

56
Expt. No. Temperature ( 0C) Pressure (atm.) Benz/IPA (mole ratio) WHSV (hr-1) Yield ( wt ) Selectivity ( wt )
1 110 1 8 3.3 0.07 77.03
2a 145 1 8 3.3 11.6 58.75
3 180 1 8 3.3 15.78 79.93
4 210 1 8 3.3 17.365 90.72
5 215 1 8 3.3 16.09 91.95
6 150 4 8 3.3 12.2 65.74
7 135 4 8 3.3 12.99 74.58
8a 110 4 8 3.3 0.71 80.82
9 100 4 8 3.3 0.19 75.02
10 110 1 10 3.3 0.55 67.74
11 110 1 8 3.3 0.24 54.85
12a 110 1 6 3.3 0.37 53.63
13 110 1 3 3.3 0.2 32.13
14 110 1 1 3.3 0.14 21.62
15 110 1 8 6.8 0.24 54.85
16 110 1 8 8 0.15 44.64
17 110 1 8 9.5 0.13 37.38
18 110 1 8 10.5 0.08 39.3
19a 110 1 8 12 0.09 39.13
20 110 1 8 13 0.07 39.1
21 105 1 8 6.8 0.3 70.38
22 110 1 8 6.8 0.24 54.85
23 115 1 8 6.8 0.35 48.25
24 130 1 8 6.8 4.61 76.68
25 185 1 8 6.8 9.2 59.23
26 210 1 6.5 3.3 20.04 91.8
27 155 1 6.5 3.3 16.93 77.4
28a 180 1 6.5 3.3 20.27 90.9
29 210 1 6.5 3.3 19.86 91.9
30 225 1 6.5 3.3 19.1 89.3
31 250 1 6.5 3.3 17.89 85.2
32 275 1 6.5 3.3 17.29 83.1
33 230 1 6.5 2.5 20.33 91.1
34 215 1 7 5 19.86 91.9
35a 215 10 7 5 19.54 92
36 215 18 7 5 18.68 89.1
37 215 25 7 5 17.74 86.8
38 195 25 6 5 18.92 85.6
39 210 25 6 5 22.1 93.7
40 230 25 6 5 22.02 93.8
41 250 25 6 5 21.35 90.7
42 280 25 6 5 20.48 86.2
57
Expt. No. Temperature ( 0C) Pressure (atm.) Benz/IPA (mole ratio) WHSV (hr-1) Yield ( wt ) Selectivity ( wt )
1 110 1 8 3.3 0.07 77.03
2a 145 1 8 3.3 11.6 58.75
3 180 1 8 3.3 15.78 79.93
4 210 1 8 3.3 17.365 90.72
5 215 1 8 3.3 16.09 91.95
6 150 4 8 3.3 12.2 65.74
7 135 4 8 3.3 12.99 74.58
8a 110 4 8 3.3 0.71 80.82
9 100 4 8 3.3 0.19 75.02
10 110 1 10 3.3 0.55 67.74
11 110 1 8 3.3 0.24 54.85
12a 110 1 6 3.3 0.37 53.63
13 110 1 3 3.3 0.2 32.13
14 110 1 1 3.3 0.14 21.62
15 110 1 8 6.8 0.24 54.85
16 110 1 8 8 0.15 44.64
17 110 1 8 9.5 0.13 37.38
18 110 1 8 10.5 0.08 39.3
19a 110 1 8 12 0.09 39.13
20 110 1 8 13 0.07 39.1
21 105 1 8 6.8 0.3 70.38
22 110 1 8 6.8 0.24 54.85
23 115 1 8 6.8 0.35 48.25
24 130 1 8 6.8 4.61 76.68
25 185 1 8 6.8 9.2 59.23
26 210 1 6.5 3.3 20.04 91.8
27 155 1 6.5 3.3 16.93 77.4
28a 180 1 6.5 3.3 20.27 90.9
29 210 1 6.5 3.3 19.86 91.9
30 225 1 6.5 3.3 19.1 89.3
31 250 1 6.5 3.3 17.89 85.2
32 275 1 6.5 3.3 17.29 83.1
33 230 1 6.5 2.5 20.33 91.1
34 215 1 7 5 19.86 91.9
35a 215 10 7 5 19.54 92
36 215 18 7 5 18.68 89.1
37 215 25 7 5 17.74 86.8
38 195 25 6 5 18.92 85.6
39 210 25 6 5 22.1 93.7
40 230 25 6 5 22.02 93.8
41 250 25 6 5 21.35 90.7
42 280 25 6 5 20.48 86.2
58
Modeling of Output vs. Input
59
Optimization

• Best values of following GA-specific parameters
  were chosen heuristically
• - population size (Npop) 25
• - crossover probability (pcross) 0.82
• - mutation probability (pmut) 0.05
• - maximum number of generations (Ngen) 100
• In order to obtain the best set of operating
  conditions, GA runs were replicated several i.e.
  50 times, using different random number generator
  seeds.
• The fitness function

60
Optimized Results
Soln. No. ANN-GA ANN-GA ANN-GA ANN-GA ANN-GA ANN-GA
Soln. No. Optimized Inputs Optimized Inputs Optimized Inputs Optimized Inputs Maximized Outputs Maximized Outputs
Soln. No. Temp. (0 C) (x1) Press. (atm.) (x2) Benz/IPA (mol ratio) (x3) WHSV (hr-1) (x4) Yield (wt ) (y1) Selectivity (wt ) (y2)
1 271.5 3.38 3.69 12.83 24.88 99.04
2 267.2 1.567 4.05 12.83 24.84 98.90
3 270.08 3.6 4.05 11.76 24.82 98.74
61
Experimental Verification
Exp. No. Experimental Conditions Experimental Conditions Experimental Conditions Experimental Conditions Yield (output 1) Yield (output 1) Yield (output 1) Selectivity (output 2) Selectivity (output 2) Selectivity (output 2)
Exp. No. Temp. (0C) Pressure (atm) Benz/IPA (mole ratio) WHSV (hr-1) GA-maximized value (wt ) Exptl. value (wt ) Error () GA-maximized value (wt ) Exptl. value (wt ) Error ()
1 271.5 3.4 3.7 12.8 24.88 24.69 0.77 99.04 98.98 0.06
2 267.2 1.6 4.0 12.8 24.84 23.79 4.41 98.90 98.70 0.20
3 270.0 3.6 4.0 11.8 24.82 24.58 0.98 98.74 98.65 0.09
Published in Chemical Engineering Journal, Vol.
97, No. 2 3, pg 115 129 (2004)
62
Benefit of the Study

• The work extended from pilot plant level to
  commercial scale
• Implemented successfully by HPCL
• Overall profit increased by almost 18
• Some more research work with HPCL and others
  regarding their multiphase operations
• Leads to optimization of Polypropylene Production
  unit of Reliance at their Hazira plant

63
Overall Conclusion

• Modeling and various approaches discussed
• ANN-based modeling introduced
• Optimization and its necessity
• Multi-objective optimization
• Genetic Algorithm methodology
• Differential Evolution a novel method
• Cumene synthesis case study

64
References

• Rumelhart, D., Hinton, G., and Williams, R.
  Learning Representations by Backpropagating
  Errors, Nature, 323, 533 - 536 (1986).
• Deb, K. Optimization for Engineering Design
  Algorithms and Examples, Prentice Hall of India,
  New Delhi (2006)
• Deb, K., Multiobjective Optimization Using
  Evolutionary Algorithms, Wiley, Chichester, UK
  (2001)
• Nandi, S. Ghosh, S. Tambe S and Kulkarni, B. D.
  Artificial Neural Network Assisted Stochastic
  Process Optimization Strategies, AIChE J, Vol.
  47, pp. 126-141 (2001).
• Nandi, S. Mukherjee, P, Tambe, S. S. , Kumar, R.
  and Kulkarni, B. D. Reaction Modeling and
  Optimization Using Neural Networks and Genetic
  Algorithms Case Study Involving TS-1 Catalyzed
  Hydroxylation of Benzene, Ind. Engg. Chem. Res.,
  Vol. 41, pp. 2159-2169 (2002).
• Nandi, S., Badhe, Y. , Lonari, J. B., Sridevi,
  U., Rao, B. S. Tambe, S. S., Kulkarni, B. D.
  Hybrid Process Modeling and Optimization
  Strategies Integrating Neural Networks/Support
  Vector Regression and Genetic algorithms Study
  of Benzene Isopropylation on HBeta Catalyst,
  Chem. Engg. Jour. Vol. 97, pp. 115-129 (2004).
• Price, K.V. (1999). An Introduction to
  Differential Evolution. In Corne, D., Dorigo,M.
  and Glover, F. (eds.) (1999). New Ideas in
  Optimization, pp. 79108. McGraw-Hill, London.
  ISBN 007-709506-5.
• Storn, R. and Price, K.V. (1995). Differential
  evolution - a Simple and Efficient Adaptive
  Scheme for Global Optimization Over Continuous
  paces. Technical Report TR-95-012, ICSI, March
  1995. Available via Internet ftp//ftp.icsi.berke
  ley.edu/pub/techreports/1995/tr-95-012.ps.Z .
• Storn, R. and Price, K.V. (1997). Differential
  Evolution a Simple and Efficient Heuristic for
  Global Optimization over Continuous Spaces.
  Journal of Global Optimization, 11(4) 341359,
  December 1997. Kluwer Academic Publishers.

65
Thank You