Presentacin de PowerPoint

1
STANFORD UNIVERSITY 01/23/2009 Particle
Swarm a powerful family of optimizers for
geophysical inversion. (Applications in
hydrogeophysics) Juan Luis Fernández-Martínez
(Visiting professor UC-Berkeley) Esperanza
García-Gonzalo (University of Oviedo) UNIVERS
ITY OF OVIEDO (ESPAÑA)
2
Two non-linear discrete inverse problem issues

• The theoretical model is a simplification of
  reality.
• The data are noisy and discrete in number
  (spatial coverage is not complete).

• The topography of the misfit function
• Valley model (Rosenbrock)
• Eggs pack model (Griewank)

• Where the unstabilities enter?

3
Equivalent models and importance sampling
The unstabilities enter when trying to solve the
optimization problem using
Prior Regularisation Other
Solution Perform I-Sampling

Algorithms to perform IS SA (MosegaardTarantola,
1995) BGA (Fernández Álvarez et al., 2008)
4
Particle Swarm Optimization
The particle swarm is an evolutionary computation
technique used in optimization. It is inspired in
social behaviour of individuals in nature, such
as bird flocking and fish schooling
(KennedyEberhart, 1995)
5
A lot of references in engineering optimization

• Electronics Electromagnetics
• Expert systemsMachine learning
• Robotics,Control,Networks
• SchedulingCombinatorial optimization
• EnergyMetallurgy,
• Biomedical Engineering, Financial, etc.

(After R. Poli, 2007)
A few references on earth sciences

• Reddy and Kumar, 2005 Reservoir optimization.
• Fernández-Martínez et al., 2006, 2008 VES, SP,
  transmision tomography. IAMG-06, SEG2008,
  ComputerGeosciences, 2008.
• ShawSrivastava VES, IP MT. Geophysics, 2007.

6
Particle Swarm Optimization

• Each individual samples the model search space M
  according to its own best historical misfit,
  l(k), and its companions , g(k), searching
  experience.
• PSO updates particles position and
  velocities as follows

7
The PSO algorithm

• Random initial swarm (on a prismatic Search Space
  in ) with null velocities.
• Choose the inertia, local and global
  accelerations
• Draw random numbers for each particle of
  the swarm.
• Evaluate the cost function for all the particles
  of the swarm, and determine local (different for
  each particle) and global leaders (common)
• To apply the two discrete gradient algorithm and
  making the swarm evolving.
• Go to 4 till convergence or maximum number of
  iterations.

8
The spring-mass analogy(Continuous PSO)
Trajectory of each individual mimics the motion
of a unit mass, m, attached to two springs with
rigidity constants ?1 and ?2, and damping 1-w,
whose equilibrium positions are l(t) (the
individual best position) and g(t) (the global
best position in the swarm) respectively
9
The introduction of the PSO continuous model
allowed us1. To properly understand the
dynamics of the particles trajectories,
considered as stochastic processes characterized
by their mean, variance and temporal covariance.
2. To generalize PSO (GPSO) and to deduce a
family of PSO versions with different first and
second order stability regions.3. To propose
criteria to choose the PSO tuning parameters
depending on the cost function (problem
physics) and on the task we want to perform
Exploitation/Exploration.
10
GPSO
PSO
GPSO
First order moments
11
PSO case first order trajectories
First order homogeneous trajectories
First order spectral radius
12
The GPSO second order trajectories
13
More about GPSO second order trajectories

14
Other PSO relatives
1. CP-PSO, or centered-progressive PSO
2. CC-PSO, or centered-centered PSO
15
PSO generalizado
First order spectral radius for the PSO, CC-PSO
and CP-PSO for ?t1
Second order spectral radius for the PSO, CC-PSO
and CP-PSO for ?t1
16
Numerical Experiments with benchmark functions
Rosenbrock Valley function in 10 dim.
17
Numerical Experiments with benchmark functions
Griewank Local Minima function in 10 dim.
18
The VES inverse problem
19
The salty water intrusion problem
Studied V.E.S
Geological map of the Aguilas zone
Apparent resistivity curve
20
The salty water intrusion problem Convergence
curves
21

The salty water intrusion problem Depth of the
intrusion posterior histogram
Depth of intrusion posterior histograms for
different PSO versions
22
Posterior analysis and Risk Assestment Percentile
curves
23
The Spontaneous Potential Inverse Problem
24
Topography of the misfit function
A Gaussian synthetic model hwaterf(mean, st.
dev)
Noise-free
3 Error
25
SP- Synthetic case example
Noise-free
20 white Gaussian noise
26
SP- Field case example
SP profile from Bogoslovski Ogilvy, 1973
27
SP- Field case example
28
SP-PSO design

• Search space
• Smooth version of gbest
• w0.8, ag2.0, al1.8
• Posterior analysis of results median model and
  inter-quartile range

29
Sampling and curse of dimensionality

• I. Oriented sampling

1. Linearize the Forward functional 2. SVD
3. Rotate the Search Space and scale it
according to Singular values and tolerance 4.
Sampling the O.S.S
30
Sampling and curse of dimensionality
II. Stretching the cost function
31
Sampling and curse of dimensionality
III. The next step Interpolation

• 1. Neighborhood algorithm
• (Sambridge, 2000)
• 2. Support Vector Machines and Kernel methods
• (Kuzma and Rector, 2004)

32
(No Transcript)
33
Some of our references

• J.P. Fernández-Alvarez, J.L. Fernández-Martínez,
  and C.O. Menéndez Pérez. Feasibility of the use
  of binary genetic algorithms as importance
  samplers. Application to a 1D-DC inverse problem.
  Math. Geosciences, 2008.
• J.L. Fernández-Martínez, E. García-Gonzalo and
  J.P. Fernández-Alvarez. Theoretical analysis of
  Particle Swarm trajectories through a mechanical
  analogy. International. Journal of Computational
  Intelligence Research. Special Issue on PSO,
  2008.
• J.L. Fernández-Martínez, E. García-Gonzalo. The
  generalized PSO a new door to PSO evolution.
  Journal of Artificial Evolution and Applications,
  2008.
• J.L. Fernández-Martínez, E. García-Gonzalo.
  Stochastic analysis of the continuous and
  discrete PSO. Swarm intelligence. In revision.
• J.L. Fernández-Martínez, E. García-Gonzalo. The
  PSO family deduction, stochastic analysis and
  comparison. Swarm intelligence. In revision.

34
El modelo PSO continuo
35
The oscillation center dynamics
36
GPSO continuous PSO comparison
Continuous PSO in complex zone
Continuous in complex zone. GPSO in real zone
37
Corresponding trajectories
38
Topography of the misfit function
Crossection on the layer having intrusion