Sum of Squares Optimisation and Potential Energy Surfaces

1
Sum of Squares Optimisation and Potential Energy
Surfaces

• Martin Burke
• Yaliraki Group
• PG Symposium 2005

2
The Potential Energy Surface
Glassy systems Protein folding Crystalline
Structures
3
The Potential Energy Surface
Interactions
Bonded
Non-bonded
3N dimensional space
4
PES What can we learn?

• Minima
• Stable structures
• Local
• Global

Introduction to Theoretical Chemistry, J. Simons
5
PES What can we learn?

• Saddles
• Transition states
• Dynamics
• Reaction pathways
• Transmission Coefficients

Introduction to Theoretical Chemistry, J. Simons
6
PES What can we learn?
Estimate thermodynamic properties
Wales Evans, J. Chem. Phys, 118, p3891
7
PES What can we learn?

• Surface Topology
• Pathways via saddles between minima

Vertices minima Edges connectivity's of minima
8
PES Why is it hard?
Non- convex
Potential Energy Surfaces are generally nonconvex
9
PES Why is it hard?

• Atomic clusters

Exponential increase of minima with N
10
PES Scaling
PES Why is it hard?

• Permutational isomers

7 particle LJ atomic cluster 4 minima 504
1680 1680 5040
11
PES Why is it hard?

• Topologies
• Connectivities

12
Current Methods
Heuristic Approaches

1. No proof unless exhaustive search
2. Sample small part of space
3. Trapped in high barriers
4. Multiple low minima far apart, poor
   connectivities

Stochastic Approaches Monte Carlo / Molecular
dynamics based Others
Monte Carlo / Molecular dynamics Simulated
Annealing J-walking Parallel Tempering Flooding Ba
sin Hopping MCM
Other Genetic algorithms Eigenvector following
13
Current Methods
Convex
Deterministic Approaches Hypersurface
deformation Branch Bound
Non-convex
J. Phys. Chem.,(1989) 93, p3339
Hypersurface Deformation Methods Diffusion
Equation Method Distance Scaling Method
14
Combinatorial Complexity Theory

• Study of how difficult a problem actually is

• Asks 3 questions
• Can you prove a solution is the correct solution
• How long does it take to verify it
• How long to find a verifiably correct solution

• Two categories of solution time
• polynomial of N
• exponential of N
• Polynomial time lt Exponential time

15
Combinatorial Complexity Theory

• 3 categories of problems

Finding global minimum is an NP hard problem
(Frank Stillinger)
16
Summary

• Methods
• Exhaustive sampling
• Stuck in subset of configurational space
• No proof for global minimum
• No uniqueness proof

• PES
• Provides useful information
• Detailed knowledge impossible not necessary
• Number of minima scales exponentially with N
• Global min NP hard

Finding Global Minimum
Mapping from NP hard to P Doesnt require any
sampling of conformational space Mathematical
proof for global mimimum Mathematical proof of
uniqueness
17
Sum of Squares Optimisation
Key Concept
Many sets that are non-convex are projections of
convex sets in higher dimensions
18
Sum of Squares Optimisation
Parrilo (03)
Mapping
Equivalent
Lifting to higher dimensional space
Guaranteed lower bound
Polynomial time computable
NP hard
19
PES to Polynomial Optimisation

• Classical Potential Energy functions

• Polynomial Optimisation

20
Sum of Squares Optimisation
Sum of Squares
What is a sum of squares?
Globally non-negative (sufficient condition)
21
Sum of Squares Optimisation
Given a polynomial U(x) of degree 2d
z is now a basis of U(x)
Linear relationship between coefficients ai Qij
22
Sum of Squares Optimisation
Example
Lift from 1 variable to 3 variables (Dimensions)
Convex ) solution on boundary
23
Semi-definite Programming
Optimisation over set of positive semidefinite
matrices
Subject to linear constraints
Convex Sets
Projections need not be convex!
Now has very efficient polynomial time algorithm
(Nesterov Nemirovski)
24
Semi-definite Programming Duality
Duality Properties Solve 2 problems
simultaneously Each gives information about the
other
25
Sum of Squares Duality

• Primal problem
• Coefficients in polynomial get lifted
• Lower bound on global minimum.

• 2 results from Dual
• Proof of lower bound
• Configurational coordinates of lower bound

• Dual problem
• Configurational coordinates get lifted
• Location of lower bound on global minimum

26
Summary of Procedure
27
Some Examples
Bound on global minimum
SOS function non-convexity
Degenerate global minima
Sum of Squares Optimisation
Energetically similar minima
Time comparison with exhaustive search methods
Poorly connected minima
28
Example 1
Global minimum Energy -2.106 Coordinates
-0.98043 Energetic similarity lt0.01 to next
minimum All minima within 0.1 Poor
connectivity Like LJ38 cluster Global minimum in
narrow well
29
Example 1
Sum of squares results
No. of minimisations 1
CPU time (s) 0.65
Lower bound -2.106
Coordinates -0.98043
Duality gap 3.8569E-6
Primal -2.106
Dual -2.106
IP iterations 14
30
Example 1
Comparison with exhaustive search
BFGS (Broyden, Goldfarb, Fletcher,
Shanno) quasi-Newton scheme
Coupled with a grid search (evenly spaced points
across domain -11)
Coordinates Energy exact
Required 11 points minimisations
Time 0.74 s
31
Example 1
Comparison with exhaustive search
Simplex method with grid search Non-linear
optimisation Very fast
Coordinates energy exact
Required 11 minimisations
Time 0.24 s
32
Example 1
Summary of comparisons
  Exact energy? Exact coordinate? Solution time (s) No. of minimisations Proof?
SOS Yes Yes 0.65 1 Yes
BFGS grid Yes Yes 0.74 11 No
Simplex grid Yes Yes 0.24 11 No
33
Example 2

• Muller potential
• Difficult reaction pathway test case
• 3 minima
• Reactants well
• Global minimum
• Intermediate well
• Requires sharp change in search direction at
  saddle

34
Example 2
Fitted to 16th degree polynomial
No. of minimisations 1
CPU time (s) 10.06
Lower bound -143.52
Coordinates (-0.5586,1.4486)
Duality gap 9.101E-7
Primal 143.52
Dual 143.52
IP iterations 28
Also found saddle by using eigenvector directions
35
Example 2
Sum of squares function
Non-convex in coordinate space Important detail
preserved
Convex in lifted space
Global minimum coordinates exact Function
non-exact but can be made so
36
Example 3

• Degenerate Minima

Classic optimisation test case 6 minima 2
degenerate global minima
37
Example 3
Eigenvalues of solution matrices Primal
solution Degeneracy's in global minima Dual
solution Coordinates of degenerate global minima
38
Conclusions
PES complexity
Today Classical Global Minimum Broader
applicability Future work
39
Acknowledgements

• Dr. Sophia Yaliraki
• BBSRC Office of Naval Research
• My fellow group members
• Andrew
• Ella
• Joao
• Leonid
• Lucy
• Dr. Mauricio Barahona (Bioengineering Dept.)
• Prof. Pablo Parrilo (MIT)
• Antonis Papachristodoulou (Caltech)

40
Sum of Squares Optimisation

• Shor 87

• Positivstellensatz (Stengle 74)
• Practical formulation

How to find Coefficients???
41
Implementation
42
Hybrid Molecular Dynamics Conformational Space
Annealing
43
Sum of Squares Optimisation

• 2 components
• Sum of Squares
• Semidefinite Programme (SDP)
• What is a Sum of Squares?

• If f(x) is SOS
• ) f(x) 0 for all x
• Globally non-negative
• (sufficient condition)
• Always convex!!!

Local minimum Global minimum
44
Exact energy? Exact coordinate? Solution time No. of minimisations
SOS Yes Yes 0.65 1
BFGS grid Yes Yes 0.74 11
Simplex grid Yes Yes 0.24 11
45
(No Transcript)
46
Semi-Definite Programme

• Optimisation over set of PSD matrices
• Any Convex domain defined by polynomial
  equalities inequalities
• 8 years ago SDPs were intractable (except
  analytically)
• Now polynomial time algorithm
• Interior Point Methods (Nesterov Nemirovski)

47
Conclusions Future Work

• Conclusions
• SOS is completely different approach to Energy
  Landscapes
• Proofs / Bounds
• Polynomial time computable
• Other global information attainable
• JCP paper submitted
• Challenges remain
• Computational issues
• Efficient way to implement linear constraints
• Embedding constraints

• Future Work
• Exploitation of properties of primal dual
  matrices
• Eigenvalues, eigenvectors
• Reaction coordinate estimation
• Saddles
• AMBER force field
• Efficiencies
• Removal of symmetries a priori

48
Semi-definite Programming Duality
Duality Properties Solve 2 problems
simultaneously Each gives information about the
other
49
PES What can we learn?

• Estimate microcanonical
• entropy
• Hyperarea A(E) of connected region at energy E
• Measure of configuration space available at
  energy E

50
Some Examples
Comparison with search using BFGS
51
Summary

• Methods
• Exhaustive sampling
• Stuck in subset of conformational space
• No proof minimum global min.
• No uniqueness proof

• PES
• Provides useful information
• Detailed knowledge impossible not necessary
• Number of minima scales exponentially with N
• Global min NP hard

Finding Global Minimum
Mapping from NP hard to P
Mapping non-convex to convex BUT Retaining
original problem
52
Some Examples
Comparison with simplex method
4 minima No degeneracy 14 attempts required by
simplex method