Algorithms

1
Algorithms

• A D Kennedy
• University of Edinburgh

2
Outline

• Linear equation solvers
• Deflation
• Local coherence
• Multigrid
• HMC dynamics
• Long trajectories
• Shadow Hamiltonians
• Multiple pseudofermions
• Improved integrators
• Chiral Fermions
• Machines

3
Linear Equation Solvers

• Deflation
• The general idea is to find the low modes of the
  Dirac operator (eigenvectors corresponding to
  small eigenvalues), and then solve the deflated
  system (the Dirac operator restricted to the
  orthogonal complement) cheaply

• The spectral density of such low modes is
  proportional to the chiral condensate (Banks and
  Casher)
• Hence spontaneous chiral symmetry breaking
  implies that the number of low modes is extensive
  (Lüscher)
• Do not need very accurate eigenvectors?
• Assume we want to invert the Hermitian operator
  DD

4
Finding Extremal Eigenvectors
5
Low Mode Averaging

• All-to-all propagators
• The cost of finding the eigenvectors is amortized
  over many solutions with the same gauge field but
  different sources
• The solutions of the deflated system can be found
  using stochastic sources
• Hartmut Neff, Norbert Eicker, Thomas Lippert,
  John Negele, and Klaus Schilling, Phys. Rev. D64
  (2001)
• Leonardo Giusti, Pilar Hernández, Mikko Laine,
  Peter Weisz, Hartmut Wittig, JHEP 0404, 013
  (2004)
• Tom DeGrand and Stefan Schaefer, Comput. Phys.
  Commun. 159, 185 (2004)
• Gunar Bali, Hartmut Neff, Thomas Düssel, Thomas
  Lippert, and Klaus Schilling (SESAM collab.),
  Phys. Rev. D71, 114513 (2005)
• Justin Foley, Jimmy Juge, Alan OCais, Mike
  Peardon, Sinead Ryan, Jon-Ivar Skullerud, Comput.
  Phys. Commun. 172, 145 (2005)
• Philippe de Forcrand, Nucl. Phys. (Proc. Suppl.)
  B47, 228 (1996)
• Ron Morgan and Walter Wilcox, Nucl. Phys. (Proc.
  Suppl.) 106, 1067 (2002)
• Leonardo Giusti, Christian Hoelbling, Martin
  Lüscher, and Hartmut Wittig, Comput. Phys.
  Commun. 153, 31 (2003)
• Peter Boyle, Andreas Jüttner, Chris Kelly, and
  Richard Kenway, Use of stochastic sources for
  the lattice determination of light quark
  physics, arXiv0804.1501

6
Exploiting Spatial Structure

• For free fields the low modes are smooth
• I.e., they have a long-wavelength spatial
  structure
• May be well-approximated on a coarse lattice
• Equivalently may be expressed in terms of basis
  of vectors that are constant within blocks
• But free fields are not a good approximation for
  low modes of Dirac operator in QCD!
• Eigenvalues are gauge invariant, but eigenvectors
  are not
• Can make eigenvectors arbitrarily rough at short
  distance by a gauge transformation
• Smooth low modes by gauge-fixing to Landau gauge?

7
Local Coherence

• Low modes can be well-approximated in basis of
  block modes with the same short-distance
  structure
• Find a few approximate low modes
• Restrict these to blocks to define basis vectors
• Deflate to orthogonal complement of space spanned
  by these block basis vectors

• Martin Lüscher Local coherence and deflation of
  the low quark modes in lattice QCD,
  axXiv0706.2298

8
Adaptive Multigrid

• Another way to make use of local coherence?
• Results only for two dimensional QCD so far
• Coarse grid overhead not included in cost

• James Brannick, Rich Brower, Mike Clark, James
  Osborn and Claudio Rebbi, Adaptive multigrid
  algorithm for the QCD DiracWilson operator,
  arXiv0710.3612

Sunday, 15 January 2012
9
Deflation Acceleration

• Inherit subspace from previous steps solve
• Also use chronological inversion
• Violates reversibility for inaccurate solutions
• Verifying reversibility by measuring energy
  deficit not recommended, as symplectic
  integrators really like to conserve energy
• What happens if rough gauge transformation after
  each step?

• Martin Lüscher Deflation acceleration of lattice
  QCD simulations, axXiv0710.5417

10
HMC Trajectory Length

• Very different from just not measuring on every
  trajectory
• Autocorrelations are smaller
• But even more worthwhile if measurements are
  expensive!
• Free field theory
• Optimal HMC trajectory length proportional to
  correlation length
• Constant of proportionality depends on quantity
  being measured
• Data for full QCD agrees
• dH should not increase with trajectory length

• Harvey Meyer, Hubert Simma, Rainer Sommer,
  Michele Della Morte, Oliver Witzel, and Ulli
  Wolff Exploring the HMC trajectory-length
  dependence of autocorrelation times in lattice
  QCD, arXiv0606004, 2006.
• Harvey Meyer and Oliver Witzel, Trajectory
  length and autocorrelation times Nf 2
  simulations in the Schrödinger functional,
  arXiv0609021, 2006.

11
Symplectic Integrators

• The basic idea of such a symplectic integrator is
  to write the time evolution operator as

12
Hamiltonian Vector Fields
13
PQP Integrator
14
Baker-Campbell-Hausdorff formula

• Such commutators are in the Free Lie algebra

• The Bn are Bernoulli numbers

15
BCH formula

• We only include commutators that are not related
  by antisymmetry or the Jacobi relation
• These are chosen from a Hall basis

16
Symmetric Symplectic Integrators

• In order to construct reversible integrators we
  use symmetric symplectic integrators

17
Shadow Hamiltonian
18
Scalar Theory

• Note that HPQP cannot be written as the sum of a
  p-dependent kinetic term and a q-dependent
  potential term
• So, sadly, it is not possible to construct an
  integrator that conserves the Hamiltonian we
  started with

19
How to Tune Integrators
20
Classical Mechanics on Group Manifolds

• We first formulate classical mechanics on terms
  of the Lie group manifold, and then rewrite them
  in terms of usual constrained variables (U and P
  matrices)

21
Fundamental 2-form
22
Poisson Brackets

• The higher-order Poisson brackets are given by
  similar expressions
• Remember that S(U) includes not only the pure
  gauge part but also the pseudofermion part

23
Exceptional Configurations?

• What limits the HMC integrator step size dt for
  light dynamical quarks?
• It is not exceptional configurations (near-zero
  modes of the Dirac operator)
• As long as the lattice is big enough
• Thermodynamic and/or continuum limits

• Luigi Del Debbio, Leonardo Giusti, Martin
  Lüscher, Roberto Petronzio and Nazario Tantalo,
  Stability of lattice QCD simulations and the
  thermodynamic limit, axXivhep-lat/0512021

24
Integrator Instabilities

• The problem is that our integrators become
  exponentially unstable when dt exceeds some
  critical value
• Equivalently this is where the BCH expansion for
  the shadow Hamiltonian diverges
• This is due to the contribution from the low
  modes of the Dirac operator amplifying
  pseudofermionic noise
• This may be ameliorated by using multiple
  pseudofermions

25
Pseudofermions

• The pseudofermion contribution to the shadow
  Hamiltonian has one more power of the inverse
  fermion kernel than the intrinsic fermion
  contribution
• The pseudofermions are fixed during a trajectory,
  so probably do not suppress this extra inverse
  power

26
Multiple Pseudofermions
27
Multiple Pseudofermion Techniques
28
Multiple Timescales

• Use different integration step sizes for
  different contributions to the action
  (SextonWeingarten)
• Evaluate cheap forces that give a large
  contribution to the shadow Hamiltonian more
  frequently
• Original application failed because largest
  contribution to shadow Hamiltonian was also the
  most expensive
• Special case of improved or higher-order
  symmetric symplectic integrators
• Omelyan integrators (de ForcrandTakaishi)
• Campostrini integrators higher order but large
  coefficients
• Force-gradient (Hessian) integrator

29
Poisson Brackets

• Measure Poisson brackets and adjust integrator
  off-line to minimise cost
• Poisson bracket measurements will allow us to
  separate effect of choice of integrator from
  those of choice of pseudofermion kernel
• Poisson bracket values would make it easier to
  compare different algorithms!

Sunday, 15 January 2012
30
RHMC with Multiple Timescales

• Semiempirical observation The largest force from
  a single pseudofermion does not come from the
  smallest shift

• Use a coarser timescale for expensive smaller
  shifts
• Invert small shifts less accurately
• Cannot use chronological inverter with multishift
  solver anyhow

31
Chiral Fermions Reflection/Refraction
32
Chiral Fermions DWF/Overlap

• Rational approximation to sign function
  (Zolotarev/tanh)
• 5D matrix with Neuberger operator as Schur
  complement
• Preconditioning of 5D matrix e.g., improved
  Zolotarev domain wall (Chiu)
• 4D or 5D pseudofermion fields
• 4D pseudofermions can vary Ns per step
• Only 4D slice of 5D pseudofermions couple to
  gauge fields (Schur complement)
• Rest are cancelled by Pauli-Villars
  pseudo-pseudo-fermions
• Very bad unless same noise is used
• How to evaluate roots of Schur complement with 4D
  pseudofermions?

Sunday, 15 January 2012
33
Chiral Fermions Topology Change

• Are small modes of Wilson kernel in Neuberger
  operator physically important?
• Global versus local topology change?
• Tunnelling between topological sectors always
  suppressed in chiral limit?
• Is the Q0 topological sector connected?

Sunday, 15 January 2012
34
Machines General Principles

• It is only worth building your own machine if its
  price/performance is 510 times better than any
  machine you can buy
• Beware of risks, delays, and effort involved
• Money spent on supercomputers by US and Japan has
  little influence on developments in chip design
• Market is driven by my children!
• Enough transistors per chip now that floating
  point operations are almost free
• Biggest bottleneck is off-chip memory access
• Cost of communications network will get more
  significant

35
Machines Blue Gene

• BG/P is mid-life kicker for BG/L
• BG/Q the next generation
• For details dont ask me I dont know
• Ask Peter or Norman (who arent allowed to tell
  you)

36
Machines GPUs

• IBM Cell
• Intel Larrabee
• Cores/Vectors?
• Does not matter for QCD
• Use data parallel model (not dynamic threads)
• 32 or 64 bits?
• Rounding modes

37
Questions?