Introductory concepts: Computer etiquette

1
Introductory conceptsComputer etiquette

• Jon Goss

2
Outline

• Get organised
• Consistency vs efficiency
• Compute machine vs file server
• Hierarchical calculation strategy
• Restarting calculations
• Throughput vs turn-around time
• Interacting with a batch queue

3
Get organised

• You are in an area of study that has the
  potential to produce vast amounts of data
• Why do we need to store it?
• It is crucial that you adopt a comprehensive,
  structured filing system from the very start.
• Alpha-numerical file naming with index files
  could be chronological.
• Potentially many files per directory.
• Hierarchical set of directories based on
  application
• Try to keep few files per directory.
• Youll probably have to revise your filing system
  over time.
• Acid test
• When asked to locate a file, can you do it within
  a few seconds?
• Minutes?
• Hours?
• Days?

4
Consistency vs efficiency

• When you begin researching an application, you
  may adopt a parameter set from some previous
  calculations to be consistent with them
• The foundation work may already be there
• Bulk material basis sets, lattice constants,
  convergence tests
• This may allow direct comparison between previous
  and current calculations
• Binding / reaction energies
• Marker method calculations for electrical levels
• Structures for better starting points
• It reduces the probability of trivial errors it
  does not eliminate them.

5
Consistency vs efficiency

• Consistency is good, but not at any cost
• The old parameter set may be inefficient in terms
  of CPU time per calculation.
• New parameters may be better designed (e.g.
  optimised basis sets) for this specific project
• In the long run it may be better to spend time at
  the start setting your parameters as this gives
  you a higher level of confidence in the accuracy
  of the results, and grounds you in the
  fundamentals of the calculations
• You may well find that you have to repeat large
  numbers of calculations if you have to revise
  from a sub-optimal setup
• Calculating the same basic values twice is
  unlikely to be exemplary efficiency

6
Consistency vs efficiency
What ever you do, it should be agreed with all
involved in the project, and you should choose
your strategy carefully at the start Think!
7
Compute machine vs file server

• This distinction is part of organisation, and
  is important in
• Avoiding duplication
• Avoiding loss of data
• Allowing access for others involved in the
  project
• Keeping access to important files for yourself

• Some computers are there for calculations
• Rhodes, Verity, Braid, Hector, HPCx,
• Other machines are designed to store files
• Trueman, Snufkin

8
File storage

• We need to answer three questions
• Why do we need to store files?
• Which files do we need to keep?
• How can we minimise file-space usage?

9
File storage why?

• Why do we need to store files?
• Scientific ethics require us to be able to back
  up our claims!
• They are one of our chief resources for future
  research.

10
File storage

• Which files do we need to keep?
• Always AIMPRO standard output, plus
• Bandstructures bandst.out, bandt.plt
• EELS / OA bandst.out, dieln output files
• Mulliken bandt.out
• NEB maybe res.neb
• AIMVIEW bandst.out, dump files (careful with
  these)
• DDS maybe derivs.txt
• DoS bandst.out , dos.out , dos.dump
• Keep all parts of a run
• If you have to restart a relaxation, for example,
  keep all parts.
• What dont we keep in our permanent filing
• Restart dump files!
• fort.99, standard error files (e.g.
  aim2.3.01b.sh.e18371)
• Aimpro input files (dat, pseudo-pots, hgh-pot,
  bandst.dat,) unless they are modified
  specifically for this run, and cannot be
  re-created from the aimpro output.

11
File storage

• How can we minimise file-space usage?
• Most files we generate are ASCII
• They can be reduced in size by compressing them
  (well look again at this in another lecture)
• bzip2 ltfilegt
• (You can learn about this by typing man bzip2 on
  snufkin.)
• A typical aimpro output file may be reduced in
  size by 80 without any loss of data.
• Bzipped files can be viewed, gresd and even
  edited.
• You have a quota on most machines
• If you exceed it, youll be unable to do very
  much
• You may be prevented from logging into the
  machine!
• If there is no quota, you may fill the disk space
  and affect all other users.

12
Calculation strategy hierarchy of costs

• The majority of the AIMPRO computational time is
  taken in obtaining total energies
• The self-consistent cycle.
• We focus on this part of the calculation for
  speed.

Obtain ?in
Generate H
?in?out?
Done
Obtain ?out
13
Calculation strategy hierarchy of costs

• Commonly, a goal may be reached in a sequential
  method, minimizing computational effort
• we want to maximise the amount of science we can
  do so we need to be able to answer the following
  questions
• How do the decisions we make in constructing a
  data file, and how we run the job affect the time
  efficiency?
• Which factors are most important?

14
Calculation strategy hierarchy of costs

• Number of atoms
• Number of different species
• Self-consistency method
• Number of basis functions
• Number of exponents
• Maximum orbital angular momentum
• Initialisation charge density basis
• Number of k-points
• Location of k-points
• Spin state
• Plane-wave basis
• Number of processors
• The amount of memory available

• K-point parallelism
• Number of symmetry operations
• DIIS
• Real-space build
• The amount of vacuum (molecules and surfaces)
• Number of images in a NEB
• Number of k-points in a band structure
• Number of bands in an EELS run
• Number of energies in an EELS run
• Number of atoms included in the derivatives

15
Calculation strategy hierarchy of costs

• Number of atoms
• Number of different species
• Self-consistency method
• Number of basis functions
• Number of exponents
• Maximum orbital angular momentum
• Initialisation charge density basis
• Number of k-points
• Location of k-points (real/complex)
• Spin state
• Plane-wave basis (vacuum)
• Number of processors
• The amount of memory available

• K-point parallelism
• Number of symmetry operations
• DIIS (extensive parallelism)
• Real-space build
• The amount of vacuum (molecules and surfaces)
• Number of images in a NEB
• Number of k-points in a band structure
• Number of bands in an EELS run
• Number of energies in an EELS run
• Number of atoms included in the derivatives

16
Calculation strategy hierarchy of costs

• For an SCF step
• The time for an energy scales as na, a2?4 where
  n is the dimension of the Hamiltonian.
• Going from real arithmetic to complex (at a
  general k-point), the time increases by a factor
  of ½-1 order of magnitude.
• Sampling the Brillouin-zone mp23 generally
  includes 4 (complex) points for a cubic cell.
• Spin polarisation doubles the time taken
• The number of SCF steps for an energy may
  increase with spin-polarisation.
• Compare the time for an energy for a pppp, pdpp,
  ddpp and dddd basis.
• Compare the time for pppp, gamma point, spin
  averaged, and dddd, 4 complex k-points, spin
  polarised.

17
Calculation strategy hierarchy of costs

• The most common calculation we perform is a
  structural relaxation.
• If we do not have an accurate starting structure,
  it is likely that the optimisation will require
  multiple optimisation iterations.
• Why is this likely to be the case?
• The number of structural iterations is
  approximately independent of how we run AIMRPO,
  provided the calculation is performed within a
  reasonable set of parameters (i.e. not
  necessarily convergent ones).
• What does this tell us about the energy surface?
• A structural iteration generally takes more SCF
  steps when were far from the structural minimum.
• This is related to recycling charge densities
  from previous structures why does this affect
  the number of SCF cycles?

18
Calculation strategy hierarchy of costs

• Example
• Substitutional gold in silicon.
• We want to know the symmetry, donor and acceptor
  levels.
• We need to check the convergence with
• Cell size
• Basis
• Pseudo-potential whether the 5d electrons are
  included in the valence or not
• How do we start? Sketch out a strategy to get all
  the data we need.

19
Calculation strategy hierarchy of costs

• Basis
• Design a hierarchical basis set sequence
• C44G ? pdpp ? dddd
• How do we know where we can start?
• And how far we need to go?

YSf
20
Calculation strategy hierarchy of costs

• Sampling
• Start with simplest viable sampling scheme
• Gamma-point, mp23,
• Use k-point parallelism
• The Hamiltonians for different k-points are
  diagonalised in parallel, rather than in serial
• parameteruse_kpar
• If nkltnp, then np must be an integer multiple of
  nk.
• If nkgtnp, then this is not the case.
• How do we know what is viable?
• How does this improve efficiency?
• What are the potential pitfalls?

k1 k2
k3 k4
k4 k4
k4 k4
k3 k3
k3 k3
k2 k2
k2 k2
k1 k1
k1 k1
21
Calculation strategy hierarchy of costs

• Spin polarisation
• For spin-polarized problems, first relax spin
  averaged this makes the calculation run twice
  as fast.
• What does spin averaged / polarised mean?
• Can we always relax S0?
• Is it always really helpful?

22
Calculation strategy hierarchy of costs

• Supercell size
• Start with a small unit cell and embed it in
  larger ones
• 64 ? 216 ? 512 ? 1000 atoms
• Use an anchor point
• Take care over symmetry
• What are the implications for timing?

23
Calculation strategy hierarchy of costs

• Starting structures continued
• Use the structure obtained from one charge state
  to start others.
• Recycle similar systems e.g. use a phosphorus
  structure to start an arsenic one.
• If youve already run in LDA and you now want it
  GGA, scale the structure according to the ratio
  of standardized lengths (typically lattice
  constants).
• What sort of errors might these short-cuts lead
  to, if any?

24
Calculation strategy hierarchy of costs

• Recycle the charge-density!
• When restarting an incomplete relaxation, or
  restarting at the end of the relaxation to get
  some analytical data (e.g. AIMVIEW dumps,
  band-structures, EELS spectra...)
• Use a restart-dump!
• What does the restart dump contain?
• restartmake-dump
• restartload-dump
• restartload-dump,override-positions
• How can the restart be used?
• How much time might this save?

25
Restarting calculations more generally

• Your default mode of operation is to always write
  to a restart dump using
• restartmake-dump,filedump.xxxxx
• You might store them all in one place
• filespace/scratch/njpg/DUMP-FILES
• This is what I do!
• The xxxxx is a unique identifier associating the
  dump-file with the run.
• The restart dump files may be very large in terms
  of disk-usage!
• Restarts also exist for other calculations
• NEB
• This will be discussed in detail elsewhere in
  this course
• Energy second derivatives with respect to
  position
• You should check the on-line documentation

26
Sometimes we have to bite the bullet apple(?!).
After all this
27
Throughput vs turn-around time

• When running a parallel job, there is a scaling
  penalty
• Doubling the number of nodes will NOT half the
  time (in general).
• Why is this?
• It WILL prevent other people from using the
  nodes.
• You will constantly balance throughput (the
  number of jobs completed per day) with
  turn-around time (wall time from start to job
  completion).
• In general
• throughput is maximised by adopting the minimum
  number of nodes for the job, given the batch
  queue time limits
• turn-around time is minimised by adopting larger
  numbers of nodes (subject to scaling).
• Minimising the turn-around time is anti-social
  behaviour.
• Big brother is watching you.

28
Interacting with a batch queue system

• The batch queue systems differ from machine to
  machine.
• It is important that you familiarise yourself
  with
• Memory per node
• CPUs/cores per node
• Time limits
• Scheduling priority
• Disk-usage
• Scaling (interconnect)
• Reliability ?
• How do these relate to the preceding discussion?

29
Interacting with a batch queue system

• Every day, check all of your jobs
• Running
• Finished
• For those still running, check to see
• Is all well?
• Can/should this run be terminated?
• When might this be?
• Maximise the available resources for all users
• For those finished
• Restart them, if need be
• Incomplete relaxation
• Analysis
• File them carefully

30
Concluding thoughts

• Be organised
• Be hardware aware
• Be efficient
• Be sociable
• THINK about what youre doing.