Cosmic Fireworks: How Black Holes Regulate Galactic Evolution

1
Cosmic Fireworks How Black Holes
RegulateGalactic Evolution

• Rob Thacker
• Dept. of Physics, Queens University
• (CITA National Fellow)

With Evan Scannapieco (UCSB) Hugh Couchman
2
Canadian Collaboration Success!
3
Outline

• National international scene, and a bit of
  subject background
• Physics computational tools
• Our discoveries
• Challenges of large data sets, UCLP possibilities
  etc

4
Canadian astronomy community background

• ISI on citations per paper Canadian astronomy
  ranks 1 in the world!
• Number of world-leading numerical astrophysicists
• Large-scale structure simulations
• Galaxy formation
• Galactic structure modelling
• Planet formation, solar system modelling
• Cosmic Microwave Background data analysis
• Magneto-hydrodynamics jets
• General Relativity

5
International Scene The Virgo Consortium
http//www.virgo.dur.ac.uk/
UK-German-Canadian-Japanese-US collaboration
Max Planck
McMaster
Sussex
Durham
Pittsburgh
2nd June 2005
Nottingham
6
What are we trying to explain?

• Global evolution of Universe
• Large scale distribution of matter
• Properties of matter between galaxies
• Formation of galaxies, properties of galaxies
• Terminology
• Redshift ratio of size of Universe now to the
  size it had in the past (actually ratio-1)
• pc parsec, 3.26 light years (For scale, size of
  visible Universe6000000000 pc6000 Mpc)

7
Dark matter state-of-the-art
Has provided the first true simulation where we
can track the formation of galaxies in
a volume equivalent to a (small) galaxy survey
8
Modelling cosmic evolution
Time

• Smooth initial distribution of mass
• 90 of matter is invisible
• cold dark matter
• Cosmological evolution includes cosmological
  constant
• Evolve over time
• solving coupled equations for gravity and
  hydrodynamics, plus radiative processes
• Structure forms via a hierarchical process
  bottom up

Smaller systems progressively merge to form
larger ones
Following dark matter is zeroth order
investigation dominates on large
scales but NOT small
9
Structure growth dark matter evolution
10
(No Transcript)
11
The length, mass and time-scale challenge

• What is the ratio in size of group of stars to a
  galaxy survey?
• 107
• What is the ratio in mass of a group of stars to
  the galaxy in which they are embedded?
• Between 104 and 108
• What is the ratio in mass of smallest galaxies to
  a well sampled, statistically viable volume for
  galaxy surveys?
• 1010
• What is the ratio of the age of the universe to
  shortest time-scale impacting evolution
  (supernovae remnant evolution)?
• 108

So we dont expect to do everything in one
simulation! (YET!)
12
Outline

• National international scene, and a bit of
  subject background
• Physics computational tools
• Our discoveries
• Challenges of large data sets, UCLP possibilities
  etc

13
The New Cosmology

• In past 10 years new observational data has
  fueled a revolution
• CMB measurements constrain key parameters
  initial conditions
• Study of galaxy clustering completely
  revolutionized
• Galaxy formation clustering is more complex
  than we thought
• Clustering studies measure across a variety of
  types of galaxies at different stages of
  evolution
• This is either incredibly annoying, or
  incredibly interesting, depending upon ones
  perspective Charles Steidel, Caltech.
• Theory/simulation just beginning to incorporate
  detailed (3d) physics
• Difficult to make orders of magnitude leaps in
  understanding
• Underlying physics is understood as separate
  subject fields
• More computing is useless without improvements in
  physical modelling

Old Cosmology
Keck Telescopes
14
Black Holes in a cosmological setting

• Singular points in spacetime, masses can vary
  from few solar masses to tens of millions
  (supermassive)
• Matter accreting on to black hole is tidally
  forced into an accretion disk
• Exact physics of accretion disks is still under
  investigation
• Must dispose of angular momentum (magnetic field
  turbulence?)

15
Active Galactic Nuclei Quasars

• Most luminous accretion disks are in Quasars
• Luminosities exceed can exceed 1000 galaxies
• Time variability suggests energy eminates from a
  region comparable to the solar system size
• Intermediate brightness systems in the nuclei of
  galaxies are called active galactic nuclei
  (AGN)
• Brightness is not just a function of size must
  supply fresh material to feed the accretion

16
(No Transcript)
17
Energy scales
1070
1060
1050
1040
1030
1020
1010
1
10-10
10-20
Joules
18
Sub-resolution modelling

• How do you accurately incorporate processes below
  your resolution level?
• Need model that is a function of your integrated
  variables
• Can calculate black hole mass directly using an
  observational derived relationship
• We can then estimate luminosity from a (well
  understood) physical model
• We can then directly calculate the energy release
  for a simple model

19
Code Details

• Simulations performed using parallel version of
  the Hydra code (Thacker Couchman, 2005)
• Shared memory (OpenMP) due to dynamic load
  balance issues (but also have MPI-2 code)
• OpenMP version scales excellently

20
Performance on HP Superdome
Speed-up
Ncpu
57x speed-up on 64 processors
22563 particles
21
Simulation details

• 5?108 mass elements (half gas, half dark matter),
  10243 grid
• Parallel (OpenMP) Hydra code (Thacker
  Couchman 2006, in press)
• 140 000 cpu hours, 11 000 iterations 16 cpu
  years
• Parallel I/O
• Run on 256 processor Origin at WestGrid, and new
  64 processor IBM P5 server
• Memory consumption 220 GB
• Total disk output 2 TB

22
Outline

• National international scene, and a bit of
  subject background
• Physics computational tools
• Our discoveries
• Challenges of large data sets, UCLP possibilities
  etc

23
Entropy evolution (slice from simulation)
24
Entropy final state
Thanks to Jon Johansson, CNS, U. of Alberta
25
Results Luminosity function of Quasars

• Y(LB,z) number of quasars per unit volume
  having luminosity in a range LB at redshift z
• Model for quasar LF was developed by Wyithe and
  Loeb (2002)
• Fundamental assumption Quasars are supplied fuel
  by during the merging of galaxies
• Model works well at high redshift but over
  predicts the abundance of low redshift

26
Our results luminosity function of Quasars

• Matches high redshift evolution well
• Impact on bright end not as big as in
  semi-analytic model why?

Brighter
Fainter
Redobs Bluesimulation Greensemi-analytic model
27
Substructure issues
Model of nearby galaxy Semi-analytic treatment
Model of nearby galaxy Reality simulation
28
How do quasars cluster? (Correlation function of
Quasars)
Bluesim
2dF results (Croom et al 2001)

• Our simulation agrees with the observed turn-up
  in the small scale clustering of quasars
• CF is explained by the halo model of clustering
• No need for special physics

Sloan binary quasar data (Hennawi et al 2005)
29
Quasar-Galaxy cross correlation function

• Bluegg
• Redqg
• HashDEEP2 (from Coil et al
• 2006, observations)
• Blue line 2x1012 DM halos
• Upturn very difficult to measure accurately (too
  few quasars)

30
Outline

• National international scene, and a bit of
  subject background
• Physics computational tools
• Our discoveries
• Challenges of large data sets, UCLP possibilities
  etc

31
The Challenge of Large Datasets

• Simulation generated about 2 TB of data not
  that much by modern standards, but still a
  headache
• Both network bandwidth and user knowledge are big
  issues
• Huge gap between throughput for raw versus tuned
  file transfer utilities
• Last mile connectivity still a problem
• User Controlled Light Path technologies (UCLP)
  would be terrific for this kind of data movement

Globus toolkit provides utilities that
help alleviate these problems
32
Visualization

• Multi-terabyte datasets present real problems
• exceed 32bit memory space of many viz tools(!)
• cant find enough disk space
• viz people (at the moment) simply arent used to
  datasets of this size
• We have clusters with hundreds of GB of memory,
  but SMP viz servers tend to have 10-20 GB
• Distributed viz tools are coming, but progress is
  slow
• Time-domain visualization is an enormous problem
• Bandwidth problem cant get TB of data in and
  out of memory

256P SGI at WestGrid
33
What of the future?

• TMT US-Canada 30m telescope (1 Bn US)
• European Southern Observatory project OWL (100m
  diameter!)
• Overwhelmingly large telescope!
• Same price!

34
Future of simulations?

• What could we do with this computer?
• A simulation with 1012 mass elements
• Large enough to resolve evolution of every galaxy
  in observable universe
• 6 Petabytes of data

35
Conclusions

• New observations and computational models have
  revolutionized cosmology
• Latest simulations use TB of memory and produce
  tens of TB of data
• Black holes have a significant effect on the
  evolution of the most massive galaxies
• Prevent mass accretion in the late universe
• Latest simulations show we dont understand some
  key details
• Many questions still to be answered