CUC Oct 2005

1
CUC Oct 2005

• Science Data Systems
• Jonathan McDowell

2
SDS Oct 2005

• CIAO Evolution Overview
• PSFs
• Chandra Source Catalog

3
Staffing

• Shrinking science team
• Martin Elvis back to full time research (2004)
• Mike Wise leaving for Amsterdam (Dec 2005)
• Eric Schlegel to U.Tex/San Antonio (Aug 2005)
• Dan Harris to retirement (aka science) (Sep 05)
• No replacements

4
CIAO Releases

• CIAO 3.2.1. - Feb 05 bugfix
• acis_build_badpix header problems, support for
  parameter block input
• CIAO 3.2.2 Jun 05 Repro3 support
• Asphist fixes for multiple observations
• Mkacisrmf bugfixes and support for new CALDB
• New algorithm for HRC degap will improve HETG
  wavelengths
• Change to bad pixel handling in VFAINT mode

5
CIAO Schedule

• CIAO 3.3 November (Tool release)
• New science tools and bug fixes
• reproject_image, reproject_aspect
• CIAO 4.0 2006 (Infrastructure release)
• Sherpa, Chips major functionality improvements
• Data Model improvements
• CIAO script releases TBD
• SAOSAC release - TBD

6
CIAO 3.3

• reproject_image, reproject_aspect
• DM ability to handle data cubes (filter on
  regions, make slices, etc.)
• Proposal tool support for Cycle 8
• Fixes to region area bugs
• Fix problem with FTOOLS/CIAO clash (renamed
  cfitsio library)

7
CIAO Future

• Areas needing improvement for users
• PSFs
• Merging observations
• Background modelling
• Easy fluxes and upper limits
• Publication quality plots
• Robust, fast and programmable spectral and
  spatial fitting (improved Sherpa)

8
PSFs User needs and our plans

• User need
• Ability to make custom PSFs
• Plan Distribute portable SAOSAC/MARX combo
• User need
• Estimate PSF fraction easily
• Plan encircled-energy fraction tables in CALDB
  and software to use them some numerical issues
  at present

9
PSFs ECF table status

• Normalization issue in wings tracking down some
  discrepancies
• HRC-only (flat plane) values characterizing
  error introduced wrt ACIS tilted-chip, appears
  significant

10
PSFs SAOSAC status

• SAOSAC (CXC Optics Group) consists of a
  heterogeneous group of 40 programs and associated
  scripts C,C, Fortran, Perl, Lua. Designed as
  engineering tool
• Available to users via CHART web interface
• Moving to portable version isolated from
  references to local disks, new build system
• Modifying to handle numerical differences between
  Sparc and x86 architecture

11
PSFs SAOSAC and users

• SAOSAC outputs ray files. Need to project to
  focal plane and intersect with instrument
• Will model instrument effects on PSF (but should
  be optional, since PSU has its own detector
  model)
• Will use MARX for both projection and default
  instrument model (may replace/enhance instrument
  model later)

12
PSFs SAOSAC/MARX driver script

• We will write a driver script with a simple
  interface. For a first release
• Single point source only, no grating
• Specify spectrum (via PHA file or Sherpa model
  syntax?)
• Specify off-axis angle and azimuth
• Specify SIM position and fiducial correction
• Specify date? (for time-dependent instrument
  cal?)
• Specify instrument model (MARX or none)

13
Implementation Plan

• CXC Optics to complete SAOSAC portable
  infrastructure by December
• Perform SDS testing and resolve Linux issues
• Identify resources for driver script work
• Documentation effort to guide users
• Identify release schedule by end January
• Release asynchronously with respect to CIAO?

14
Chandra Source Catalog

• Chandra Level 3 Pipeline
• Make a definitive catalog of sources useful for
  individual and statistical studies
• Blind, automatic processing capable of handling
  different instrument configurations and different
  kinds of field (crowded, extended, etc.)
• Runs over full field (handle large off-axis PSFs)
• Good for stars, galaxies, AGN, clusters, etc..
  not just optimized for one kind of source

15
Chandra Source Catalog

• Not just a static catalog
• Access to extracted data products for each source
  via the catalog
• Dynamic interaction to produce user-configured
  views with on-the-fly processing
• Continuous updating as mission continues and data
  becomes public

16
Chandra Source Catalog

• In early years of mission, steadily improved L1
  and L2 calibrated products (event lists)
• We ran detect and made source lists, but these
  products were provided as a guide only.
• Our understanding of the instruments is now at
  the point where automatic processing can go
  further along the analysis chain this was
  always part of our plan.

17
Chandra Source Catalog

• New standard data products extracted spectra,
  ARF, RMF and 4-band postage-stamp images for each
  source.
• New standard data product high quality source
  list
• New standard data product source properties
  table for each source, with accurate positions
  and fluxes from 2D PSF fitting as well as extent
  and variability flags

18
Chandra Source Catalog

• Energy bands for ACIS
• Broad 0.2-7.5 keV
• Soft 0.2-0.5 keV
• Medium 0.5-2.0 keV
• Hard 2.0 7.5 keV
• Only one broad band for HRC
• Bands selected after comparison with those in use
  in the community compatibility with XMM

19
Chandra Source Pipeline - 1

• Reprocess event lists with latest cal, and
  perform high background (flare) cleaning
• Make exposure maps in each band for false edge
  source suppression and construction of
  sensitivity map
• Run multi-scale WAVDETECT in each band
• Scales 1,2,4,8,16
• Identify sources across scales and bands
• Define source and (annular) background regions
  for each source

20
Overlapping sources
21
Chandra Source Pipeline - 2

• Make postage stamp fluxed image in each band
• Use SAOSAC to make PSF for source
• 2D gaussian (PSF-convolved) fits to image,
  attempt to identify source as single, double or
  complex
• Derive source fluxes and position(s). Use power
  law spectral fit with alpha1 and galactic NH.
  (Free alpha if enough counts, currently n300).
• Correct with ECF (PSF fraction) tables
• Variability test (currently KS, soon
  Gregory-Loredo)

22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Near term enhancements

• Merge pipeline combine source data from multiple
  observations and instruments, get best fluxes and
  long-term variability estimate
• Remove readout streaks (out of time events)
• Revise wavdetect parameters based on pilot
  studies and characterize completeness,
  sensitivity, false source rate
• Complication 2D fitting can discover extra
  sources, so threshold is not simple to define.

28
Proposed enhancements

• PSF fraction correction (as fn. of energy) for
  sources. Plan to use fraction within given
  contour, rather than PSU method of selecting
  contour based on given fraction. (Will be useful
  user tool). Method more appropriate from blind
  pipeline with avoidance of nearby sources
  (interactive region tweaking not possible in
  pipeline)
• Background maps

29
Longer term enhancements

• Handle very extended sources (SNRs, clusters)
  using Voronoi tesselation detect
• Mosaic regions (M31, Orion..)
• Gratings data (some day...)

30
Reviews of other processing pipes

• ACIS_EXTRACT reviewed by M. McCullogh, F. Primini
  for functionality further tests planned. We are
  adopting the important approach of
  energy-dependent PSF fraction corrections to the
  ARF.
• AE Currently uses mkpsf for PSF generation, which
  we don't have confidence in.
• XASSIST (A. Ptak) also under review has no PSF
  fitting yet
• XMM Catalog Plan is to do simple celldetect with
  no position refinement, and bright threshold
  (fewer sources)
• CHAMP Used in initial evaluation no HRC support.

31
Operations plan

• Initial evaluation based on 6 test sequences
• Run larger number of sequences for science
  testing over a period of months, while merge
  pipeline in development
• Start production processing circa Fall '06
• Release initial public product when about 1/5 of
  mission has been processed, catalog will evolve
  as more is processed.
• 1 year to process first 5 years
• Access to live evolving catalog and to fixed
  snapshots

32
User interface

• Web-based GUI interface
• Links to data, source and background events,
  images, spectra and responses searches on all
  columns
• VO-aware interface with ability to define
  on-the-fly catalogs in spreadsheet-like way
  iterative searches
• Later enhance to allow workflows (run queries
  which involve processing on the actual data)