The Multiband Imaging Photometer for Spitzer:

1
The Multiband Imaging Photometer for Spitzer
MIPS 24 Micron Pipeline

• Deborah Padgett
• MIPS Instrument Support Team/24 µm Lead
• Spitzer Science Center

Spitzer Data Analysis Workshop August 2005
2
Outline

• Introduction to MIPS-24
• Data Taking Strategy
• Basic Calibrated Data pipeline and products
• Post-BCD Products
• Common problems

3
MIPS 24 ?m Detector Characteristics
40
4
BCD Automated Pipeline Results Single Exposure
(Frame or DCE)
Raw image
BCD image
5
on-orbit performanceAutomated Mosaics
6
MIPS 24 ?m Data Taking Strategy

• All arrays take data simultaneously
• Continuous readout in 4 readout channels
• Takes 0.524 sec to readout entire array (sample
  time)
• Non-destructive reads every 0.5 sec throughout
  exposure
• 16-bit signed integers and 16384 pixels
  high data rate 16 reads 0.52 MB
• If every read saved, spacecraft buffer overflows
  after 10 minutes
• Needed lossy compression scheme Sample Up the
  RampSUR mode

7
Sample Up the Ramp
Slope count rate (DN/0.5 sec)
1st Difference (DN/0.5 sec)
8
MIPS-24 Data CompressionSUR Mode

• MIPS instrument does a linear least-squares fit
  to data samples on board and saves the count rate
  (slope)
• Instrument also differences the values of the
  first and second samples (selectable) and saves
  as another count rate (difference)
• If pixels saturate during the exposure, but not
  before the end of the second sample (1 sec), then
  pixel is soft saturated and difference image
  gives useful count rate
• Raw data arrives from spacecraft as a 2 plane
  image
• Problem in nomenclature IRS saves every sample
  in their exposure and calls their data-taking
  mode sample up the ramp we call it RAW mode

9
Early Pipeline Results Pinhole Data

• Data from Low Background Test Chamber with
  Pinhole optical fiber light source
• RAW mode shows accumulation of charge
• Last image is equivalent to SUR mode slope

Note latent images and faint mux bleed
10
Slope DifferenceExpands Dynamic Range

• On-board thresholding only saves first difference
  values above 50 MJy/sr others set to zero to
  lower data rate
• Pipeline threshold for replacing slope with
  difference value above a high value which
  indicates soft saturation during exposure
• In the SSC BCD file, soft saturated pixels in the
  slope images are replaced by those pixels from
  the difference image
• Hard saturation (even in difference image) at 4.1
  Jy (1 sec) or 260 MJy/sr (10 sec)
• Because we calibrate both the slope and
  difference images, 1 Data collection event (DCE)
  10 FITS files for Basic Calibrated Data (BCD)

11
MIPS 24 ?m SUR is the Only Data Collection Mode
for Science Data
Slope Image
Difference Image
12
Challenges for MIPS-24 SUR Pipeline

• Use of SUR mode means we only get the slope image
  and first difference image some tasks more
  difficult
• Linearization
• Saturation
• Latent images
• Cosmic Rays

13
MIPS-24 Saturation

• Saturated pixel may have lower slope than
  non-saturated pixels, causing donut in slope
  image of PSF in SUR mode
• Replacement of saturated pixels with scaled
  difference image is used to recover correct
  photometry

What we get from slope image
What we get from difference image
What we want
14
MIPS-24 Cosmic Rays in SUR Mode

• Since only a slope image is returned in an
  individual frame, cosmic rays are diluted
• Lack of multiple samples means no redundancy in
  single exposure as in Ge arrays where partial
  ramps are fit and spliced together
• Single frame detection of possible CR at BCD
  level bits set in BCD level masks beware of
  overzealous CR identification in BCD mask files
• Multiple dithers (? 5) in all MIPS AOTs allows CR
  repair at mosaic level this is the principal
  tool to remove CR from 24 ?m images

15
High-Level Pipeline Requirements for MIPS-24

• Absolute flux calibration 10 (doing about 7)
• Photometric repeatability 4 within/between
  AORs including calibration uncertainties much
  better than this
• Photometric repeatability at different positions
  on the array 5
• Monochromatic flux density lt5 error at
  effective wavelength
• Pointing accuracy lt1.4 absolute, lt0.6 (0.1
  lambda/D) relative
• Point source sensitivity 0.37 mJy, 5 sigma in
  500 sec, low background Actual sensitivity is
  0.11 mJy 5s in 500 sec
• FITS format data products Simple FITS binary and
  header
• Pipeline throughput gt24 Gb/day
• Pipeline speed process 12 hours in lt 6 hours
• Reprocessing speed Process entire year in 1 month

16
MIPS-24 BCD Data Products

• Raw Data products
• Unprocessed FITS format image files with 2 image
  planes (24mm arrays acquire data in all observing
  modes, all is useful science data).
• Basic Calibrated Data (BCD) products Highest
  quality final calibrated data that can be
  obtained from an automated pipeline system. One
  BCD image will be produced per DCE (Data
  Collection Event). Processing includes
• read-2 correction
• droop correction
• dark subtraction
• linearization
• flat-fielding
• flux calibration (DN/sec to MJy/sr)
• cosmic ray flagging
• saturation flagging
• FITS keyword coefficients for optical distortion
  correction
• NOTE no optical distortion correction applied
  for BCD

17
MIPS-24 BCD PipelinePart 1
Sanity Check
Input 2 plane raw image (slope, difference)
Translate Header Keywords
Flip image in x-direction
Convert to floating point detect saturation
Post-launch modification
Convert DN/sample time to DN/sec
Read-2 Correction
Read-2 cal file
Slope desaturation using difference image
Next page
Droop correction
18
MIPS-24 BCD PipelinePart 2
Rowdroop correction (not needed)
Droop-corrected slope and difference images
Subtract sky dark
Sky dark cal file
Nonlinearity correction
Flat Field Division
Synthesized Scan mirror dependent flats
superflat
Convert DN/sec to MJy/sr
Replace saturated slope pixels with difference
Ensemble latent image detection and pointing
transfer
Split multiplane images into single plane images
Single frame radhit detection
19
Intermediate BCDPipeline Products
Dark subtracted
Raw slope
Calibrated slope
Droop Correction
Linearized
20
Read-2 effect and Gradients

• An offset in the second read causes SUR slope to
  be too large
• For typical backgrounds, total gradient of 1.5
  in BCD corrected in pipeline
• Other gradients seen in mosaics due to zodi, flat
  field residuals 2 in mosaic images

pixel in row 2 RAW mode data samples
21
Droop Effect

• Droop is an effect which makes signal of each
  pixel proportional to signal seen by rest of
  (entire) array
• Analysis of test images confirmed that droop
  coefficient is 0.32 for MIPS 24 a large effect
• A/D converter pegs at 1/3 detector full well
• We do not know droop well if many saturated
  pixels
• This explains the high overall background in
  images with many saturated pixels

22
MIPS-24 Detector is Slightly Nonlinear
Deviation from linear fit only /- 2 in 10
second exposure
DN vs. time for a single pixel
23
MIPS-24 Linearization Analysis

• Pixel value versus time easily fit by 2nd order
  polynomial as shown in analysis fit is to
  non-saturated part of the ramps
• Module SLOPECOR uses these quadratic fits to IOC
  data in applying linearity correction to slope
  images

Different colors denote different individual
pixels
24
Calibration Files

• Most calibration files are not currently
  generated automatically per campaign
• Those that are not generally have little effect
  on data
• Darks are very stable dark counts are very low
• Few pixels permanently affected by solar storms
  anneal at campaign start good strategy
• Read-2 calibration file will probably be updated
  this spring to improve background gradients
• Flat fields are a major complication since they
  change with scan mirror position and campaign

25
Uncorrected Instrumental Signatures

• Latent images - 1, decay time tens of seconds
  flagged
• Stray light/glints 1 glint 1.5 degrees from
  center possible ghost only seen for extremely
  bright source off-frame linear glints
• Saturation artifacts
• Strong jailbars
• Dark latents (2) decay time of many hours
• Long term bright latents (0.5) last until
  anneal
• DCE0 SUR algorithm 10-15 lower signal
  currently recommend that DCE0 frames not be used
• Post-anneal response drift (lt 1) 3 hr timescale
• Other background gradients (ltlt1 in BCD 1-2 in
  mosaics)
• Low level residual jailbars (ltlt1) noisy,
  time-variable readout
• Pick off mirror spots (fix implemented but
  time-variable requiring complex solution
  discussed shortly)

26
What the Heck are These?
27
Pick-off Mirror Dust Contamination

• Pick off mirror has dust or paint flecks from
  launch
• As scan mirror moves, the position of these dark
  spots move around the array
• Image is a map of spot positions over the normal
  range of mirror travel
• Spot pattern is stable no evidence for growing
  or jumping spots

28
Moving Spots
Spots move as scan mirror moves
Ratio of scan mirror dependent flat to
standard flat.
Scan Mirror Angles
1864.5 1886.0 1907.5 1929.0 2106.5 2128.0 2149.5
Diffuse Features 1-5 Spots 15-20
29
Result of Dividing Data from One Campaign with
Flat from Another
30
Spot Positions Between MIPS Campaigns
Cross-Scan due to spacecraft jitter
In-Scan Due to changing scan mirror zero position
Scan mirror positions do not repeat exactly
between instrument campaigns
31
MIPS-24 versus 2MASS Scan MirrorOffsets Cause
In-scan Pointing Errors
32
Campaigns 11 - 16
Spots Still Moving Between Campaigns
33
Spots Moving within Campaigns
34
Why We Used Scan Mirror Dependent Flats
Through S11

• Mirror position does not accurately repeat from
  one campaign to the next varies by 0.5
• Currently collecting full set of flats for 24
  micron primary data mirror positions
• Started 8MC, but several (most) sets affected by
  latents due to poor scheduling
• Now, constraining flat scheduling to first few
  hours of campaign improves yield
• Unfortunately, spots can move inside each
  campaign variation up to 0.5 rarely from AOR to
  AOR

35
S12Synthesized Interpolated Flats

• F. Masci S. Stolovy implemented synthesized
  flats based on actual data
• Flats consist of interpolation of flat library at
  different spot positions
• Complication scan mirror moves during scan map
  exposures, so dark spots are streaks
• Each DCE will have darkest spot/streak
  centroided output from this module will
  automatically select flats
• Worst case scenario (fast scan, high background)
  97 effective some bright/dark spot pairs
  remain
• Multiplied by a per campaign superflat to correct
  variations in flat field gradients
• Saves the observatory many hours of calibration
  time per campaign

36
Images Flattened with Synthetized Flat vs.
Campaign SMD Flat
37
MIPS-24 User Archived BCD Pipeline Products

• SPITZER.M1.aorkey.expnum.dcenum. root which
  gives important observing information
• raw.fits raw tranheaded 2-plane data with
  pointing information
• bcd.fits BCD slope image with pixels over the
  threshold replaced by difference image pixels
• bbmsk.fits mask file for BCD
• bunc.fits uncertainty image for BCD
• slope.fits slope image
• slunc.fits uncertainty for slope
• slpmk.fits mask file for slope
• diff.fits first difference image
• dfunc.fits uncertainty for difference
• dfmsk.fits mask for difference

38
DCE Mask Bit Definitions

• A 16-bit D-Mask (DCE-Mask) is associated with
  each BCD science product.
• This reports a summary of pipeline processing for
  every pixel in the BCD.
• Currently we have 13 conditions per pixel

• Bit Condition
• ----- -----------------------------------------
  ---------------------
• 0 Incomplete or questionable row-droop
  correction (rowdroop)
• 1 No row-droop correction applied
  (rowdroop)
• 2 Hard saturated (satmask)
• 3 Read-2 correction could not be applied
  (rowfluxcorr)
• 4 Corrected for soft saturation and slope
  value replaced by difference value (desatslope
  and
• satmask respectively)
• 5 Latent-image flag
• 6 Droop removed using questionable value,
  e.g., either due to saturation or data missing
  from downlink (droopop)
• 7
• 8 Flat field could not be applied (flatap)
• 9 Radhit detection (detect_radhit)
• 10
• 11 Pixel masked in pmask - bad hardware
  state (satmask)
• 12 Non-linearity correction could not be
  computed (slopecorr)
• 13 Soft saturated (satmask)
• 14 Data missing in downlink (cvti2r4)
• 15 ltreserved sign bitgt

39
Selected Archived Calibration Products

• mips24_darkest_sur1.fits Dark for DCE0
• mips24_darkest_sur2.fits Dark for other DCEs
• mips24_flatfield_0._1864.75.fits Flat field for
  all exposures at mirror position 1864.75
• As of S12, get separate spot map and superflat
  files. Multiply these together to get your flat
  for each DCE
• mips24_lincal.fits Linearity calibration file
• mips24_pmask.fits permanent bad pixel file
• mips24_rowfluxcorr.fits Read-2 calibration file
• mips24_PRF_Map_Image.fits PSF for BCDs
• mips24_PRF_mosaic.fits PSF resampled and
  reoriented as mosaic
• Note We have not released the MIPS-24 BCD
  pipeline, so darks and linearity calibration
  files are only for illustrative purposes
• the last step in the BCD pipeline is flattening
  (and application of the conversion factor)
• Thus, it is possible to re-flatten MIPS-24 data
  using non-SSC software

40
MIPS-24 User Archived Post-BCD Pipeline Products

• MIPS_M1_aorkey_expid(of first exposure)_processid
  
• msaic.fits MOPEX mosaic (coadd of exposures
  with outlier rejection)
• mscov.fits mosaic coverage map
• msunc.fits mosaic uncertainty map
• In the future, may have source list from APEX
• Beware of automated source extraction which is
  not optimized for your data!
• False detections and/or low completeness may be
  common

41
New Products inS12

• Interpolated flats for all prime MIPS-24 data 1
  per DCE
• Interpolated flats for non-prime MIPS-24 data
  (taken during Ge-prime modes)
• Median background subtracted image from
  combination of all MIPS-24 data per AOR
• uncertainty image included
• enables latent image correction
• In S13, will have NaNs for zero-valued difference
  image pixels

42
Should I wait for S13?

• S11 is current standard for all campaigns
• S12 will be completed by
• Why is it taking so long?
• Generation of SMD flats is by hand
• We are trying to eliminate latents which have
  contaminated the S10 flats in several campaigns
• S11 does not contain significant changes for
  MIPS-24 from S10
• S12 was a big change to the pipeline, but most
  science results will only change slightly mostly
  background gradient will change flat introduced
  latents will vanish in most cases
• Future changes will be incremental possible DCE0
  fix is next
• Data is very good in any case go ahead and do
  your science! ?

43
3 Square Degree Mosaic of Ophiuchus Cloud
44
MIPS-24 PSF Simulation

• Point Spread Function (PSF) is Nyquist sampled
  (lambda/2D)2.9 Pixel size is 2.54. PSF
  shows a great deal of structure.
• John Krists TinyTim for SIRTF produces good
  matches for observed PSF
• SSC provides observed PSF from cal stars to use
  for PSF fitting photometry part of MOPEX

Log scale
Linear Surface Plot
45
Common Gotchas

• Photometry from distorted BCDs
• BCD images do not have distortion removed
• Distortion is significant in images (up to 10)
• Point source photometry from BCDs will change
  with array position unless distortion removed
• MOPEX can produce undistorted BCDs
• First Frame Effects
• First frame in commanded exposure set (DCE0) is
  depressed in response by 10 15
• DCE1 2 also depressed by 1 2
• Dont use these images for photometry unless they
  are rescaled fix is coming S13
• Asteroids! Beware of MIPS-24 sources without
  shorter wavelength counterparts near ecliptic
  plane

46
Asteroids in Ophiuchus Use 2 Epochs at Low
Ecliptic Latitudes
47
Where to Find Additional Information

• MIPS Data Handbook (online, SSC)
• MIPS Webpages (SSC)
• Franks ADASS paper (see SSC web link)
• Karl Gordons paper (see SSC web link)
• No substitute for playing with data!