RFI Flagging in the LOFAR Imaging Pipeline

1
RFI Flagging in the LOFAR Imaging Pipeline
David Rafferty and the Flagging Team
2
Some Typical RFI
LBA data (SB64 of L2009_14319) No flagging or
squashing
HBA data (SB48 of L2009_16167) No flagging or
squashing
3
Current Flagging Scheme

• To date, best results achieved using the MAD
  (Median Absolute Deviation) Flagger
• First, run on unsquashed data with a medium
  frequency window and small time window
• Next, run repeatedly on squashed data with a wide
  time window to catch broad-band RFI
• Runs in ½ real time for 20-station data (e.g.,
  1 hour for a 2-hour dataset)

4
LBA Data after MADFlagger
SB57 of L2009_14319
SB72 of L2009_14319
5
HBA Data after MADFlagger
SB38 of L2009_16167
6
NDPPP

• New version of DPPP by Ger van Diepen
• Many improvements and new features, but those
  most relevant to RFI flagging include
• Significantly faster than IDPPP
• Ability to flag on any combination of
  polarizations
• New PreFlagger allows flagging of individual
  channels, baselines, etc.
• Many more flagging strategies are now practical

7
Example NDPPP Run

• 2 passes on unsquashed data using XX and YY only
• Freq. window 101 channels, time window 1
• Freq. window 1, time window 101 time slots
• 1 pass on squashed data using all polarizations
  with wide time window
• Runs in ¼ real time for 20-station data (e.g.,
  30 min on a 2-hour dataset), IDPPP requires gt
  real time

8
Example NDPPP Run on HBA Data
SB50 of L2009_16167
9
Example NDPPP Run on HBA Data
SB50 of L2009_16167
10
Flagger of André Offringa

• Not yet part of DPPP
• Run with command rficonsole ltinput.msgt
• Best strategy appears to work very well
• Runs at 1/2 real time for 20-station data
• Has two main advantages over the MADFlagger
• Uses a more sophisticated fit to visibilities
• Flags neighboring point that dont meet flagging
  criteria on their own but do when summed

11
Summary

• NDPPP allows for many more practical flagging
  schemes these need testing
• Need strategy to deal with broad-band,
  long-duration RFI (i.e., wider than half the
  flagging window)
• Need a thorough characterization of the typical
  RFI seen in LBA and HBA
• Further extend DPPP flagging capabilities