Assimilation of AIRS

1
Assimilation of AIRS IASI at ECMWF

• Andrew Collard

Acknowledgement to Tony McNally
2
Overview

• Introduction
• Assimilation Configuration
• Channel Selection
• Cloud Detection
• IASI First Guess Departures
• AIRS IASI Forecast Impacts
• Water Vapour
• Data Compression
• Conclusions

3
Introduction
4
Current and Future High-Spectral Resolution
InfraRed Sounders on Polar Orbiters
5
An IASI Spectrum
6
IASI vs HIRS The Thermal InfraRed
7
AIRS vs HIRS Jacobiansin the 15µm CO2 band
100 hPa
HIRS-4
HIRS-5
Selected AIRS Channels 82(blue)-914(yellow
)
HIRS-6
HIRS-7
HIRS-8
1000 hPa
8
HIRS vs IASI Temperature Retrieval Accuracy
9
HIRS vs IASI Response to Important Atmospheric
Structure
IASI
HIRS
Response to a structure the observation of which
would have improved the forecast of the
reintensification of Hurricane Floyd over SW
France and SW England on 12th September 1993.
(Rabier et al., 1996)
10
Assimilation Configuration
11
Current Operational Configurations

• AIRS
• Operational at ECMWF since October 2003
• 324 Channels Received in NRT
• One FOV in Nine
• Up to 155 channels may be assimilated (CO2 and
  H2O bands)
• IASI
• Operational at ECMWF since 12th June 2007
• 8461 Channels Received in NRT
• All FOVS received Only 1-in-4 used
• 366 Channels Routinely Monitored
• Up to 168 channels may be assimilated (CO2 band
  only)

12
Assumed Noise for AIRS IASI Assimilation
AIRS Instrument Noise Assumed Observational
Error Clear Sky Background Departures
Noise or First Guess Departure (K)
Wavelength (µm)
13
Assimilation ConfigurationChannel Selection
14
Why Select Channels?

• The volume of IASI data available is such that we
  do not have the computational resources to
  simulate and assimilate all these data in an
  operational timeframe
• Not all channels are of equal use when
  assimilated into an NWP system
• We choose channels that we wish to monitor (often
  with a view to future use)
• We choose a subset of these channels which we
  actively assimilate

15
Pre-screened channels
16
Selected Channels (2)
17
AIRS 324 vs IASI 366
IASI 366 AIRS 324
18
Comparison of Actively Assimilated Channels (1)
19
Comparison of Actively Assimilated Channels (2)
20
Jacobians of 15µm CO2 Band
Pressure (hPa)
Temperature Jacobian (K/K)
21
Assimilation ConfigurationCloud Detection
22
Cloud detection scheme for Advanced Sounders
A non-linear pattern recognition algorithm is
applied to departures of the observed radiance
spectra from a computed clear-sky background
spectra. This identifies the
characteristic signal of cloud in the data and
allows contaminated channels to be rejected
AIRS channel 226 at 13.5micron (peak about 600hPa)
The large number of AIRS or IASI channels allows
improved measurement of the cloud-top height
compared to HIRS
obs-calc (K)
Vertically ranked channel index
AIRS channel 787 at 11.0 micron (surface sensing
window channel)
unaffected channels assimilated
CLOUD
pressure (hPa)
contaminated channels rejected
temperature jacobian (K)
23
Number of Clear Channels
High Peaking Channels
Window Channels
24
IASI First Guess Departures
25
First Guess Departure Standard Deviations in
15µm CO2 Band
Calculated Std. Dev.
Observed Std. Dev.
26
First Guess Departure Biases in 15µm CO2 Band
Ordered by Wavenumber
Ordered by Jacobian Peak Pressure
27
First Guess Departure Standard Deviations and
Biases in the Longwave Window
Bias
Standard Deviation
28
First-Guess Departure Biases in Water Band
29
First Guess Departure Standard Deviations in
Water Band
Calculated Std. Dev.
Observed Std. Dev.
30
First Guess Departure Standard Deviations in
Shortwave Band
Observed Std. Dev.
Calculated Std. Dev.
31
AIRS IASI Forecast Impacts
32
AIRS Impact at ECMWF
500hPa Geopotential Anomaly Correlation
Northern Hemisphere
Southern Hemisphere
33
IASI Impact on SH Geopot. AC
No IASI With IASI
34
IASI Forecast Scores Again 500hPa Geopot. AC
8th March-5th May 2007
NH
IASI Better
IASI Worse
SH
35
IASI Forecast Scores Again 500hPa Geopot. AC
11th July-26th August 2007
NH
IASI Better
IASI Worse
SH
36
AIRS/IASI Impact at ECMWF in Context N. Hemis.
No satellite experiments
37
Single instrument experiments
Anomaly correlation of 500hPa height for the
Southern Hemisphere (average of 50 cases summer
and winter 2003 verified with OPS analyses)
38
168 channels assimilated 8293 to go
39
Using the IASI SpectrumLongwave CO2 Band
156 of 1st 500 channels are active Adjacent
channels not used because of oversampling Other
channels omitted because of ozone and humidity
contamination
40
Using the IASI SpectrumShortwave CO2 Band
Short wave temperature bands 350 channels Solar
contamination, High Noise, Interfering species
41
Using the IASI SpectrumChannels Primarily
Sensitive to the Surface
2900 Window Channels
42
Using the IASI SpectrumTrace Gases and RT
Challenges
700 channels affected by non-LTE during the day
350 channels in the main ozone band
Many other channels (not shown) affected by O3,
CH4, NO2, CO etc.
43
Using the IASI SpectrumThe 6.3µm Water Band
Water band 3800 channels
44
Water Vapour
45
Use of Water Vapour Channels at ECMWF
(For AIRS)

• We get a small positive impact from using the
  water vapour channels
• We use a cloud detection scheme that uses the
  first guess departures in the water band itself
• The signal from water vapour can mimic cloud
• The resulting clear channels also tend to be
  those where water vapour departures are smallest
• We also assume 2K observation errors (c.f.
  instrument noise is 0.2K).

46
In-band vs Cross-Band Cloud Detection
AIRS 1583 1402cm-1
Cross-Band
In-Band
47
Choosing 10 IASI Water Vapour Channels
Grey channels are the 120 H2O channels
distributed via the GTS
48
Fit to other observations 10 IASI Water Channels
Best value at 1.5K
Normalised to unity here
49
Fit to other observations Other Platforms
Black is expt. with IASI humidity channels Red
is Control
NOAA-17 HIRS S.Hemis.
EOS-Aqua AIRS S.Hemis.
The addition of the IASI water band improves the
analysis fit to HIRS on NOAA-17 and increases
usage of AIRS data.
50
Fit to other observations Sondes
The addition of the IASI water band improves the
analysis fit to radiosondes
Spec. Hum. N.Hemis. Sondes
Spec. Hum. S.Hemis. Sondes
Rel. Hum. S.Hemis. Sondes
Black is expt. with IASI humidity channels Red
is Control
51
RH500 Forecast Impact Root Mean Square Error
verified vs Operational Analysis
1st-23rd August 2007
N.Hemis.
Expt Better
Cntrl Better
S.Hemis.
52
RH500 Forecast Impact RMS Error verified vs
Experiments Analysis
CY32R3 Control is Analysis of Experiment with H2O
channels 1st Aug.-9th Sept. 2007
N.Hemis.
Expt Better
Cntrl Better
S.Hemis.
53
Operational Implementation of IASI Humidity
Channels

• To be implemented in next operational upgrade
  (CY35R2)
• 10 IASI channels with 1.5K assumed error
• 7 AIRS channels with 1.5K error for consistency

54
Water Discussion

• We have to greatly inflate water vapour errors to
  avoid degrading the model
• This is because of the large number of channels
  with error correlations between them (including
  bias)
• By assuming greatly increasing the water vapour
  observation errors we are negating the influence
  of inter-channel differences that allow us
  greater vertical resolution
• Can we do a better job?

55
Data Compression
56
Data Compression

• Advanced IR sounder radiances contain a lot of
  information (30pieces)
• but there are two orders of magnitude more
  channels.
• Hence there is a large amount of redundancy
• How can we use these data more efficiently?

57
Why is data compression important?

• Very large data volumes need to be communicated
  in near-real time (e.g., EUMETSAT to NWP centres)
• Simulation of spectra (needed for assimilation)
  is costly
• Assimilation is costly
• Data storage

58
Spectral data compression with PCA
The complete AIRS spectrum can be compressed
using a truncated principal component analysis
(e.g. 200PCAs v 2300 rads)
Leading eigenvectors (200,say) of covariance of
spectra from (large) training set

• To use PCs in assimilation requires an efficient
  RT model to calculate PCs directly
• PCs are more difficult to interpret physically
  than radiances

Mean spectrum
Original Spectrum
N.B. This is usually performed in
noise-normalised radiance space
Coefficients
Principal Component Analysis
This allows data to be transported efficiently
59
Loop of Jacobians of PCs
60
Spectral data compression and de-noising
The complete AIRS spectrum can be compressed
using a truncated principal component analysis
(e.g. 200PCAs v 2300 rads)
Leading eigenvectors (200,say) of covariance of
spectra from (large) training set
Reconstructed spectrum
Mean spectrum
Original Spectrum
N.B. This is usually performed in
noise-normalised radiance space
Coefficients
Each reconstructed channel is a linear
combination of all the original channels and the
data is significantly de-noised.
If N PCs are used all the information is
contained in N reconstructed channels
(theoretically)
61
AIRS Reconstructed Radiances

• Data are supplied in near-real time by
  NOAA/NESDIS in the same format as the real
  radiances.
• The same channels are supplied, except some
  popping channels are missing
• Based on 200 PCs
• QC Flag supplied

62
First Guess Departures for AIRS are Reduced
Instrument noise is the main contributor to FG
departure in the 15µm CO2 band only
1.0
First Guess Departure Std. Dev. (K)
Original
Reconstructed
0
14
4
Wavelength (µm)
63
A look at Reconstructed Radiances Errors
Reconstructed Radiances
Instrument noise is reduced (std. dev. Is
approximately halved) but has become correlated.
15µm
14.5µm
14µm
13µm
Original Radiances
Instrument noise is dominant and diagonal.
Correlated noise is from background error
Covariances of background departures for clear
observations in 15µm CO2 band
64
Improvements in Cloud Detection
Reconstructed Radiances

• ECMWF Scheme
• Ranks Channels Height
• Applies low-pass filter
• Tests for non-zero gradient

First Guess Departure (K)
Original
Last Clear 70
First Guess Departure (K)
Height-ranked Channel Index
lt 2 of Channels are flagged differently for RR
vs Normal radiances
Last Clear 75
Height-ranked Channel Index
65
Reconstructed Radiances Result in Different
Increments
Mean Temperature Increments over
Antarctic Plateau March-April 2005
Reconstructed Radiances Control
66
and can improve the analysis
Damping of stratospheric Oscillations before use
of GPSRO and AMSU-A Ch 14 fix
Control Reconstructed Radiances
67
Forecast Impact of Reconstructed Radiances
Essentially Neutral
68
Future Dissemination and Use of Advanced Sounder
Radiances

• Is it acceptable to receive radiances in the form
  of Principal Component Amplitudes?
• While forecast impact on changing to
  reconstructed radiances has been neutral, the
  detailed increments and forecast changes are not
  identical
• Would we lose information we could use?
• Can we make better use of the observations?
• RR still seems more promising than direct use of
  PC scores as Jacobians of PCs are far less
  localised.
• Exploitation of information in IASI spectrum is
  not currently limited by the number of channels
  that can be handled by the minimisation
• See water example
• Good coverage of LW temperature channels

69
Conclusions

• AIRS and IASI have been operational at ECMWF
  since October 2003 and June 2007 respectively
• IASI and AIRS have demonstrated positive impacts
  on the ECMWF NWP model and form an important part
  of the assimilation system
• We are in the process of better understanding our
  use of the humidity channels. IASI humidity
  channels will be assimilated with the next
  operational upgrade.
• Advanced IR Sounder radiances may be efficiently
  compressed through Principal Component Analysis.
  This may be used for NRT dissemination. The
  advantages of this datatype in NWP remain to be
  demonstrated.

70
Questions?
71
IASI Spectral Correlation
Correlations from forecast model
IASI Channel Number
Covariance of first-guess departures 1st 200
channels
Nearly diagonal instrument noise
IASI Channel Number
72
IASI Spectral Correlation
Expected correlation structure from apodisation
of IASI spectrum
IASI Channel Number
IASI Channel Number
73
Sensitive areas and cloud cover
Location of sensitive regions Summer-2001 (no
clouds)
monthly mean high cloud cover
sensitivity surviving high cloud cover
monthly mean low cloud cover
sensitivity surviving low cloud cover
From McNally (2002) QJRMS 128
74
O-A Stats for NOAA-16 AMSU-B
Larger Assumed AIRS H2O Errors Better Fit to
AMSU-B
75
Choosing 84 IASI H2O Channels
Chosen from the the 366 Channels Black channels
have stratospheric contribution
76
Fit to other observations84 IASI Water Channels
Best value at 4K
Normalised to unity here
MetOp HIRS-12 Tropics
77
Cloud
78
Cloud

• Cloud in the field of view can greatly affect our
  ability to use IR radiances
• Studies have show that the most important areas
  to measure for accurate forecasts often have
  cloud
• We need to identify strategies to deal with cloud

79
Dealing with Cloud

• Use only clear fields of view
• Low (5) yield
• Use only channels unaffected by cloud
• Low yield in lower tropospheric channels
• Cloud clearing
• Simulate a clear observation by using multiple
  fields of view and assume that only cloud
  fraction changes between them
• Simultaneous analysis of cloud optical properties
• These observations are rich in cloud information

Used now operationally
Under Investigation
80
Cloud Cleared Radiances
Cloud Cleared Radiances derive a single clear
spectrum from an array of partially cloudy
fields-of-view (9 in the case of AIRS) Assumes
the cloud height in each FOV is identical and
only cloud fraction varies between the
FOVs. Initialised with regression from AMSU-A
radiances Can calculate a noise amplification
factor which is the basis of the QC
flag Supplied by NOAA/NESDIS in near-real time
but with poorer timeliness than conventional
radiances
81
Cloud-Cleared Radiances
AIRS Ch. 221. QC Flag Used. No Cloud Detection
82
Cloud Cleared Radiances
AIRS Ch. 221. QC Flag Used. With Cloud Detection
Still have positive tail
83
Cloud Cleared Radiance Current Status

• The geographical location of outliers around
  regions which are flagged as of poor quality
  indicates that the supplied QC flag may need
  revising.
• Referred back to data suppliers before looking at
  revising cloud detection
• Recent degradation of AMSU-A Ch 4 may affect
  quality

84
Simultaneous Analysis of Cloud Properties(Tony
McNally)

• Derive a cloud-top pressure (CTP) and cloud
  fraction from observed radiances with a 2-D
  least-squares fit in the screening run
• For overcast FOVs use all channels (that are
  currently used for clear sky case)
• This has the advantage of reducing the degrees of
  freedom
• For other FOVs revert to operational
  cloud-detection scheme to identify clear channels
• Assimilate these radiances with CTP as a sink
  variable

85
Using data in cloudy areas
Tony McNally
Clear data coverage of mid/lower tropospheric
sounding radiances IASI 434 (METOP-A) AIRS 355
(AQUA) HIRS 7 (NOAA-17 / METOP-A) Colour
indicates first guess departure
Additional overcast locations where cloudy
radiance analysis fills gaps due to cloud
detection rejections IASI 434 (METOP-A) AIRS
355 (AQUA) HIRS 7 (NOAA-17 / METOP-A) First
guess departures similar to clear data after QC
of complex clouds
86
Temperature increments at the cloud top
Tony McNally
Cell of very high overcast clouds off the coast
of PNG
Temperature increments (IASI)
All channels collapse to near delta-functions at
the cloud top giving very high vertical
resolution temperature increments just above the
diagnosed cloud
blueops redops cloudy IR
87
Temperature increments
Tony McNally
Monthly averaged RMS temperature increments at
500hPa CTRL minus EXPT
Possibly some reduced increments at isolated
stations
88
Cloudy Retrievals Current Status

• Technically works
• The restriction to overcast and heavy QC
  currently yields lt 10 extra data
• The small amount of additional data to not
  influence the bulk characteristics of the
  analysis or departure statistics
• The extra cloud top increments look reasonable at
  overcast locations have very high vertical
  resolution
• Forecast performance essentially neutral
• Will continue to search for individual cases of
  forecast impact using singular vectors
• Next steps use the data in more complex cloud
  conditions / Marine Stratocumulus / use NWP cloud
  information ?

Tony McNally
89
Land Surface
90
Land and Sea Ice

• AIRS data is used over land for channels not
  sensitive to the surface
• Addition of similar IASI channels had neutral
  impact and was not pursued
• We do use these data over sea ice treating the
  surface as sea. Not ideal but
• Removing channels results in negative impact
• Modelling surface emissivity results in negative
  impact
• In the future we aim to implement an emissivity
  analysis using leading EOFs of the emissivity
  spectrum
• the interaction with the cloud detection scheme
  will be challenging