VARSY progress meeting

1
VARSY progress meeting
Robin Hogan and Nicola Pounder (University of
Reading)
12 April 2013
2
Brief summary of progress

• No plots today
• Full error descriptors now implemented for liquid
  clouds and rain (ice already done)
• Solar radiance forward model code included to
  describe scattering phase function with Legendre
  polynomials but still needs to be coupled to the
  LIDORT radiative transfer model
• Plots today
• Liquid cloud retrievals using multiple scattering
  from single field-of-view lidar Calipso
• Overcoming multiple minima in the cost function
  for liquid cloud
• Possible algorithm speed-up being investigated
  Levenberg-Marquardt minimization rather than
  quasi-Newton, plus GPU computation of Jacobian
  matrix
• Ability to simulate EarthCARE data (including
  Doppler and HSRL) from A-Train retrievals, then
  retrieve from the simulated EarthCARE data

3
Unified retrieval
1. New ray of data define state vector Use
classification to specify variables describing
each species at each gate Ice extinction
coefficient, N0, lidar extinction-to-backscatter
ratio, riming factor Liquid extinction
coefficient and number concentration Rain rain
rate, drop diameter and melting ice Aerosol
extinction coefficient, particle size and lidar
ratio

• Ingredients developed
• Not yet developed

2. Convert state vector to radar-lidar
resolution Often the state vector will contain a
low resolution description of the profile
3. Forward model
6. Iteration method Derive a new state vector
3a. Radar model With surface return and multiple
scattering
3b. Lidar model Including HSRL channels and
multiple scattering
3c. Radiance model Solar IR channels
Not converged
4. Compare to observations Check for convergence
Converged
7. Calculate retrieval error Error covariances
averaging kernel
Proceed to next ray of data
4
Liquid cloud retrieval

• We have found that the multiple scattering signal
  from Calipso can be inverted to get extinction
  profile for optical depth up to at least 30
• Benefits from a constraint on LWC to be no
  steeper than adiabatic
• We can validate with CloudSat PIA, or assimilate
  PIA too
• Example from 1 minute (400 km) of oceanic
  stratocumulus

• Forward modelled backscatter
• Observed backscatter

5
Assimilate only Calipso backscatter

• LWC
• Effective radius
• Optical depth
• CloudSat PIA

6
Assimilate also CloudSat PIA

• LWC
• Effective radius
• Optical depth
• CloudSat PIA

7
Will this work with EarthCARE?

• Simulated retrieval of optical depth for
  idealized adiabatic clouds, using spaceborne
  lidar with varying field of view (FOV)
• For FOV less than around 50 m, there is simply
  too little multiple scattering signal to retrieve
  extinction and optical depth
• Will need to rely more on radar PIA over ocean
  and solar radiances in the day
• Night-time land a problem

FOV gt 55 m (e.g. Calipso)
FOV lt 50 m (e.g. EarthCARE)
8
Why can the first guess matter?

• Consider a cloud with an optical depth of 50
• If the first guess had an optical depth of 1 then
  the simulated molecular scattering below the
  cloud would look a bit like the measured
  multiple scattering
• Algorithm has difficulty getting over hump in
  cost function because increasing optical depth
  first reduces simulated backscatter below cloud
  top (leading to poorer agreement with obs) before
  multiple scattering builds up (leading to better
  agreement)

9
Possible solution

• Consider all possible true optical depths (but
  only triangular profiles so that profiles can be
  described uniquely by optical depth)
• Algorithm will converge provided first guess is
  outside the shaded areas
• Should be able to pre-analyse the profile (e.g.
  by integrating the backscatter with height) to
  tell if we are in the low or high optical depth
  regime, then set the first guess appropriately

10
Potential optimization

• We need to speed-up the retrieval algorithm
• Can we exploit parallel architectures, e.g.
  multicore machines or GPUs?
• Trade-off between minimization schemes
• Quasi-Newton (L-BFGS)
• Uses only the gradient of the cost function,
  which is fast to calculate
• Many iterations required
• Levenberg-Marquardt (LM more stable version of
  Gauss-Newton)
• Uses also the curvature of the cost function
  which is slow to calculate
• But few iterations required, and a little more
  robust (in my experience)
• Currently works for ice and rain, not yet for
  liquid
• Adepts algorithm for computing the Jacobian
  matrix (needed by LM) is potentially
  parallelizable
• m parallel threads, where m is number of
  observations (100)
• At best, the cost of an LM iteration would be the
  same as a quasi-Newton iteration, so LM would be
  much faster overall
• I am currently employing a programmer with GPU
  experience to code up a parallel Jacobian
  algorithm using CUDA (for NVIDIA hardware)

11
Example case

• Levenberg-Marquardt algorithm run on ice and rain
  region
• CloudSat and Calipso observations and forward
  model

12
Convergence comparison

• Quasi-Newton needs around five times more
  iterations on average (depending on convergence
  criterion)

Levenberg-Marquardt
Quasi-Newton
13
Convergence comparison cont.
Levenberg-Marquardt
Quasi-Newton
CloudSat
Calipso
14
Computational cost
Proportional to number of iterations
Computational cost (arbitrary)
Potentially parallelizable

• Levenberg-Marquardt is already competitive but if
  Jacobian can be sped up it would be much faster
  than qausi-Newton
• Further change perform wide-angle multiple
  scattering at half the vertical resolution would
  gain factor 4 speed-up

15

• CloudSat
• EarthCARE CPR Z
• Higher sensitivity
• CPR Z error
• CPR Doppler
• Use Japanese random error
• CPR Doppler error

Unified retrieval of cloud precip then
simulate EarthCARE instruments
16

• Calipso backscatter
• ATLID Mie channel
• Note liquid!
• ATLID Mie error
• Not rigorous!
• ATLID Rayleigh channel
• ATLID Rayleigh error

Unified retrieval of cloud precip then
simulate EarthCARE instruments
Liquid cloud
17
Compare ice retrievals
Extinction Number concentration Extinction
Number concentration

• A-Train retrieval
• Pseudo-EarthCARE retrieval
• Assimilate Doppler and HSRL
• (Some difference due to lidar ratio not being
  carried between retrieval and simulation)

18
Compare liquid clouds and rain

• A-Train
• EarthCARE
• Poor LWC not enough lidar multiple scattering!

Liquid water content Rain rate Liquid
water content Rain rate
19
Remaining algorithm development

• Minimization
• Parallelize Jacobian calculation on GPU and
  compare speed of Levenberg-Marquardt to
  quasi-Newton
• Forward models
• Finish implementation of LIDORT solar radiance
  model
• Ice clouds
• Add riming factor
• Add Baran phase functions where appropriate
• Liquid clouds
• Test impact of solar radiances on retrievals
• Test size retrieval from two solar wavelengths
• Rain
• Test impact of various observations (PIA, radar
  multiple scattering)
• Aerosols
• Implement an aerosol retrieval scheme (contract
  extension)