http://www.emc.noaa.gov/research/osse

1
Observing System Simulation Experiments
The New Nature run International Collaboration
Michiko Masutani
NOAA/NWS/NCEP/EMC
http//www.emc.noaa.gov/research/osse
2
NCEP Michiko Masutani, John S. Woollen, Yucheng
Song, Stephen J. Lord, Zoltan Toth, Russ
Treadon JCSDA John LeMarshal, Jim Yoe, Waymen
Baker, NESDIS Thomas J. Kleespies, Haibing Sun,
SWA G. David Emmitt, Sidney A. Wood, Steven
Greco, Chris OHandley, NASA/GFSC Lars Peter
Riishojgaard, Oreste Reale, Joe Terry, Ron
Errico, Runhua Yang, Juan Juseum, Gail
McConaughy NOAA/ESRLTom Schlatter, Yuanfu Xie,
Steve Weygandt, Gil Compo ECMWFErik
Andesrsson KNMI Gert-Jan Marseille, Ad
Stoffellen Japan JMA, MRI and Earth Simulator
Center
3
Existing data Proposed data DWL, CrIS, ATMS,
UAS, etc
Nature Run
Current observing system
DATA PRESENTATION
OSSE DATA PRESENTATION
OSSE Quality Control (Simulated conventional
data)
Quality Control (Real conventional data)
Real TOVS AIRS etc.
Simulated TOVS AIRS etc.
OSSE DA
DA
GFS
OSSE
NWP forecast
NWP forecast
4
Need for OSSEs

• Quantitativelybased decisions on the design
  implementation of future observing systems
• Evaluate possible future instruments without
  cost of developing, maintaining using observing
  systems.
• There are significant time lags between
  instrument deployment and eventual operational
  NWP use.

5
OSSEs are a very labor intensive project.

• DA (Data Assimilation) system will be prepared
  for the new data
• OSSE helps understanding and formulation of
  observational errors
• Enable data formatting and handling in advance of
  live instrument

DA system will be different when the actual data
become available
If we cannot simulate observation, how could we
assimilate observation?
We need to present levels of confidence of the
results from OSSEs. Comparison of OSSE by various
DA system will be very important.
6
Nature Run Serves as a true atmosphere for
OSSEs Preparation of the Nature Run and
simulation of basic observations consume a
significant amount of resources. If different
NRs are used by various DAs, it is hard to
compare the results.

• Need one good new Nature Run which will be used
  by many OSSEs.
• Share the simulated data to compare the OSSE
  results by various DA systems to gain confidence
  in results.

7
Forecast run is used for the Nature Run
Because the real atmosphere is a chaotic system
governed mainly by conditions at its lower
boundary, it does not matter that the Nature Run
diverges from the real atmosphere. The Nature
Run should be a separate universe, ultimately
independent from but parallel to the real
atmosphere. The Nature Run must have the same
statistical behavior as the real atmosphere in
every aspect relevant to the observing system
under scrutiny. A succession of analyses is a
collection of snapshots of the real atmosphere.
Each analysis marks a discontinuity in model
trajectory. Considering a succession of analyses
as truth seems to be a serious compromise in the
attempt to conduct a clean experiment. I favor
a long, free-running forecast as the basis for
defining truth in an OSSE. -- from Tom
Schlatter
Posted at http//www.emc.ncep.noaa.gov/research/os
se/NR
8
New Nature Run by ECMWF Based on
Recommendations by JCSDA, NCEP, GMAO, GLA, SIVO,
SWA, NESDIS, ESRL
Low resolution Nature Run Spectral resolution
T511 Vertical level L91 3 hourly dump Initial
condition 12Z May 1st, 2005 End at 0Z
Jun 1,2006 Daily SST and ICE (Provided by
NCEP) Model Version cy31r1
High resolution Nature Run for a selected
period T799 resolution, 91 levels, one hourly
dump Get initial conditions from L-NR
1x1 degree 31 level pressure data Potential
temperature level data Selected time series of
1x1 degree are also available Convective
precipitation, Large scale precipitation, MSLP,
Z1000, Z500, U500, V500, T2m,TD2m, U10,V10, HCC,
LCC, MCC, TCC, Sfc Skin Temp More with requests
To be archived in the MARS system on the THORPEX
server at ECMWF Accessed by external
users expveretwu Copies for US is available to
designated users and users known to
ECMWF (Contact Michiko Masutani)
Nature Run home page http//www.emc.ncep.noaa.gov/
research/osse/NR
9
Contacts for the New Nature Run ECMWF Erik
Andersson NCEP Michiko Masutani NASA/GSFC Lars-Pe
ter Riishojgaard(GMAO), Oreste Reale(GLA) Joe
Terry (SIVO) JCSDA John LeMarshall NESDIS
Thomas J. Kleespies SWA Steven Greco ESRL Tom
Schlatter THORPEX Pierre Gauthier(DAOS) David
Person(USA) Zoltan Toth (GIFS) Met Office
Richard Swinbank Meteo France Jean Pailleux KNMI
Gert-Jan Marseille EUMETSAT Jo Schmetz ESA
Eva Oriol JMA Munehiko Yamaguchi, Kozo
Okamoto MRI Tetsuo Nakazawa, Masahiro Hosaka ES
Takeshi Enomoto
Extended international collaboration within
Meteorological community is essential for timely
and reliable OSSEs JCSDA , NCEP, NESDIS,NASA,
ESRL ECMWF, ESA, EUMETSAT THORPEX,
IPO Operational Test Center OTC Joint
THORPEX/JCSDA
Simulation of the data must be done from model
levels and at full resolution.
Pressure level data will be available for
diagnostics and evaluation only limited
isentropic level data will become available. BUFR
format will be used
10
Nature run home page http//www.emc.ncep.noaa.gov/
research/osse/NR
Background Progress Discussion Representativeness
errors Credible OSSE Strategies for simulation of
various observations Evaluation metrics Summary
of some results Diagnostics
11
The results depend on the representativeness
error assigned. We have to assign
representativeness error carefully. The
discussions of representativeness error are
posted at http//www.emc.ncep.noaa.gov/research/o
sse/NR/record/ Errico.reperror.061116.ppt RepE.Jun
06-061116.doc Stoffelen.representation3.061116.pdf
Lorenc,A.C et al 1992, Lorenc.1992.TIDCCR4129.p
df
12
Some initial diagnostics
The SST, ice and Ts fields look OK, with the
expected seasonal variations. The Z 500 also
looks OK. Looking quickly at daily 1000 hPa Z
maps for the Caribbean, I've been able to spot
nine hurricanes between June and November. One
made landfall in Florida (see attached ps-file).
There might be some more hurricanes visible in
the wind field? -- Erik Andersson
13
Tropical Cyclones in The Nature Run August 22-27
14
Cyclone tracks in the Nature Run Thomas Jung,
ECMWF
Annual total cyclone track
15
May
February
November
August
16
Total precipitation By Juan Carlos Jusem.
NASA/GSFC
SON
JJA
MAM
DJF
17
JJA Precipitation anomaly
Nature run
Observed
18
Comparison between the ECMWF T511 Nature Run
against climatology. 20050601-20060531,
expeskb, cycle31r1 Adrian Tompkins, ECMWF
TechMemo 452 Tompkins et al. (2004)
http//www.emc.ncep.noaa.gov/research/osse/NR/ECM
WF_T511_diag/ tm452.pdf Jung et al.  (2005)
TechMemo 471 http//www.emc.ncep.noaa.gov/research
/osse/NR/ECMWF_T511_diag/tm471.pdf Plot files are
also posted at http//www.emc.ncep.noaa.gov/resear
ch/osse/NR/ECMWF_NR_Diag/ECMWF_T511_diag The
description of the data http//www.emc.ncep.noaa.g
ov/research/osse/NR/ECMWF_T511_diag/climplot_READM
E.html
19
Quickscat SFC wind
SSMI 10m wind
- Quikscat does not provide winds in rainy
areas - Shows known bias in the W Pacific. Model
winds are too low in deep convective areas.
20
Total precipitation, against GPCP, SSMI, and
XieArkin
TRMM, NASDA and RSS
- These comparisons confirm the lack of rainfall
over the tropical land masses. - We have an
overestimation of precip over the high-SST
regions in the tropics. - There is a tendency for
deep convection to become locked in with the
highest SSTs, which in the east Pacific results
in a narrow ITCZ. - The TRMM NASDA-3b43 algorithm
is presumed to be the most accurate of the two
TRMM retrieval products.
21
Area averaged precipitation
Tropics
NH midlatitude
SH midlatitude
Convective precipitation Large Scale
precipitation Total precipitation
It takes about one month to settle tropical
precipitation.
22
More diagnostics are being conducted
at NASA/GLA, ESRL/PSD, JMA, NCEP, etc.
23
Some recent results from OSSEs at NCEP
andJCSDA (using T213 Nature Run)
24
Targeted DWL experiments
Combination of two lidar
25
200mb V
(Feb13 - Mar 6 average )
10 Upper Level Adaptive sampling (based on the
difference between first guess and NR, three
minutes of segments are chosen the other 81 min
discarded)
Doubled contour
100 Upper Level
10 Uniform DWL Upper
NonScan DWL
26
Target and 200U
Target in Jet region
Target and 200V
Target in North America and Eurasia associated
with Northerly wind
27
850mb
(Feb13 - Mar 6 average )
10 Upper Level Adaptive sampling
100 Upper Level
Doubled contour
NonScan DWL
10 Uniform DWL Upper
28
Anomaly correlation difference from
control Synoptic scale Meridional wind
(V) 200hpa NH Feb13-Feb28
DWL-Lower is better than DWL-NonScan only With
100 DWL-Lower DWL-NonScan is better than
uniform 10 DWL-Upper Targeted 10 DWL-Upper
performs somewhat better than DWL-NonScan in the
analysis DWL-NonScan performs somewhat better
than Targeted 10 DWL-Upper in 36-48 hour forecast
CTL
29
Reduction of RMSE from NR for V by adding
NonScan lidar to low level scan lidar.
V850
V200
NonScan lidar by itself showed a reasonable
impact but exhibited some negative impact with
data from scanning lidar at lower levels. Note
the experiments are performed using old NCEP SSI
DA system. This problem is expected to be
resolved in the new GSI DA system. We have to
work on a DA system for lidar and new instruments
before the data become available.
V500
30
Do not show the difference
100L 0U 100L100U 100L10U
NODWL NODWL NOTOVS
AC to Nature Run 500hPa height Total scale
NH
SH
noDWL with TOVS
noDWL noTOVS
Z500 presents a very limited story
70
90
72
72
31
Data and model resolution
OSSEs with Uniform Data
More data or a better model?
Fibonacci Grid used in the uniform data coverage
OSSE
40 levels equally-spaced data 100km, 500km,
200km are tested
Skill is presented as Anomaly Correlation The
differences from selected CTL are presented
- Yucheng Song
Time averaged from Feb13-Feb28 12-hour
sampling 200mb U and 200mb T are presented
32
U 200 hPa
Benefit from increasing the number of levels
5
500km Raob T62L64 anal T62L64 fcst
L64 anlfcst
500km Raob T62L64 anal T62L28 fcst
500km RaobT62L28 anal fcst
L64 anl L28 fcst
1000km RaobT170L42 analT62L28 fcst
CTL
1000km RaobT62L28 anal fcst
T 200 hPa
L64 anlfcst
500km obs
T170 L42 model
High density observation give better analysis but
it could cause poor forecast
High density observations give a better analysis
but could cause a poor forecast
Increasing the vertical resolution was important
for high density observations
L64 anl L28 fcst
33
High density observations cannot help forecasts
if the model does not have good
resolution. Increasing vertical resolution in
the analysis is important for high density
observations. We have to work on a DA system
for new instruments before the data become
available. OSSE will be a very useful tool to
prepare the DA system for new instruments.
34
Summary
The current NCEP/JCSDA system has shown that
OSSEs can provide critical information for
assessing observational data impacts.
The results also showed that theoretical
explanations will not be satisfactory when
designing future observing systems.
The new Nature Run has been prepared with
international teamwork ECMWF, NOAA, NASA,
THORPEX EUMETSAT, ESA
35
Summary
Extended international collaboration within the
Meteorological community is essential for timely
and reliable OSSEs to influence decisions.
OSSE and its evaluation will become affordable to
the University and academic communities.
36
END
http//www.emc.ncep.noaa.gov/research/osse