Basic tools for ocean modeldata comparison

1
Basic tools for ocean model-data comparison
EurOceans-CarboOcean Summer School, Kiel, 14
September 2007
James Orr
Marine Environmental Laboratories
International Atomic Energy Agency 4 Quai
Antoine 1er, Monaco

http//www.ipsl.jussieu.fr/OCMIP
2
It is a capital mistake to theorize before one
has data. Insensibly one begins to twist facts
to suit theories instead of theories to suit
facts. Sir Arthur Conan Doyle, The Sign of Four,
A Scandal in Bohemia
3
The sciences do not try to explain, they hardly
even try to interpret, they mainly make models.
By a model is meant a mathematical construct
which, with the addition of certain verbal
interpretations, describes observed phenomena.
The justification of such a mathematical
construct is solely and precisely that it is
expected to work. Johann Von Neumann
(1903-1957)
4
OCMIP-2 Group

• AWI (Bremerhaven, Germany) R. Schlitzer, M.-F.
  Weirig
• CSIRO (Hobart, Australia) R. Matear
• IGCR/CCSR (Tokyo, Japan) Y. Yamanaka, A. Ishida
• IPSL (LSCE, LODyC, Paris, France) J. Orr, P.
  Monfray, O. Aumont, J.-Cl.Dutay, P.
  Brockmann
• LLNL (Livermore, CA, USA) K. Caldeira, M.
  Wickett
• MIT (Boston, USA) M. Follows, J. Marshall
• MPIM (Max Planck Institut fuer Meteorologie
  Hamburg, Germany) E. Maier-Reimer
• NCAR (Boulder, CO, USA) S. Doney, K. Lindsay, M.
  Hecht
• NERSC (Bergen, Norway) H. Drange, Y. Gao
• PIUB (Bern, Switzerland) F. Joos, K. Plattner
• PRINCEton (Princeton, USA) J. Sarmiento, A.
  Gnanadesikan, R. Slater, R. Key
• SOC (Southampton Oceanography Centre/ Hadley
  Center, UK) I. Totterdell, A. Yool
• UL (University of Liege/University Catholique de
  Louvain, Belgium)
• A. Mouchet, E. Deleersnijder, J.-M. Campin
• PMEL/NOAA (Seattle, USA) J. Bullister, C. Sabine
• PSU (Penn. State, USA) R. Najjar, F. Louanchi
• UCLA (Los Angeles, USA) N. Gruber, X. Jin

5
OCMIP-2 models differ
Resolution
Seasonality
Boundaryconditions
Sub-grid mixing
Mixed Layer
Sea-ice Model
Offline/Online
6
How Good is a Model?

• Relative to data
• Relative to other models
• Skill assessment depends on
• Our Objectives (e.g., mean state vs. variability)
• Our Vision
• Rose colored glasses
• Dark Sunglasses
• Clear glasses?
• Local, Qualitative Analysis
• Global, Quantitative Analysis

7
OCMIP-2 Simulations

• Tracers
• CFC-11 and CFC-12
• Natural C-14 and Bomb C-14
• He-3 and He-4
• Carbon
• Preindustrial
• Abiotic
• Common Biogeochemistry (?CO2, Alk, PO4, O2, DOM)
• Preindustrial to Present
• Future (two IPCC scenarios IS92a, S650)
• Sequestration (7 sites, 3 depths, 2 scenarios)

3-
8
Annual Mean Sea-Air CO2 Flux in 1995 (mol m-2
yr-1)
9
Basin Zonal IntegralsAnnual Mean Sea-Air CO2
Flux in 1995 (Pg C yr-1 deg-1)
10
GlobalSeasonalZonal Integral Sea-Air CO2 Flux
in 1995 (Pg C yr-1 deg-1)
11
Some impressions

• Qualitative picture general agreement of the
  models with the observed air-sea CO2 flux in
  1995
• Are we missing something?
• Difficult to make quantitative statements

12
Some Summary Statistics
A Useful Diagram
Key relationship

• Standard deviations
• reference
• model
• Correlation Coefficient R
• Centered Pattern RMS error
• Overall Bias

Law of Cosines
Taylor, K.E., Summarizing multiple aspects of
model performance in a single diagram, J.
Geophys. Res., 106, D7, 7183-7192, 2001
13
Basic Taylor diagram
14
Case 1 single-point time series (interannual
variability of the air-sea CO2 flux at BATS)
15
HOT Sea-air CO2 flux anomalies (12-mo running
mean)
Raynaud et al., 2006 (Ocean Science, 2, 43-60)
16
BATS Sea-air CO2 flux anomalies (12-mo running
mean)
Raynaud et al., 2006 (Ocean Science, 2, 43-60)

• Why general underprediction?
• Data errors?
• Low horizontal resolution (near west. boundary)
• Weak Forcing (atm. reanalysis)
• from above
• affects lateral lags

17
NCEP underestimates real wind speed variability
North Atlantic

• Interannual var. in wind speed
• ?NCEP lt (1/3) ?ERA40
• NCEP wind speeds lower than WOCE ship track winds
• NCEP atm. transport variability only half that
  observed (Waliser et al., 1999)

Raynaud et al. (2006, Ocean Science)
Smith et al. (2001, J. Climate)
18
Case 2 gridded maps (annual means)
19
Restored tracers usually Rgt0.9, but low ?
20
Fully prognostic tracers less agreement
21
Case 3 time series of 2-D maps (annual cycle of
modern air-sea CO2 fluxes)
22
Annual Mean Sea-Air CO2 Flux in 1995 (mol m-2
yr-1)
23
Taylor Diagram Sea-Air CO2 Flux (1995
Annual Mean, Global Map)
Taylor, K.E., J. Geophys. Res., 106, D7,
7183-7192, 2001
24
Taylor Diagram Sea-Air CO2 Flux (1995
Full Global Space-Time Distribution)
25
Annual cycle of flux is anticorrelated

• Air-sea flux of CO2 (1995)
• ? Zonal mean OK
• ? Seasonal Cycle Bad
• R of monthly zonal mean R in the Atlantic
• ? Anti-correlation
• high latitudes
• tropics

26
Std. Dev. for Monthly Zonal Integral CO2 Flux
27
Annual cycle of ?pCO2 and PO43-at BATS (31?N,
64?W) NABE (47?N, 18?W)
NABE (prognostic)
NABE (diagnostic)
BATS (diagnostic)
?pCO2
PO43-

• Need prognostic ocean BGC model
• (no PO43- restoring)

28
Components of seasonal ?pCO2 variation at BATS
(31?N, 64?W)
Solubility
Biology
Anthropogenic
Total
Preindustrial
29
Components seasonal ?pCO2 variation at NABE
(47?N, 18?W)
Solubility
Biology
Preindustrial
Anthropogenic
Total
30
Summary Modern Air-Sea CO2 Flux

• Zonal Annual Variability models and data agree
• Seasonal Variability
• Subtropics
• large variability
• models and data agree (minor role of biology)
• Tropics
• small variability
• model-data disagreement (anticorrelation)
• High latitudes (simulated variability gt
  observed)
• large variability
• model-data disagreement (anticorrelation)
• Possible causes
• Imposed atmospheric pCO2 but seasonal cycle is lt
  20 ppm
• Circulation model (model diversity argues
  problem despite model diversity)
• Common diagnostic BGC model spring bloom too
  weak

31
Should we use a OCMIP-2 like diagnostic
nutrient-restoring model?

• not suited to study seasonal variability
• nor to study interannual variability
• nor to study future climate effects on ocean C
  cycle
• but irrelevant for modern anthropogenic CO2 uptake

32
Case 4 discrete or sparse data (C-14 DMS)
33
Taylor Plot for Deep ?14C (below 1000 m)
Models subsampled at data points (weighted
equally)
34
Natural ?14C (West Atlantic, GEOSECS Section)
Data
Some models under-predict ?14C
Some models over-predict ?14C
35
Pacific OceanWOCE P16 ?14C
Data
Some models over-predict ?14C
Some models under-predict ?14C
36
Surface DMS comparison
Gridded
Sparse
Belviso et al., 2004 (Global Biogeochem. Cycles,
18, GB3013)
37
Conclusions

• Taylor Diagram graphical evaluation of 5 global
  summary statistics (sdata, smodel, r, RMS, Bias)
• Offers quantitative vs. qualitative statements
• Provides quick, overall vision (global roadmap)
• Resolves complex patterns (discrepancies) in
  space time
• but it is not mechanistic
• Motivates further evaluation (both qualitative
  quantitative)
• Remains largely underexploited, particularly in
  our community
• ? Software already exists to make Taylor Diagrams
  (Python-CDAT, Ferret, etc.)
• see your local climate modelers, Google,
• future links on this summer-school web page?
• Other model-data comparison tools are being
  developed (e.g., portrait plots) see PCMDI web
  page literature

Taylor, K. E., J. Geophys. Res., 106, D7,
7183-7192, 2001
38
(No Transcript)
39
Why more than 1 model?? statistics reality
check (e.g., CFC-11)
40
Writing readable scientific papers
EurOceans-CarboOcean Summer School, Kiel, 14
September 2007
James Orr
Marine Environmental Laboratories
International Atomic Energy Agency 4 Quai
Antoine 1er, Monaco

41
Why should I care?

• Research doesnt count until it is published
• Publish or Perish
• Quantity is valued by some
• Quality is valued by all

42
References

• W. Strunk Jr. and E. B. White, The Elements of
  Style, Longman Publishers for the 4th edition,
  1999 (the 1st edition was published in 1959!) 
• J. M. Williams, Style The Basics of Clarity and
  Grace, 6th edition, ISBN 0-313-33040-9 Greenwood
  Press, 2003, 320 pp.
• R. A. Day and B. Gastel, How to Write and Publish
  a Scientific Paper, 6th edition, ISBN
  0-313-33040-9 Greenwood Press, 2006, 320 pp.
• W. C. Booth, G. C. Colomb, and J. M. Williams,
  The Craft of Research,2nd edition, University of
  Chicago Press, ISBN 0-226-06568-05, 325 pp.