Kevin (Minxue) He

1
An Integrated Uncertainty and Ensemble Data
Assimilation Approach for Improved Operational
Streamflow Predictions

• Kevin (Minxue) He

NOAA/NWS Office of Hydrologic Development
(OHD) Riverside Technology, Inc.
Acknowledgements Haksu Lee (OHD), Yuqiong Liu
(NASA GSFC) Andy Wood and Stacie Bender (CBRFC),
Terri Hogue and Steven Margulis (UCLA)
NOAA/NWS/NCEP/EMC, Camp Springs, MD September
16, 2011
2
Outline

• Introduction
• Hydrologic forecasting in US
• Hydrologic Ensemble Forecast System
• Focus of this presentation
• Methodology
• Integrated unCertainty and Ensemble data
  Assimilation (ICEA)
• Study area and experimental design
• Verification metrics
• Results
• Simulation results and parameter uncertainty
• Prediction results
• Conclusions and ongoing work

2/24
3
Hydrologic Forecasting in US

• Focus streamflow
• Two components
• Forecast model
• Non-snow basins rainfall-runoff model (SAC-SMA)
• Snow-covered basins snow model (SNOW17)
  SAC-SMA
• Forcing numerical weather prediction models
  (NCEP)
• Products

I. Deterministic (4,925 sites, short-term)
II. Ensemble (climatology forcing, seasonal)
3
4
Hydrologic Ensemble Forecast System (HEFS)
Hydrologic Ensemble Forecast System (HEFS)
Atmospheric Ensemble Pre-Processor
GFS/GEFS, CFS/CFSv2, NAEFS, SREF
Data Assimilator
SNOW17 SAC-SMA
Hydrology and Water Resources Models
Hydrologic Ensemble Post-Processor
Hydrology and Water Resources Ensemble Product
Generator
Forecasters
Users
5
Data Assimilator Prototypes of the HEFS

• Deterministic techniques (Variational method
  (VAR))
• 1D-VAR, for routing model
• 2D-VAR, for lumped SAC-SMA model
• 4D-VAR, for distributed SAC-SMA model (4 km
  resolution)
• Ensemble techniques (for SNOW17 SAC-SMA)
• Ensemble Kalman Filter (EnKF)
• Ensemble Kalman Smoother (EnKS)
• Integrated unCertainty and Ensemble data
  Assimilation (ICEA)
• Hybrid deterministic and ensemble technique
• Maximum Likelihood Ensemble Filter (MLEF) for
  SAC-SMA

5
6
Outline

• Introduction
• Hydrologic forecasting in US
• Hydrologic Ensemble Forecast System
• Focus of this presentation
• Methodology
• Integrated unCertainty and Ensemble data
  Assimilation (ICEA)
• Study area, modeling procedure, and scenarios
• Verification metrics
• Results
• Simulation results and parameter uncertainty
• Prediction results
• Conclusions and ongoing work

6/24
7
ICEA

• Model in a systematic view

• ICEA Uncertainty Analysis Ensemble DA
  (ISURFEnKF)
• Part 1 Uncertainty analysis (He, 2010 He et
  al., 2011a, b)
• Integrated Sensitivity and UnceRtainty analysis
  Framework (ISURF)
• Sensitivity analysis screening tool
• Uncertainty analysis Markov Chain Monte Carlo
  technique
• ? Parameter uncertainty info. the optimal
  parameter set

7
8
ICEA

• ICEA Uncertainty Analysis Ensemble DA
  (ISURFEnKF)
• Part 2 EnKF (He, 2010 He et al., 2011c)

I. Basis Bayes theorem
II. EnKF approximates the Bayesian updating
scheme using a Monte Carlo approach
8
9
Study Area
SNOw TELemetry (SNOTEL) network 800 sites,
Natural Resources Conservation Service (NRCS),
daily snow observations (e.g., snow water
equivalent (SWE))
9
10
Experimental Design

• Areal SWE for upper sub-basin
• Non-negative least-squares algorithm
• Study period Training (water year 1979-1984)
  Prediction (1991-1996)
• Modeling procedure

(obs. MAT/P other than FMAT/P used)
Steps to implement ICEA
Step 1 ISURF ?SNOW17/SAC-SMA, upper ? Para. Unc.
(training period) Step 2 EnKF ? SNOW17 model,
upper ? assimilate areal SWE (prediction
period) Step 3 Lower, RFC para. ? flow upper
flow? UH routing? outlet flow
10
11
Experimental Design

• Scenarios
• S1 RFC parameters S2 ISURF optimal parameters
  (S1 S2 deterministic)
• S3 Stand-alone EnKF S4 ICEA
  (S3 S4 ensemble)
• Similarities between S3 and S4 (sensitivity tests
  conducted in He, 2010)
• Precipitation uncertainty
• Temperature uncertainty
• Measurement uncertainty
• Ensemble size 100
• Assimilation frequency every week
• No uncertainty assumed in initial condition
  (start date Oct. 1, no snowpack)
• Difference between S3 and S4
• Parameter uncertainty ranges
• S3 entire feasible para. range S4
  ISURF-derived optimal para. range

11
12
Verification Metrics

• Deterministic metrics
• Correlation (R), Percent Bias, RMSE,
  Nash-Sutcliffe Efficiency (NSE)
• (for S3 S4, ensemble mean is used when
  calculating above metrics)
• Ensemble metrics
• Normalized RMSE Ratio (NRR)
• 95th Percentile Uncertainty Ratio (UR95)

• Measure of ensemble dispersion
• Value 1 (perfect)
• gt 1 (little spread)
• lt 1 (much spread)
• (Anderson, 2002)

Aggregated variability of prediction relative to
observation Range 0-100, perfect value 0
(Hossain and Anagnostou, 2005)
12
13
Outline

• Introduction
• Hydrologic forecasting in US
• Hydrologic Ensemble Forecast System
• Focus of this presentation
• Methodology
• Integrated unCertainty and Ensemble data
  Assimilation (ICEA)
• Study area, modeling procedure, and scenarios
• Verification metrics
• Results
• Simulation results and parameter uncertainty
• Prediction results
• Conclusions and ongoing work

13/24
14
Simulation Results

• Annual statistics of simulated and observed
  streamflow during the training period 1979-1984
  (S1 S2)

• ISURF-derived optimal parameters outperform RFC
  parameters
• ISURF-derived parameter uncertainty information ?
  trustable

14
15
Parameter Uncertainty

• ISURF identifies four sensitive parameters, their
  marginal distributions (in bars) and correlation
  structure (in dots)

15
16
Prediction Results

• Overall performance (entire prediction period)

Bias
RMSE

• RFC prediction can be improved via advanced
  calibration (e.g., ISURF)
• DA has added value ? RFC/advanced calibration
  methods
• ICEA outperforms EnKF

R
NSE
16
17
Prediction Results

• Performance on high flow (gt95th percentile)

RFC ISURF EnKF ICEA
R 0.80 0.85 0.83 0.87
Bias () -19.58 -13.70 -14.91 -10.02
RMSE (m3/s) 65.37 56.47 58.33 49.59
NSE 0.50 0.62 0.60 0.71

• Scatter Plot ICEA mean (best), RFC (worst) DA
  methods provide ensemble info.
• Statistics ICEA (best) RFC (worst) ISURF
  EnKF comparable

17
18
Prediction Results

• EnKF vs. ICEA ensemble statistics (annual)

NRR measure of ensemble dispersion (perfect
value 1 too little
spread when gt1)

• comparable, but not enough spread

UR95 variability relative to observations
(perfect value 0)

• Overall, ICEA has less variability but not in
  1993, 1995, and 1996

18
19
Prediction Results

• EnKF vs. ICEA finer resolution (daily)

I. Selection of the wettest year,1995, for
demonstration
II. EnKF and ICEA flow predictions in this year

• RFC misses peak/recession
• Both ens. capture peak flow high flows spread
  is narrow
• EnKF ens. wide in early melting period, but
  underestimate later ? melting parameter samples

19
20
Prediction Results

• EnKF vs. ICEA performance at various lead times

• Deterministic metrics (a-d) overall, ICEA
  outperforms EnKF in all lead days
• ICEA ensemble less variability (e) all lead
  days comparable dispersion (f)

20
21
Outline

• Introduction
• Hydrologic forecasting in US
• Hydrologic Ensemble Forecast System
• Focus of this presentation
• Methodology
• Integrated unCertainty and Ensemble data
  Assimilation (ICEA)
• Study area, modeling procedure, and scenarios
• Verification metrics
• Results
• Simulation results and parameter uncertainty
• Prediction results
• Conclusions and ongoing work

21/24
22
Conclusions

• Simulation ISURF optimal para. outperform RFC
  para.
• Parameter uncertainty 4 sensitive para. 3
  normal, 1 uniform
• Prediction
• DA methods (EnKF/ICEA) provide improved flow
  predictions (vs. RFC/ISURF) and ensemble
  predictions
• ICEA ensemble mean prediction ? best in overall
  performance high flow prediction in 4
  scenarios better in all lead days vs. EnKF mean
  prediction
• ICEA ensemble predictions generally have less
  variability comparable dispersion vs. EnkF
  ones, both on annual basis and at various lead
  days
• ICEA and EnKF ensembles capture high flows, but
  too narrow

Take Home Message

ICEA has the
potential to supplement the current operational
method in 1) providing improved single-valued
(ens. mean) forecasts 2) meaningful ensemble
forecasts.
22
23
Ongoing and Future Work

• Enhance the experimental prototype (ICEA) by
• Investigating ensemble initialization
  (uncertainty in I.C.)
• Verifying ensembles via other metrics
  (reliability, resolution)
• Considering model structural uncertainty (He et
  al., 2011a)
• Evaluating it against the operational snow
  updating system used at CBRFC across multiple
  watersheds
• Evaluate the enhanced prototype in real-time
  forecasting
• To digest forecasted ensemble forcing (e.g.
  processed GFS/PQPF) with educated perturbations
  of I.C. ? predictions with wider spread

23
24
Thank youQuestions?
Contact Kevin.He_at_noaa.gov

• References
• He, M. (2010) Data assimilation in watershed
  models for improved hydrologic forecasting, Ph.D.
  Dissertation, Civil and Environmental
  Engineering, University of California, Los
  Angeles, 173 pp.
• He, M., Hogue, T. S., Franz, K. J., Margulis,
  S. A., and Vrugt, J. A. (2011a) Corruption of
  parameter behavior and regionalization by model
  and forcing data errors A Bayesian example using
  the SNOW17 model, Water Resour. Res., 47,
  10.1029/2010WR009753.
• He, M., Hogue, T. S., Franz, K. J., Margulis,
  S. A., and Vrugt, J. A. (2011b) Characterizing
  parameter sensitivity and uncertainty for a snow
  model across hydroclimatic regimes, Adv. Water
  Resour., 34, 114-127.
• He, M., Hogue, T. S., Margulis, S. A, and
  Franz, K. J. (2011c) An integrated uncertainty
  and ensemble-based data assimilation approach for
  improved operational streamflow predictions,
  Hydrol. Earth Syst. Sci. Discuss., 8, 7709-7755,
  10.5194/hessd-8-7709-2011.

24
25
Extra Slides
26
ISURF
ISURF a step-wise framework (He, 2010 He et
al., 2011b) Step 1 Generalized sensitivity
analysis (GSA) (Spear and Hornberger,
1980) screening tool ? sensitive
parameters Step 2 Differential Evolution
Adaptive Metropolis (DREAM) (Jasper et al.,
2008) ? parameter uncertainty
27
ISURF Methodology

• GSA
• Identify feasible parameter ranges
• Monte Carlo sampling Latin Hypercube Sampling
  (sample size )
• Behavioral /non-behavioral classification
• Nash-Sutcliffe efficiency (NSE)0.3
  (Garbrecht, 2006) ?
• Bin division and CDF calculation
• - ? bins
• - CDF of NSE for each bin
• Kolmogorov-Smirnov test
• ? KS value
• (Kottegoda and Rosso, 1997)

28
ISURF Methodology
DREAM
Candidate point
Modify proposal
Metropolis acceptance Prob.
(Jasper et al., 2008 He et al., 2011a, b)
29
SNOW17 Model
Key notes 1. Ppt forcing SCFPptPpt forcing
TairgtPXTEMP rain TairltPXTEMP snow 2.
Non-rain melt M N(Tair-MBASE) 3.
Rain-on-snow melt M UADJ(Tair-32)
Ppt precipitation Tair air temperature
30
SAC-SMA Model
RIVA
Upper zone
Lower zone