DSST ShortTerm Ensemble Planning STEP Call

1
DSST Short-Term Ensemble Planning (STEP) Call 2

• RFC Presentations and Discussion

2
Agenda

• Introduction (DJ)
• RFC presentations and Q/A
• AB
• AP
• CB
• CN
• MA
• MB
• NC
• NE
• NW
• Comments by other RFCs
• Discussion (All)
• Towards developing (near-) consensus operations
  concept and identifying overarching needs and
  issues (service, operations and science)
• What next? (DJ)

3
ABRFC
4
Operations Concept

• I see the ABRFC using short term ensembles (STE)
  as the "official" way to issue almost all of our
  forecasts. That includes our 27 daily forecast,
  as well as our forecasts we issue for the other
  200 flood forecast points we have.  Only our
  long term water supply forecasts would be
  different. If I had our way, I would have
  implemented the STE as the "AWIPS" requirement
  fulfillment long ago. 98 of our users do not
  want long term ESP forecasts!
• I picture that we would issue the forecasts
  mainly in a graphical method, much like the
  examples that this office has provided (see
  below).   Instead of issuing just deterministic
  forecasts like we do now, I see STE as a
  replacement/addition to this. 

5
Operations Concept (cont.)

• Another use of short-term ensemble (STE)
  forecasts is by the WFOs as a basis for
  triggering a river flood watch
• If we got the procedures efficient enough, we
  could issue our STE forecasts routinely for all
  of our forecast points, not just for flood
  situations. This way, with experience and
  possibly local WFO procedures, the WFOs could use
  the STE forecasts as the main part of the
  decision process in determining when to issue
  flood watches and outlooks.

6
Needs and Issues

• One thing that NEEDS be to accomplished is for us
  to determine or define a relationship between
  probabilistic forecasts and a single
  deterministic forecast. I think all STE forecasts
  should also include the offices best guess of a
  deterministic forecast, and this deterministic
  forecast should be located near the 50
  probability line. We have to define how
  deterministic forecasts relate to probabilistic
  forecasts before we can go any further.
• Many of our users just want one forecast, i.e.
  the deterministic forecast, while other more
  sophisticated users will enjoy the probabilistic
  forecast. Issuing a STE with both data would be
  the best bet I think!
• I also think that the STE info could be presented
  in a text format, but the exact format could be
  determined later.  

7
Needs and Issues (cont.)

• Operationally, I see MANY significant things that
  need to be accomplished. Once you get the science
  down, we need software that will produce the
  graphics/text output of this data in a user
  friendly manner (i.e. xsets/hydrograph creation
  software).
• We need training for those producing and using
  these forecasts.
• We need a solid verification set of data proving
  that the science behind STE is reliable and
  sound. I have yet to see any verification
  (including that which I have done) that convinces
  me that the science behind the STE is producing
  reliable and accurate short term forecasts.
• As far as the DA goes, I still am not convinced
  it works. The little I have heard from WGRFC
  users is that they were not satisfied with it. I
  think we need some extensive verification of it,
  and HICS/DOHS/RFCS need to be in the decision to
  buy off on this. At this point, I have not seen
  any convincing evidence that it works.

8
Needs and Issues (cont.)

• Overall, I am excited about STE, but still am not
  convinced that the science behind our current
  configuration works.
• I do however foresee a future where the vast
  majority of forecasts issued from the ABRFC are
  STE, which will allow for a great advance in the
  hydrologic program.

9
APRFC
10
APRFC HAS ConOps

• Ability to select model(s) of choice
• Ability to evaluate 6-hour QPF for a set of
  stations
• Ability to apply post processing to get pcpn
  range for each station
• Ability to review/edit pcpn ranges
• Ability to apply Mountain Mapper functionality
  to get MAPs
• Ability to review/edit MAP value ranges
• Ability to repeat process for temperatures

11
APRFC Hydro ConOps

• Ability to review forecast and range of MAP and
  MAT values
• Ability to run IFP and see resulting hydrographs
  for range of inputs
• Ability to adjust MAP/MAT relationships
• Ability to run post processing to get most likely
  and range of hydrographs

12
APRFC Summary Comments

• We think that short term ensembles have equal or
  greater value than long term ensembles for our
  users
• We have concerns about
• the ability to quantify the uncertainty in QPF
  given its significant spatial and temporal
  variability and individual forecaster bias
• the ability to quantify the uncertainty of the
  interrelated QPF and QTF variables and the
  resulting impact on the ensemble of hydrographs
• the apparent conflicts of wanting control over
  the output (tweaking the results) vs. maintaining
  unbiased statistical ensembles

13
CBRFC
14
CBRFC Short-term Ensemble
Opportunity We can improve our short to medium
range esp forecasts by using numerical model
predictions in place of historical data.
15
But... We can not directly use output from a
weather model, even if it is in ensemble form.
Our conceptual model does not necessarily want to
see reality. It wants to see its own twisted
version of reality. We get that twisted version
by determining a relationship between what the
weather model predicts and what our river model
has been calibrated with. Otherwise known as
downscaling.
16
Project Area 27 Segments Above Cameo, Colorado
River
All recently recalibrated and set up For ESP.
17
Downscaling

• MRF Variables
• 2m air temp
• Precipitation
• 700mb Relative Humidity
• Sea Level Pressure
• 10m Vector Wind
• Total Column Precipitable Water

• Basin Scale Variables
• Mean Areal
• Temperature
• Mean Areal
• Precipitation

18
In order to downscale we need several years, if
not decades of forecast data. This requires a
re-forecast process of the weather model. In
addition, for operational use, we need that
weather model output to stay consistent with the
relationships we have determined. If it is
changed to use better physics, for example, we
would then need a re-forecast data set and
determine new downscaling equations.
19
Downscaling Results
MRF is colder than normal in this case.
20
Input into ESP
MRF derived MAT/MAPs are attached to historical
years (ensembles) and fed to ESP.
21
ESP peak flow
Smaller peaks because MRF is colder for first 14
days causes less melt.
22

• Operational challenges
• Scale it may be too large for precipitation.
  In general, smaller scale should be better
• Availability we need commitment to use this
  operationally
• Re-forecast process any changes in model
  physics or scale requires a re-forecast and new
  downscaling equations. The good news is the
  weather community is starting to see the value in
  statistical correction.
• Presentation we should keep these forecasts
  distinct which will require new data storage and
  handling techniques

23
CNRFC
24
CNRFC comments on short term ensemble

• Need customer buy-in, So how good are you guys
  anyway?
• 50 or median.need a measure of reliability that
  is easily understood by users
• 90 10 bounds.is the 10 exceedance number
  really exceeded only 10 of the time?
• A reasonable range of expected values should be
  very useful to the emergency folks
• Need to adequately explain how uncertainty
  increases as you move into the future
• Conditional uncertainty.Some storms (strong cold
  front crossing the state) are much easier to
  forecast that others (cut-off low just off the
  coast) We should be able to modulate the
  uncertainty accordingly.

25
A Vision for Operational Hydrologic Short-Term
Ensemble Forecasting - Rob Hartman (AHPS Theme
Team, June 2005)

• For many years, NWS customers have benefited from
  probabilistic long-range water supply forecasts
  in the Western U.S. The potential benefits of
  accurate short-term probabilistic flood forecasts
  are very significant. This becomes obvious when
  one considers the cost of local emergency
  management activities.
• For several years now, OHD has been working on
  short-term ensemble prototypes. These efforts
  have been concentrated on developing ensemble
  inputs through model downscaling or simulation
  based on the joint distribution of forecasts and
  observations. Additional activities such as post
  processing and data assimilation (DA) have been
  identified. To date, however, the operational
  environment for short-term ensemble based
  probabilistic forecasts has not been described.
• The time-line for AHPS implementation of
  short-term probabilistic forecasts is such that
  the forecasting environment will certainly still
  involve OFS and the IFP. As such, operational
  ensemble forecasts must function in this
  environment. This will require several changes
  to the ESP and OFS architecture. Here is a
  typical scenario

26

• The RFC is experiencing moderate flooding in
  several watersheds. A forecast update is due out
  at 03Z. Data come in after 00Z for the second
  period of the day, the 5-day QPF is updated by
  the HAS function. Input data are QCd and the
  forecaster starts his/her IFP run to update
  guidance.
• The first segment is a flood forecast point. The
  output includes the single-value forecast as well
  as shaded regions that depict the probabilistic
  forecast with user definable regions (10 EP ,
  25 EP, Ensemble Mean, 75 EP, 90 EP, etc). The
  output indicates that there is a 40 chance that
  the river will rise to 3 feet over flood stage by
  noon tomorrow (mouse tracker interpolation).
  Three feet over flood stage is a critical stage
  for local mitigation. But wait, the simulation
  appears to be a bit screwy. Upon examination,
  the forecaster sees that bad precipitation data
  made it through the QC process. The MAP for the
  18-00Z period needs to be increased by 50. A
  MOD is made. The forecaster reruns the segment.
  The ensembles are regenerated and included in the
  display as before. The single-value forecast and
  probabilities shift slightly. The guidance looks
  reasonable. The phone rings. Its the WFO and
  their local EM needs a forecast update right now.
  The forecaster selects Issue guidance from the
  pull-down menu and the system initiates the
  process that generates and issues the
  single-value and probabilistic guidance for this
  location.

27

• This scenario identifies several issues that are
  not currently supported with OFS and ESP. These
  include
• 1. The notion of carryover must to be re-examined
  so that ESP can be run interactively from any
  point in time and for individual components of a
  forecast group. Ensemble generation must become
  interactive rather than a batch process.
  Performance must support interactive use (rerun
  small pieces in a few seconds).
• 2. The ESP process must thoroughly re-examine the
  notion of MODs and their impacts on short-term
  ensembles. Some believe that DA and automatic
  state updating is the only solution to avoid
  MODs. In the midst of forecasting, this is
  unrealistic. There will always be times when a
  forecaster needs to drive the model to the
  appropriate outcome. Thats why we have
  forecasters.
• 3. Statistical post-processing techniques must to
  be fast and interactive.
• 4. Visualization tools must to be developed
  within the IFP framework to support ensemble and
  probabilistic information.
• 5. Ensemble and probabilistic information must be
  managed to facilitate the generation of products
  and guidance.
• 6. OFS does not write information back to the
  processed DB until the segment is exited. This
  prevents product and information generation while
  looking at the IFP display.
• Without doubt, well find lots of other issues as
  we attempt operational implementation of
  short-term probabilistic forecasts. As such, it
  is important to being addressing the issues and
  developing an operational prototype.

28
MARFC
29
Shorter Term Probabilistic Forecasts

• 7-day probabilistic river forecasts
• PQPF/PQTF
• Demonstration of short-term approach
• Juniata Schuylkill Basins
• 18 points issued daily

30
7-Day Probabilistic River Forecasts

• Current Basin Conditions
• Short term probabilistic precipitation and
  temperatures (PQPF/PQTF)
• 48-hour PQPF merged with 5 days of climo
• QPF scenarios based on comparison of historic
  forecast and observed MAPs
• 3 graphics generated daily for 18 basins in
  PA-Juniata (CTP) and Schuylkill (PHI) Basins

31
WFO CTP AHPS Page
32
Frankstown Br. Juniata River at Williamsburg, PA
33
Difficult to Find on AHPS Web Page
34
7-Day Expected Value Plot
35
7-Day PQPF Traces
36
Lessons Learned

• 7-day PQPF/PQTF forecasts are good contingency
  forecasts
• Describe a range of outcomes that help address
  HSA questionsWhat if..
• Wide range of potential river responses is not
  always pleasingdepicts difficulty in forecasting
  precipitation

37
Lessons Learned

• Only limited use by WFOs and cooperators (much
  less than 30-day ESP forecasts)
• Difficult for non-technical audience
• Software runs well and can generate graphics in a
  hands-off mode
• Once rain is on the ground we have considerable
  modeling uncertainties not incorporated in this
  method

38
Lessons Learned

• Await results of OHD verification work to assess
  validity of products
• MARFC would like to generate probabilistic
  forecasts beyond day 2 based on more than
  climatology
• Potential collaboration with WFO CTP and SREF
  ensembles

39
MBRFC
40
Define User Requirements

• Have user create own on the fly if capable
• Graphics within hydrograph plot if adequate
  description provided

41
Vision of Operations

• Balance rivers without ensemble forecasts
• Switch on ensemble forecasting and look at rivers
  again
• Data assimilation used as guides to mods
• Displays of historical precip and streamflow to
  compare distributions

42
Concerns

• How are segments linked? Is a precip value at
  one basin linked to a precip value at a basin
  nearby? Is trace for one location linked to a
  trace at the downstream location?
• Users will think uncertainty distribution in
  hydrograph is really a set of hydrographs
• Recent historical QPF and MAP are limited and
  have been dynamic with time

43
Concerns (cont.)

• Basin boundary changes and addition of new basins
  require recomputing MAPs and QPFs
• Inconsistency between 14-day short term and
  90-day long term
• How do you handle forecast locations that are
  model deficient and require manual override?
  Blending of regulation modeling?
• Use of mods in the future?

44
Concerns (cont.)

• How can distributions be adjusted operationally?
• What can forecaster view to get some idea of
  possible needed adjustments to the distribution
  plots? Are ensemble data out of bounds?

45
NCRFC
46
Operations Concept Ideas

• Need the ability to operate and generate
  forecasts at the sub-basin level (segment)
• Ensemble generation and viewing should be
  controlled by an option switch (on/off/default)
  to allow for interaction with the model for basic
  tuning before introducing complications of
  ensembles (assumes integration w/ IFP)

47
Operations Concept Ideas, cont.

• System needs full transparency for the forecaster
  Summary views of inputs and model internals but
  with the capability to drill down and view
  detailed information
• Essential for quality control and debugging
• Smart tools to assist in diagnosis of the
  ensemble inputs, data assimilation, and results

48
Operations Concept Ideas, cont.

• Probabilistic hydrograph displays should include
  an overlay of historical ranges
• Historical perspective of Model inputs
• Selectable MODS parameter to designate a MOD
  for deterministic, ensemble or both
• Drawing tool to redraw forecasts necessary for
  timeliness and expediency in problem cases

49
Issues and Challenges

• Hydraulic routings complicates the ability to
  operate at the segment level
• Model BIAS correction error model
• DA ability to trace what it did and provide
  capability to undo or nudge differently
• Reservoirs ensembles on autopilot can introduce
  unrealistic results

50
(No Transcript)
51
Issues and Challenges, cont.

• Ability to use alternate model sources (inputs)
• Need to develop meaningful displays for end-users
  to illustrate the model inputs (focus group
  requirement)
• Hand-off / Receive mechanism for upstream /
  downstream RFCs

52
NERFC
53
NERFC Requirements

• At least 3-5 days of precip temperature data
  input
• Generate hydrograph ensembles out 5-7 days using
  graphics that have been reviewed by customers
• Generate SHEF output for selected confidence
  levels (or other means of having data readily
  available in AWIPS database)

54
NERFC Concerns

• How do you generate 72 hours of statistical
  ensemble input when normal operations never
  exceed 48 hrs of QPF?
• WFO Customer likes what-if contingency
  scenarios that are based upon known models
• Training for NWS staff needs to be included as
  part of development

55
NWRFC
56
The Shooting Target Meteorological forcing for
the hydrologic models originates from multiple
sources. The final product is a combination of
these procedures. Sources HPC, WFO ISC/GFE
grids, HAS, the numerical model of the day. Can
the methodology be developed to determine the
uncertainty and address all the inputs to the
hydrologic model? How do we adapt these
processes to the change in science and
operational procedures?
57
Scale Meteorological forcing for the hydrologic
models arrive at the RFC in many different
scales. These are then converted to a base scale
used in the hydrologic models. Can the current
data networks adequately support the proposed
level of downscaling? Will the uncertainty
results be beneficial? Do we see a universal
methodology to define the uncertainties generated
by the downscaling?
58
Transitions are where the money is Defining the
uncertainties is based on understanding the
statistics between the forecast and the
observation. Are we applying the uncertainties
appropriately? Transitions or storms are where
short-term forecasts make their money. The
statistics and uncertainties during these events
tend to shift from the climatology. Can we expect
to provide a true picture of the uncertainties
during the events that are most beneficial to
short-term ensembles techniques?
59
How does it work in a network How do
uncertainties translate from point to point in a
hydrologic system?
60
déjà vu Short- and long-term ESP needs to
address the man made influences such as
diversions and regulations. In the NWRFC area,
over 90 of the forecast points are influenced by
reservoir regulation and diversions. ESP is
unable to work on a variable set of rules used by
water managers. How do we quantify the
uncertainty of forecaster MODS?
61

• Benefits
• Hydrologic users like uncertainty information to
  assess the confidence in the forecast. Is there
  benefit and how is it determined?
• what is the timeline of short-term ESP being
  operational?
• will the uncertainty represent the true
  knowledge of the forecast error?
• need a system to develop and keep track of the
  uncertainty component to the forecast.
• NWS needs the ability deliver/provide more ESP
  derived information.
• requires a verification and validation system.

62
Products Don wants
1)
2) Similar displays or information as number 1,
but for meteorological forcings. 3) Verification
outputs which are converted to layman terms.
63
End of RFC presentations