Wes Junker

1
SEMI-INTELLEGENT USE OF THE NCEP MODELS (SPRING
2000)

• Wes Junker
• e-mail norman.junker_at_noaa.gov

With help from Geoff Mannikin
Presented COMAP Symposim 00-2By Wes
JunkerTuesday, 28 March 2000
2
Why models have forecast problems

• Initialization and quality control smooth data
  fields, but some of the lost detail may be
  important.
• May have poor first guess
• Lack of data over the oceans and Mexico.
• Atmospheric processes are non-linear small
  changes in initial conditions can lead to large
  forecast variations (this is the basis for
  ensemble forecasting).
• Model physics are approximations (radiation,
  cloud physics, convection, boundary layer, etc.)
• for lower resolution models (the current
  operational models), convection is parameterized
• for higher resolution models (models with a
  resolution below 5 km) the micro-physical
  processes are parameterized

3
INTELLIGENT USE OF THE MODEL REQUIRES THAT THE
FORECASTER

• COMPARE THE INITIAL 00HR FORECAST WITH DATA
• MAY BECOME HARDER TO DO AS NEW DATA STREAMS ARE
  USED.
• BE FAMILIAR WITH CHARACTERTIC MODEL ERRORS AND
  BIASES.
• THESE SEEM TO VARY BY SEASON AND REGIME
• HAVE A ROUGH UNDERSTANDING OF HOW APPROXIMATIONS
  OF THE PHYSICS MAY NEGATIVELY IMPACT A FORECAST.
  

4
Understanding how the physics may impact a
forecast is tough because the atmosphere is
complicated and acts in a non-linear fashion.
For example, whenever the parameterization for
convection kicks in, it

• redistributes and generates heat
• Changes the vertical stability
• redistributes and removes moisture
• redistributes momentum
• makes clouds

Adapted from notes of Bernard Meisner
5
Eta Model Physics

• Eta model calculates grid-scale precipitation
  using a simplified explicit cloud water scheme
• includes super-cooled water, simplified snow
  processes and the advection of cloud water and
  cloud ice
• but does not include horizontal advection of snow
  and rain.
• In fast flow snow can advect 50 to 100 km
  downwind of its source region (Rauber, 1992))

6
EXPLICIT CLOUD PREDICTION SCHEME (large scale)

• Cloud condensation is allowed to occur when the
  RH reaches a critical value
• Cloud evaporation is allowed to take place only
  when the RH falls below the critical value
• 70 over land, 80 over water
• the difference in the critical value between land
  and water can produce discontinuities along the
  coast
• this may be one of the reasons the Eta over
  predicts cold season precipitation along the Gulf
  and Atlantic Coasts.
• the other reason was the models convective
  scheme used different reference profiles over
  land and water.

7
The BMJ Convective Scheme has been changed

• 1st looks for deep convection
• step 1 is to look for most unstable layer within
  the lowest 130 mb
• Next calculates LCL to get cloud base
• then lifts parcel to Equilibrium Level to get
  cloud top
• then looks to see if the cloud layer is deep
  enough to generate precipitation. If needs to be
  (200(surface pressure/1000 mb)
• If the cloud is not deep enough it looks for
  shallow convection

8
The BMJ scheme

• Was developed for tropical systems
• in the past the scheme has not handled elevated
  convection well.
• the convection may not extend through a deep
  enough layer
• does not develop realistic downdrafts/outflow
  boundaries. These boundaries often focus
  convergence and convection.
• Does not need low level convergence to form
  convection
• the real atmosphere does
• The scheme tends to sometimes break convection
  out too far south because it does not need low
  level convergence to trigger convection.
• The model should be a little better at predicting
  elevated convection but still should have a low
  bias

9
THE ETA OFTEN FORECAST TOO MUCH RAINFALL NEAR THE
GULF AND SOUTHEAST COASTS BECAUSE OF THE PROBLEMS
WITH THE WAY THE ETA HANDLES THE LAND-SEA
INTERFACE
In the past
12-36 HR PRECIPITATION FORECAST V. T. 12Z 1 APR
24 HR PRECIPITATION ANALYSIS V. T. 12Z 1 APR
10
The parallel test for convection with the
reference profiles held the same over land and
water reduced the bias problem over the southeast
during winter
WILL IT CHANGE OTHER PERFORMAANCE
CHARACTERISTICS? FOR EXAMPLE, MAKE IT DRIER
FARTHER INLAND. TIME WILL TELL.
11
A forecaster needs to know how the model terrain
compares to the actual terrain
12
THE MODELS TERRAIN IS AVERAGED OVER THE GRID BOX
SO THE SLOPE OF THE TERRAIN IS USUALLY NOT STEEP
ENOUGH
THIS CAUSES THE VERTICAL MOTION FIELD TO BE
SHIFTED AWAY FROM THE MOUNTAINS
13
THINGS TO REMEMBER ABOUT MODEL QPFS IN COMPLEX
TERRAIN DURING WINTER
BECAUSE OF THE SIMPLIFIED MICROPYSICS AND
INADEQUATE RESOLUTION OF MOUNTAINS. MODELS
USUALLY
1) PREDICT PRECIPITATION TOO FAR WEST AWAY FROM
MOUNTAIN PEAKS
2) DO NOT ALLOW ENOUGH PRECIPITATION ON THE
IMMEDIATE DOWNWIND SIDE OF MOUNTAIN RANGES
THIS IS ESPECIALLY TRUE IN WINTER
14
A models behavior may alter

• when there is a change in model physics (at the
  end of the month),
• by season
• and regime
• other factors, ie. a poor first guess may
  produce a model error that does not fit the
  norm.

15
When the NGM and AVN sheared 500 troughs
approaching the east coast in 98-99, the eta
often amplified the trough and overdeepened the
surface low. An example
48 HR ETA 500
48 HR ETA SFC
48 HR NGM SFC
48 HR NGM 500
16
In 1999, the Eta predicted several major
snowstorms that did not verify. The NGM and AVN
predicted light snow at best
36-48 hr ETA precipitation
36-48 hr NGM precipitation
17
HOW THE MODEL VERIFIED. NO MAJOR SNOWSTORM
DEVELOPED.
48 HR ETA 500
VERIFYING 500 MB
48 HR ETA SFC
VERIFYING SFC
18
500 forecasts valid 12Z 25 Jan. 2000 (purple),
observed (orange)
Eta is trending slower and deeper
12Z 24 Jan run
00Z 24 Jan run
Note similarities between the forecast 500 mb
patterns with this system and the one in 1999
19
surface pressure forecasts
Comparison of eta from 00Z 24 Jan (red) and 12Z
24 Jan (green)
From 12Z 24 Jan run, eta (red), avn (green)
20
the eta trended westward with its QPF
QPF valid 12Z 25 Jan
00Z 24 Jan run
12Z 24 Jan run
21
(No Transcript)
22
12Z Jan 24 ETA 250 mb analysis of speed (color
fill) and observed RAOB speed. Note the 62 m/s
observation at ATL where the initial analysis
thought the speed was 30 m/s
From Mannikin
23
A comparison of the ATL observed speed (ms-1,
red) with the first guess (green) and initial
analysis (blue)
24
Eta continues to pull storm to west
24 hr eta qpf from 18Z 24 Jan. A little later,
the 18Z run off the Avn predicts between a .25
and .50 over DCA
Eta surface pressure forecast from 12z 24 Jan
(red) and 18Z 24 Jan v.t. 12Z 25 Jan
25
What happened?
26
WHEN THE MODEL FIRST GUESS THINKS THE SOIL
MOISTURE IS HIGH,
THEN, THE MODEL FORECASTS SURFACE DEWPOINTS TOO
HIGH AND SURFACE TEMPS TOO LOW. FORECAST CAPES
WILL BE TOO HIGH
32
SURFACE TEMPERATURE
28
24
OBSERVED
20
ETA FORECAST
24
DEWPOINT TEMPERATURE
20
16
12
1024
1020
SURFACE PRESSURE
1016
1012
THE MODEL UNDERPREDICTS THE BOUNDARY LAYER WINDS.
HOWEVER, MODEL FORECASTS OF 850 MB WINDS ARE
OFTEN TOO STRONG
20/00
18/00
19/18
19/12
19/06
19/00
18/18
18/12
18/06
27
WHEN THE FIRST GUESS THINKS THE SOIL MOISTURE IS
LOW IN SUMMER IN THE PLAINS, THE SURFACE DEWPOINT
WILL BE LOW AND THE TEMPERATURE WILL BE TOO HIGH.
FORECAST CAPES WILL BE TOO LOW
OKLAHOMA CITY
36
SURFACE TEMPERATURE
32
OBSERVED
28
24
ETA FORECAST
20
20
DEWPOINT
16
TEMPERATURE
12
1016
SURFACE PRESSURE
1008
ETA SURFACE WINDS WERE TOO WESTERLY, WAS THERE
TOO MUCH DOWN-SLOPE?
21/00
20/180
20/12
20/06
20/00
19/18
19/12
19/06
19/00
MAY 1998
28
Forecast -Vs- Observed Best CapeSpring 96
Line xy
Note the large spread. The model stability
forecasts are worst when precipitation is forecast
Line xy
Forecast precipitation
1 - less than .25
2 - more than .25
29
The performance characteristics of the eta
changed again in 2000.

• QPF forecasts during the past winter were more
  competative with the AVN during winter
• ETA surface and 500 mb forecasts were also
  better this year though not as good on average as
  the avn.
• June-August 1999, The eta model QPF usually
  verified better than the AVN or NGM.

30
LOWS TO THE LEE OF THE ROCKIES

• THE AVN AND NGM USUALLY PREDICT THEM TO FORM TOO
  FAR NORTH
• THE ETA IS SOMETIMES A LITTLE TOO FAR SOUTH
• USE THE 300 MB UPPER LEVEL JET. THE SURFACE LOW
  IS USUALLY FOUND IN THE LEFT EXIT REGION OF THE
  JET, USUALLY JUST TO THE NORTH

31
The Eta surface low and associated fronts can
also be affected. The slower eastward movement
of the ridge axis may allowed for the flow along
the east to be more northwesterly which allowed
the surface boundary to sink farther to the
south. This bias did not show up much during the
winter of 2000
00 hr Eta v.t. 12Z 10 Apr
48 hr Eta v.t. 12Z 10 Apr
32
During the winter of 2000, subjective evaluation
of Dec-Feb suggested this was the more common
problem
How would the differences at 500 mb affect the
surface pattern?
33
MORE ON ETA PERFORMANCE

• TOO WET IN FLORIDA
• SOMETIMES OVERDEVELOPS LOW-LEVEL JET
• DURING WINTER 1998-1999 WAS TOO FAST BRINGING
  SHORTWAVES THROUGH THE ROCKIES INTO THE PLAINS.
• DURING THE 1999-2000 WINTER, THE MODEL DID NOT
  APPEAR TO BE TOO FAST WITH SIMILAR SYSTEMS.
• HAS BEEN TOO FAR SOUTH WITH CLOSED LOWS COMING
  EASTWARD INTO THE PLAINS
• OVERFORECASTS THE STRENGTH OF ANTICYCLONES
• CAN HAVE PROBLEMS INITIATING INTENSE MESOSCALE
  FEATURES.

34
Models have problems with arctic airmasses.The
reasons why are listed below

• Terrain is averaged
• Initialization process sometimes robs shallow
  airmass of its coldness
• Models have problems handling strong inversions
• Models have problems handling cold air damming
• The sigma coordinate system, the Eta coordinate
  system does better
• The leading edge of the ETA LI gradient is often
  the best indicator of the frontal position

35
THE NGM AND AVN/MRF HAVE SERIOUS PROBLEMS WITH
ARCTIC AIRMASSES.
L
36 HR NGM V.T. 00Z APR 09, 1995
AVN ANALYSIS V.T. 00Z APR
09, 1995
TEMPERATURES ACROSS KANSAS WERE IN THE LOW TO MID
50s WITH STRONG NORTH WINDS. SOUTH OF THE FRONT
TEMPERATURES WERE IN THE UPPER 70s TO LOW 90s.
WHEN THE ETA 500 H FORECAST IS COMPARABLE TO THE
OTHER MODELS IT WILL DO A BETTER JOB IN HANDLING
THE COLD AIR SURGE
36
VERIFYING PRECIPITATION

• BIASFORECAST/OBSERVED
• EQUITABLE THREAT(H-E)/(FO-H-E)
• THREAT SCOREH/(FO-H)
• NNUMBER OF HITS, FNUMBER OF GRID POINTS
  FORECAST, OGRID POINTS OBSERVED, E(FO)/N

37
MODEL BIAS AND THREAT SCORE

• IS DEPENDENT ON RESOLUTION OF MODEL
• HOW THE MODEL IS DISPLAYED. THE FAX VERSION OF
  ETA IS NOT DISPLAYED WITH FULL MODEL RESOLUTION!
• HOW THE MODEL IS VERIFIED
• WHETHER VERIFIED AT A POINT, OR AVERAGED OVER A
  GRID BOX

38
Study of heavy rainfall events in middle of
country. Note low bias for mesohigh events and
general low bias for heavier thresholds
From Watson et. Al, 1999
39
Accuracy decreases rapidly as threshold increases
From Watson et. Al, 1999
40
ETA THREAT SCORES FOR 1.00 WERE LOWER THAN THOSE
FROM THE SUBJECTIVE AND AVN GUIDANCE DURING
WINTER, AVN LAGS ETA IN SUMMER.
41
12-36 hr Eta 1.00 model verification during Feb
2000

• .50 threat score.366, bias.87
• 1.00 threat score.204, bias.264
• 2.00 threat score.04, bias.14
• the subjective guidance
• .50 T.S..390, bias1.08
• 1.00 T.S..303, bias1.02
• 2.00 T.S..18, bias.307
• the day 3 subjective qpf for 1.00 had a
  T.S..313, higher than the Eta.

42
Regional ETA verification using model grid (80 km)
WARM SEASON 1.00 OR MORE VERIFICATION
VERIFIED TO AN 80 KM GRID
.64 .15
.97 .18
.98 .15
.93 .17
.65 .14
.59 .19
.35 .09
.47 .08
.83 .12
BIAS TOP NUMBER, EQUITABLE THREAT BOTTOM
43
Regional ETA verification using model grid (80 km)
From 1998 data
COLD SEASON 1.00 OR MORE VERIFICATION
VERIFIED TO AN 80 KM GRID
.69 .17
1.07 .23
.94 .18
1.36 .22
.74 .09
.58 .10
.71 .27
.71 .15
1.04 .19
BIAS
ETS
AGAIN NOTE HIGH BIAS ALONG EAST COAST AND LOW
BIAS OVER WEST
44
Regional ETA verification using model grid (80 km)
.01 OR GREATER AMOUNTS DURING COLD SEASON
VERIFIED TO AN 80 KM GRID
1.43 .25
1.05 .35
1.07 .35
.81 .37
1.23 .23
.79 .32
.95 .26
1.11 .34
1.07 .35
HIGHEST THREATS ALONG WEST COAST. HIGH BIAS OVER
UPSLOPE AREAS EAST OF ROCKIES AND OVER PLAINS
45
Regional ETA verification using model grid (80 km)
.01 OR GREATER AMOUNTS DURING WARM SEASON
VERIFIED TO AN 80 KM GRID
1.11 .28
.96 .39
.92 .37
.81 .34
1.21 .19
1.00 .37
.82 .23
1.01 .32
.99 .38
BIG DIFFERENCES WITH POINT VERIFICATION. USING A
POINT VERIFICATION, YOU SEE THE HUGE BIASES OVER
THE SOUTH
46
ETA .50 OR MORE PERFORMANCE DURING WARM SEASON
VERIFIED TO AN 80 KM GRID
.77 .21
1.10 .23
1.09 .25
1.07 .24
.88 .12
.82 .17
.82 .28
.62 .14
.86 .20
BIAS
ETS
DURING SUMMER ETA UNDERPREDICTS .50 OR GREATER
AMOUNTS IN PLAINS.
47
ETA PERFORMANCE FOR .50 OR GREATER AMOUNTS APR
96-NOV 97
VERIFIED TO AN 80 KM GRID
.89 .23
1.32 .31
1.00 .15
1.10 .23
1.13 .31
.97 .13
.83 .35
1.10 .26
.90 .23
BIAS THREAT
ETA OVERPREDICTS .50 OR GREATER ACROSS SOUTH AND
ALONG EAST COAST. MESO-ETA HAS SAME BIAS
48
IN CONCLUSION

• THE ETA MODEL HAD SERIOUS PROBLEMS WHEN 3DVAR WAS
  FIRST IMPLIMENTED. .
• BUT WAS MUCH BETTER IN 1999-2000 AND IS NOW
  COMPETATIVE WITH THE AVN/MRF.
• HAS A LOW BIAS FOR 1.00 OR GREATER AMOUNTS.
• BETTER VERIFICATION IS NEEDED OF OPERATIONAL
  MODELS. REGIONAL VERIFICATION IS NEEDED.
• EMC AND HPC ARE NOW MAINTAINING A VERIFICATION
  SECTION ON THEIR HOMEPAGE.
• EACH TIME A MODEL IS CHANGED, IT MAY AND PROBABLY
  WILL CHANGE THE PERFORMANCE CHARACTERISTICS.