Source: NHI course on Travel Demand Forecasting 152054A

1
Session 11 Model Calibration, Validation, and
Reasonableness Checks
Source NHI course on Travel Demand Forecasting
(152054A)
2
Objectives

• Identify and interpret trends affecting travel
  demand
• Explain difference between calibration and
  validation
• Identify critical reasonableness checks
• socioeconomic
• travel survey
• network
• trip generation
• mode split
• trip assignment

3
Terminology

• Model Calibration
• Model Validation
• Reasonableness checks
• Sensitivity checks
• Special generators
• Screen lines (some modelers do not think this is
  important)

Is the model sensitive to policy options?
4
Key Concepts

• Not enough attention on model evaluation and
  reasonableness checks
• Checks performed after each step
• reduces error propagation

Errors can also cancel
5
(No Transcript)
6
Planner responsibilities

• Actively involve all participants
• Modelers
• Planners
• Decision makers
• Public
• Fairly present all alternatives
• Timely
• Unbiased
• Identify (clearly) the decision making process
• Who, when, and how
• Allows input from all interested groups
• You must rely on the TDM
• Therefore, must be validated
• Accurate and easy to understand (documented)

7
Trends Affecting Travel Demand

• Planners should monitor the following trends
• Demographics
• Composition of the labor force
• Immigration and emigration
• Regional economic development
• Modal shares
• Vehicle occupancy
• Average trip length
• Freight transport
• Are trends consistent with assumptions made in
  the modeling process?

Must be aware of trends to ensure reasonable
forecasts
8
Information Requirements for Validation and
Reasonableness

• Demographics and employment
• Highway and transit networks
• Model specification
• Base year survey
• Base year traffic counts

9
Sources of Error

• Coding
• Sampling
• Computation (if done by hand)
• Specification
• Data Transfer
• Data aggregation

Improper structure of model, e.g., wrong variables
10
How do you judge a model/recommend improvement?

• Scrutinize these characteristics
• Data requirements
• Hardware requirements
• Logic of structure and conceptual appeal
• Ease of calibration
• Effectiveness of the model (accuracy,
  sensitivity)
• Flexibility in application
• Types of available outputs
• Operational costs
• Experience and successes to date
• Public or private domain availability
• Compatibility with other models and model types

11
Evaluation and Reasonableness Checks Overview
Complete? Level of Detail?
Reasonable? Methodology? Source?
Sensitive? Documentation of calibration? Valid
for base year?
Current? Reasonable?
12
Model Calibration and Validation
Feedback Loop
13
Model Validation

• Validation of new model
• Model applied to complete model chain
• Base year model compared to observed travel
• Judgment as to model suitability, return to
  calibration if not
• Validation of a previously calibrated model
• Compare to a new base year, with new
• SE data
• Special gen.
• Network
• Counts

Transportation Conformity Guidelines (Air
Quality) require model validated lt 10 years ago
14
Socioeconomic Data Check Reasonableness

• reviewSource for estimates and forecasts
• Population and household size trends (graph 1950
  to present and check trend)
• Household income trends (graph as far back as
  this goes 1990?)
• Check dollar values used in forecast (use
  constant dollars)
• If used, check trend of automotive availability
  (S curve?)
• Check distribution of employment by type (basic,
  retail, service) over time
• Plot and check trend of employees per household
  and per capita rate of increase is decreasing
• Check future household and employment changes by
  zone

15
Travel not sensitive to fuel price?
http//www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf
16
Travel Survey Data Reasonableness Checks

• Determine source of travel survey data
• Types of survey conducted
• Year of survey
• If no survey (borrowed)
• Check source of trip rates, lengths, TLFD
• Is area similar
• Geographic area?
• pop/HH/empl. characteristics?
• Urban density and trans system?
• Compare to similar regions and to same
• region in earlier times
• Person trip rates by trip purpose
• Mean trip lengths by trip purpose
• HBW longest? HBO shortest?
• TLFDs by trip purpose

17
Network Data Reasonableness Checks

• Check Trees for 2-3 major attractions
• Check coded facility types how used (BPR?)?
• Verify speed and capacity look-up table (what LOS
  used for capacity?)
• Significant transportation projects narrative
  included? Still viable?
• Consistency with MTP
• Plot (facility types, lanes,
• speeds, area types) to detect
• coding errors

18
Trip Generation Reasonableness Checks

• Examine trip production and attraction models
• Form?
• sensitivity?
• Examine trip purposes used
• External-through and external-local trips how
  modeled?
• Truck trips how modeled?
• Person trip or vehicle trip rates used?
• PA balance (0.9-1.1 ok)
• Special generators (check, and be consistent in
  future model)

19
Trip Generation CalibrationTypical Values

• Person trips per household 8.5 to 10.5
• HBW person trips per household 1.7 to 2.3
• HBO person trips per household 3.5 to 4.8
• NHB person trips per household 1.7 to 2.9
• HBW trips 18 to 27 of all trips
• HBO trips 47 to 54 of all trips
• NHB trips 22 to 31 of all trips

20
Trip Generation Calibration
TRIP PURPOSES Scaling Factor HBW low
income 0.795 HBW low-middle income 0.823 HBW
middle income 0.861 HBW upper middle
income 0.908 HBW high income 0.936 HB
elementary school 0.733 HB high
school 1.991 HB university 0.895 HB
shopping 0.698 HB social-recreation 0.945 HB
other 0.875 NHB work-related 0.858 NHB
other 0.820 Truck 0.985 Internal-external 0
.591
Note each income class is a purpose!
Scale survey for participation (relative
participation)
Colorado Springs 1996 Travel Demand Model
Calibration
21
Trip Generation Calibration

• Reasonableness checks compare to other cities,
  check future trends
• Population 503,345
• Households 201,116
• Average Household Size 2.50
• Basic employment 76,795 (33)
• Retail employment 50,465 (24)
• Service employment 101,697 (43)
• Military employment 42,800
• Population per employee 1.81
• Person trips per person 4.26
• Person trips per household 10.65
• HBW attractions per employee 1.44
• HBW productions per household 1.74
• HB shopping attractions per retail employee 5.99

Colorado Springs 1996 Travel Demand Model
Calibration
22
Trip Distribution Reasonableness Checks

• Examine
• Mean trip length (increasing or decreasing?)
• TLFDs
• Treatment of friction factors (same?)
• Treatment of terminal times (logic?)
• Treatment of K factors
• Comparison with JTW trip length
• Comparison with JTW sector interchange volumes or
  percentages.

23
1st iteration
Calibrate friction factors
24
Calibrating a Gravity Model Adjusting Friction
Factors
25
2nd iteration
26
Trip Distribution Calibration and Validation

• Check modeled vs. household survey TLFD and mean
  trip lengths
• Get HBW area-to-area flows from JTW
• HBW 1990 JTW TLFD and Area-to-Area Flows for
  Kansas City

27
(No Transcript)
28
Mode Split Reasonableness Checks

• Automobile occupancy factors by trip purpose
  used?
• Basis?
• Constant?
• Mode split model?
• Form?
• Variables included in the utility functions?
• Coefficients logical?
• Value of time assumptions
• Parking cost assumptions
• How do mode shares change over time?
• Mode share comparisons with other cities

29
Mode Split Calibration and Validation

• Experienced planning consultant required
• Form of LOGIT model
• Variables included in utility functions
• Calibration of coefficients for utility function
  variables
• Testing for IIA properties
• Analysis of household survey data
• Analysis of on-board transit survey data
• Calibration tasks we can do
• Compare highway and transit trips
• Total
• By purpose
• Compare Ridership by route
• CBD cordon line survey (if bus service is
  downtown only)

30
Trip Assignment Reasonableness Checks

• All-or-nothing assignment
• study effect of increasing capacity
• Compare to Equilibrium assignment
• Check volume delay equation (BPR parameters)
• Compare
• screen line volumes
• Cut line volumes
• Time-of-day assignments?
• Source of factors
• Peak spreading used for future?
• If not, conversion factors source?
• (peak hour to 24-hour)
• Local VMT ( assigned to
• intrazonals and centroid connectors

Equil ibrium
All or Nothing
31
Trip Assignment Calibration and Validation
Assignment calibration performed
last

• Overall VMT or VHT check
• 40 to 60 miles per day per HH in large metro
  areas
• 30 to 40 miles per day per HH in medium metro
• /- 10 OK on screen lines
• Sign is important

32

• Compute by
• volume group
• facility type
• transit assignments
• time of day

33
(No Transcript)
34
Other Factors Impacting Forecasted Travel Demand
(use your noodle)

• Can be implied in travel surveys (but not
  explicit)
• Telecommuting
• Flexible work hours
• HB business
• How to account for
• Aging population
• Internet shopping
• Roadway congestion (will it affect generation in
  the future)
• New modes