CE 451 - Urban Transportation Planning and Modeling

1
CE 451 - Urban Transportation Planning and
Modeling Iowa State University Calibration,
Adjustment and Validation
Sources Calibration and Adjustment of System
Planning Models Note Date 1990 (need to
adjust for inflation, other changes) Model
Validation and Reasonableness Checking
Manual NHI course on Travel Demand Forecasting
(152054A)
2
Objectives

• Identify and interpret trends affecting travel
  demand
• Explain difference between calibration and
  validation
• Identify critical reasonableness checks
• socioeconomic
• travel survey
• network
• trip generation
• mode split
• trip assignment

3
Terminology

• Model Calibration
• Estimate parameters
• Match observations (OD, AADT)
• Model Validation
• Reasonableness checks
• Sensitivity checks
• Special generators
• Screen lines, cut lines, cordons

Is the model sensitive to policy options?
4
Planner responsibilities

• Actively involve all participants
• Modelers
• Planners
• Decision makers
• Public
• Fairly present all alternatives
• Timely
• Unbiased
• Identify (clearly) the decision making process
• Who, when, and how
• Allows input from all interested groups
• You must rely on the TDM
• Therefore, must be validated
• Accurate and easy to understand (documented)

www.readysetpresent.com
5
How do you judge a model/recommend improvement?

• Scrutinize these characteristics
• Data requirements
• Logic of structure and conceptual appeal
• Ease of calibration
• Effectiveness of the model (accuracy,
  sensitivity)
• Flexibility in application
• Types of available outputs
• Operational costs
• Experience and successes to date
• Public or private domain availability

cio.com
6
Trends Affecting Travel Demand

• Planners should monitor the following trends
• Demographics
• Composition of the labor force
• Immigration and emigration
• Regional economic development
• Modal shares
• Vehicle occupancy
• Average trip length
• Freight transport
• Are trends consistent with assumptions made in
  the modeling process?

Must be aware of trends to ensure reasonable
forecasts
Image sources scu.edu usda.gov illinois.edu
uwex.edu mwcog.org fhwa.gov
transportation1.org
7
How sensitive is travel to fuel price?
doe.gov
http//www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf
8
(No Transcript)
9
Tips for building a good model

• Build accurate road network
• Use aerial photos behind
• Make sure road attributes are correct, esp.
  traffic
• Use hourly counts
• Income and auto ownership dont fully explain
  travel
• age, gender, life cycle and personal interest
  come into play
• Use survey data
• visualize these data
• Survey can be done cheaply
• Cooperation will be good if theres a good reason
  for it mayor sends letter, e.g.

• Employer based surveys get good response (but may
  be biased)
• Some will give home addresses, customer
  addresses, license plates
• Use trip chaining (tour based) and activity based
  trip generation
• We dont know much about attractions ITE sample
  too small do your own
• Drive the network using GPS
• Get some data and do some statistics to derive
  your parameters

Howard Slavin, Caliper Corp. 3/13/04 peer review
10
Tips for building a good model

• Some models are completely made up except traffic
  counts
• See if you really believe the counts
• Create your OD matrix from ground counts
• May be better than trip gen/dist if you made up
  the whole model (no surveys)
• TransCAD has a tool for this
• If still want to use trip gen/dist, this method
  can be used to determine K factors
• Could also use the row and column totals as the
  dependent variables in your trip gen model

• Examine individual links after model run
• Where are the trips coming from and going to that
  use the link?
• In TransCAD, what is the process used to
  determine this (for a particular link)?
• In TransCAD, what is the process used to show
  where traffic from a particular zone is going to?
• Familiarity with your region is helpful

Howard Slavin, Caliper Corp. 3/13/04
11
Sources of Error

• Coding
• Sampling
• Computation (if done by hand)
• Specification
• Data Transfer
• Data aggregation

Improper structure of model, e.g., wrong variables
12
Key Concepts

• Not enough attention on model evaluation and
  reasonableness checks
• Checks should be performed after each step
• reduces error propagation

Errors can also cancel
13
(No Transcript)
14
Evaluation and Reasonableness Checks Overview
Number and location of households and employment (demand) Socioeconomic Data
Complete? Level of Detail?
Transportation system (supply) Network Data
Reasonable? Methodology? Source?
TDF Model Specification Model validation and calibration
Travel survey data
Transportation system performance
Sensitive? Documentation of calibration? Valid
for base year?
Current? Reasonable?
15

• CALIBRATION and VALIDATION are sometimes
  confused.
• Model development is sometimes called calibration
  or estimation as we are estimating parameters and
  constants for the particular model structure.
• estimating is a statistical process want high
  correlation coefficients and significant
  parameter values
• can "import" a model - or borrow structure and
  parameters from a "similar" area
• VALIDATION is checking if the model is accurately
  estimating traffic volumes by calculated measures
  (like RMSE)

Model Calibration
Feedback Loop
Model Validation
Model Application
16
Model Validation

• Validation of new model
• Model applied to complete model chain
• Base year model compared to observed travel
• Judgment as to model suitability, return to
  calibration if not
• Validation of a previously calibrated model
• Compare to a new base year, with new
• SE data
• Special gen.
• Network
• Counts

Transportation Conformity Guidelines (Air
Quality) require model validated lt 10 years ago
17
Validation suggestions

• - Systemwide
• - compare traffic counts across
• - Screenlines
• (long lines, check major flows) check trip
  interchange (distribution) between large sections
  or quadrants
• need a survey local knowledge of commute
  patterns helps
• - Cordon lines (surround a major generator, e.g.
  university, CBD...)
• - Cutlines (shorter, verify corridor flows, fine
  tuning)
• if "importing" should validate all borrowed
  parameters and constants

18
IT IS VERY IMPORTANT TO HAVE A GOOD COUNT PROGRAM
DESIGNED TO SUPPORT VALIDATION!
iowadotmaps.com
19
To "calibrate" the model, need an OD database
from a survey. This is time consuming and
expensive. Few, if any cities have developed OD
databases since 1980, but many have updated old
ones since then using a small survey (e.g. 1)
The Calibration and Adjustment manual is not
intended to replace good OD data, and is intended
more for small urban areas. (and has some old
data in it! more recent data area available in
the Barton-Ashman publication).
20
Calibration and Adjustment Steps 1) verify
network and socioeconomic data 2) run the
model 3) develop region-wide values (e.g.
trips/person, vmt/person) 4) compare region wide
values with Appendix A values 5) develop
screenlines and cutlines 6) compare model results
with ground counts for crossings 7) determine
problems (system level, local, combination) 8)
modify one or more equations, parameters or
variables according to chapters on -
networks - trip generation - auto occupancy -
trip distribution - traffic assignment
21
Other chapters focus on - transit - external
stations - system vs. local changes - expected
vs. required accuracy - conclusions - trouble
shooting
22
Network Data Reasonableness Checks

• Check Trees for 2-3 major attractions
• Check coded facility types how used (BPR?)?
• Verify speed and capacity look-up table (what LOS
  used for capacity?)
• Speed adjust (can lower the freeway speed if it
  is being overloaded tweak?)
• Significant transportation projects narrative
  included? Still viable?
• Consistency with MTP
• Plot (facility types, lanes,
• speeds, area types) to detect
• coding errors
• Items we can check in labs

23
Details 2. Network Errors 2.1 Centroid
Connectors - represent local streets - check
access (all 4 sides?) - not connected to
intersections - make sure they are not blocked by
a physical barrier (river, etc.)
24
Des Moines Model Capacity Look-up Table
25
Des Moines Model Capacity Look-up Table (cont.)
26
Des Moines Model Capacity Look-up Table (cont.)
27
Des Moines Model Capacity Look-up Table (cont.)
28

• 2.3 Intersection Penalties (check them!)
• - most congestion here
• - more important in sub-area modeling
• - turn penalties
• account for congestion
• - speed volume function
• - can include delay on approach links
• - can do it manually for small networks
• check for circuity (correct with small turn
  penalties!)

29

• See TransCAD Manual B Chapter 10 Traffic
  Assignment with Volume Dependent Turning Delays

30

• 2.4 Intrazonal times
• increasing intrazonal trips (in distribution)
  decreases interzonal trips (useful if too many
  trips are being loaded on the network)
• number of trips is a function of travel time
  (gravity model)
• can adjust travel time on intrazonals
• can adjust friction factor curve to produce more
  shorter trips (which intrazonals usually are)
• can change definition of zones (size, land use)
• Air quality analysis implications???

31
3.1 Trip generation - socioeconomic data can be a
source of error - initial step is to check system
trip totals, compare w/ Table 4 and A1 and A2
(next pages) - if there is a problem, check the
system number of dwelling units - still a
problem?, check production/attraction rates
32
Trip Generation Calibration

• Reasonableness checks compare to other cities,
  check future trends
• Population 503,345
• Households 201,116
• Average Household Size 2.50
• Basic employment 76,795 (33)
• Retail employment 50,465 (24)
• Service employment 101,697 (43)
• Military employment 42,800
• Population per employee 1.81
• Person trips per person 4.26
• Person trips per household 10.65
• HBW attractions per employee 1.44
• HBW productions per household 1.74
• HB shopping attractions per retail employee 5.99

Colorado Springs 1996 Travel Demand Model
Calibration
33
(No Transcript)
34
Table A2
35
More recent data
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
36
(No Transcript)
37
3.2 Income - be sure you are using real dollars
38
3.3 P and A rates

• Problems old, borrowed, small survey
• may work OK at the system level, but not for
  sub-areas
• check system-wide values (see tables, next pages)
• raise or lower trip generation rates
• Person trip or vehicle trip rates used?
• we usually have person trip by purpose, but can
  apply occupancy factor and check against vehicle
  rates (ITE)
• later, screen line counts can be adjusted by
  varying trip generation rates (post assignment)
• check cutlines and cordon counts
• coordinate all of the above

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
Non-
home
43
Trip Generation CalibrationTypical Values
More recent data

• Person trips per household 8.5 to 10.5
• HBW person trips per household 1.7 to 2.3
• HBO person trips per household 3.5 to 4.8
• NHB person trips per household 1.7 to 2.9
• HBW trips 18 to 27 of all trips
• HBO trips 47 to 54 of all trips
• NHB trips 22 to 31 of all trips

44
Trip Generation Reasonableness Checks

• Examine trip production and attraction models
• Form?
• sensitivity?
• IMPORTANT keep parameters reasonable (e.g. don't
  use negative coefficients in regression models
  just because they provide the best fit.)
• If you think you need to use unintuitive
  parameters, check the whole process...
• Check models for
• External-through and external-local trips
• Truck trips
• To calibrate trip generation and trip
  distribution, sometimes we may use ...
• default values from past surveys
• very limited new surveys
• census journey to work data (CTPP)

45
Examine trip purposes used Use more trip
purposes?
TRIP PURPOSES Scaling Factor HBW low
income 0.795 HBW low-middle income 0.823 HBW
middle income 0.861 HBW upper middle
income 0.908 HBW high income 0.936 HB
elementary school 0.733 HB high
school 1.991 HB university 0.895 HB
shopping 0.698 HB social-recreation 0.945 HB
other 0.875 NHB work-related 0.858 NHB
other 0.820 Truck 0.985 Internal-external
0.591
Note each income class is a purpose!
Scale survey for participation (relative
participation)
Colorado Springs 1996 Travel Demand Model
Calibration
46
Travel Survey Data Reasonableness Checks

• Determine source of travel survey data
• Types of survey conducted
• Year of survey
• Scale survey for participation
• If no survey (borrowed)
• Check source of trip rates, lengths, TLFD
• Is area similar
• Geographic area?
• pop/HH/empl. characteristics?
• Urban density and trans system?
• Compare to similar regions and to same
• region in earlier times
• Person trip rates by trip purpose
• Mean trip lengths by trip purpose
• HBW longest? HBO shortest?
• TLFDs by trip purpose

47
Socioeconomic Data Check Reasonableness

• Review source for estimates and forecasts
• Visualize (plot) trends
• Population and household size
• Household income
• automotive availability
• distribution of employment by type (basic,
  retail, service)
• employees per household and per capita rate of
  increase is decreasing
• Check future household and employment changes by
  zone

48

• 3.4 Special generators
• e.g. universities, airports, malls, ...
• Use ITE or survey

49
3.5 trip balancing factors4.0 Auto occupancy

• initially, Ps and As should balance to should be
  0.9 to 1.1 if not, check your PA rates and
  socioeconomic data
• NHB is usually out of balance
• Automobile occupancy
• by trip purpose?
• Basis?
• Constant?
• see table 6 and A9 (next pages are these still
  good?)

50
(No Transcript)
51

• 5.0 Trip Distribution
• 5.1 Mean Trip Length
• - recall shape of curve affects trip length
  distribution
• See below for effect of changing friction factors

52

• varying trip length has a big impact on assigned
  volumes
• portions of a friction factor table can be
  adjusted (more flexible than adjusting equations)

53
(No Transcript)
54

• 5.2 Estimate Trip Length
• compare average trip lengths (in minutes) by
  purpose to
• HBW t 0.98 x p.19
• HBSR t 2.18 x p.12
• HBSh t 8.1
• NHB t 0.63 x p.20
• where p is population
• SR social/recreation
• Sh shopping

55
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
56
Source Virginia Travel Demand Modeling Policies
and Procedures Manual
57
Source Virginia Travel Demand Modeling Policies
and Procedures Manual
58
Source Virginia Travel Demand Modeling Policies
and Procedures Manual
59
5.3 Employment Distribution Problems (large
cities, mostly) problem match low income
households with low income jobs solution 1
disaggregate trip purposes by income
quartile solution 2 use k-factors (trial and
error) yuk
Jobs/Housing Imbalance!
60

• 5.4 Special Treatment, other trip purposes
• - schools (ignore if small ?)
• - trucks (calibrate with externals?)
• Taxi
• normally, distortions are insignificant

61
Trip Distribution Reasonableness Checks

• Examine
• Mean trip length (increasing or decreasing?)
• TLFDs
• Treatment of friction factors (same?)
• Treatment of terminal times (logic?)
• Treatment of K factors
• Comparison with JTW trip length
• Comparison with JTW sector interchange volumes or
  percentages.

62
Calibrating Friction Factors
1st iteration
Calibrate friction factors
63
Calibrating a Gravity Model Adjusting Friction
Factors
Travel Times Ranges from Skims Observed Trip Expanded from Surveys Input Friction Factors Gravity Model Trips Adjustment Factor Observed Gravity Model New Friction Factors Friction Adjustment Factor x Friction Factor
2.5 7,100 30.0 8,200 0.87 25.98
5.0 14,950 2.50 16,300 0.92 2.29
7.5 17,850 1.80 19,250 0.93 1.67
10.0 16,000 1.50 19,100 0.84 1.26
12.5 15,500 1.20 17,100 0.91 1.09
15.0 15,900 1.00 12,300 1.29 1.29
17.5 16,400 0.95 18,000 0.91 0.87
20.0 15,150 0.90 14,300 1.06 0.95
22.5 13,500 0.85 11,900 1.13 0.96
25.0 11,000 0.80 9,250 1.19 0.95
27.5 9,500 0.75 8,100 1.17 0.88
30.0 9,100 0.70 6,100 1.49 1.04
32.5 5,700 0.65 4,900 1.16 0.76

64
2nd iteration
65
Trip Distribution Calibration and Validation

• Check modeled vs. household survey TLFD and mean
  trip lengths
• Get HBW area-to-area flows from JTW
• HBW 1990 JTW TLFD and Area-to-Area Flows for
  Kansas City

Commute Length in Minutes Percent Journey-to-Work Flows Percent
lt 15 27.87 Central-Central County 31.49
15-29 41.63 Central-Suburban County 7.48
30-39 17.04 Suburban-Central County 15.13
40-59 7.70 Within Suburban County 32.98
gt60 3.00 To Other Suburban County 10.81
Mean 21.44 Work out of area 2.11
66
(No Transcript)
67
OD validation
Using cell phone and/or GPS location to determine
travel patterns is nothing new. But leave it to
Google to make it really easy - maybe too
easy. http//googleblog.blogspot.com/2009/08/brigh
t-side-of-sitting-in-traffic.html Adam
Shell Office of Systems Planning Iowa Department
of Transportation
Link
POA price of anarchy (30?) Nash equilibrium vs.
system optimality
OD data are destroyed! (privacy)
68
6.0 Traffic assignment 6.1 All or nothing -
adjusting link speeds will change assigned
volumes - initial speeds should be set to LOS C
speeds (0.87 x free flow speeds)
69
6.2 capacity restraint - volume f(time) -
final volume is average of all iterations or
later iterations can be weighted more heavily -
adjust free flow time or c (capacity) to change
volumes
IF THEN Link Speed Travel
Assigned Capacity Time Volume
70
6.2.1 definition of capacity design LOS C
(0.87c) ultimate LOS E (1.00c) parameters
differ depending on definition of capacity if
defined as LOS C, 0.15(v/c)4 if defined as LOS E,
0.80(v/c)4 (see HCM)
71
6.3 equilibrium - multiple paths may be selected
6.3.2 free speeds in systems with good
progression should be coded at about 1.1 times
the speed limit time
- more than 10 iterations may be needed for small
areas
72
7.0 Transit Ridership - for small/medium cities,
may not have to build a transit network - If not
using a transit network, can use the following
method (if trip generation includes transit
trips) 1. increase auto occupancy by transit
percentage (e.g. if auto occupancy is 1.05, then
change to 1.05 x 1.38 1.45) if transit
percentage is 38 2. decrease trip production or
attraction rates (one of them only, then balance)
if you use productions, can vary mode split by
income class 3. modify productions or attractions
by zone - get data from transit company -
adjust socioeconomic data or make direct P/A
adjustments
73
Mode Split Reasonableness Checks

• Mode split model?
• Form?
• Variables included in the utility functions?
• Coefficients logical?
• Value of time assumptions
• Parking cost assumptions
• How do mode shares change over time?
• Mode share comparisons
• with other cities

http//www.bts.gov/publications/journal_of_transpo
rtation_and_statistics/volume_08_number_02/html/pa
per_05/figure_05_03.html
74
Mode Split Calibration and Validation

• Experienced planning consultant required
• Form of LOGIT model
• Variables included in utility functions
• Calibration of coefficients for utility function
  variables
• Testing for IIA properties
• Analysis of household survey data
• Analysis of on-board transit survey data
• Calibration tasks we can do
• Compare highway and transit trips
• Total
• By purpose
• Compare Ridership by route
• CBD cordon line survey (if bus service is
  downtown only)

75

• 8. External stations
• - externals have no socioeconomic data
• - Ps and As are prepared by matching ground
  counts
• - I/E treated with the gravity model
• E/E
• - compare with Table 11 below

76

• System vs. local checks
• check 1. system wide (screenlines)
• 2. major movements (cutlines)
• 3. links
• if all screenlines are high or low, vary
• - auto occupancy
• - trip generation rates
• - trip lengths
• - intrazonal times - all zones
• - socioeconomic data - all zones

if corridor volumes are high or low, vary (for
zones affecting corridor) - auto occupancy -
trip generation rates - intrazonal travel times -
land use - centroid connectors - intersection
penalties if links are high/low, vary - speed -
intersection penalty - centroid locations -
special generators - local network configuration
77

• 10. Expected/Required accuracy
• We are concerned about errors that would require
  a design change (e.g. number of lanes)
• Note that ground counts also contain error
• Perfectly calibrated models produce link
  estimates with 1/3 above the standard error in
  ground counts and 2/3 below the standard error.
• Need ground counts for 65 of freeways and
  arterials, and a good sample from other facilities

From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
78

• 10. Expected/Required accuracy (cont.)
• The correlation coefficient should be greater
  than .88
• VMT estimate (region-wide) should be within 5
  (take care to compare same roads in systems)
• VMT/person should be 17-24 for large areas, 10-16
  for smaller areas (see also Table A7, next page)
• VMT/household should be 40-60 for large areas,
  30-40 for smaller areas

79
(No Transcript)
80
From CTRE Employment Data Project
81
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
82
Source Virginia Travel Demand Modeling Policies
and Procedures Manual
83
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
84
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
85
Source Virginia Travel Demand Modeling Policies
and Procedures Manual
86
From Minimum Travel Demand Model Calibration and
Validation Guidelines for the State of TN
87
Trip Assignment Reasonableness Checks

• All-or-nothing assignment
• study effect of increasing capacity
• Compare to Equilibrium assignment
• Check volume delay equation (BPR parameters)
• Compare
• screen line volumes
• Cut line volumes
• Time-of-day assignments?
• Source of factors
• Peak spreading used for future?
• If not, conversion factors source?
• (peak hour to 24-hour)
• Local VMT ( assigned to
• intrazonals and centroid connectors

Equil ibrium
All or Nothing
88
Trip Assignment Calibration and Validation
Assignment calibration performed
last

• Overall VMT or VHT check
• 40 to 60 miles per day per HH in large metro
  areas
• 30 to 40 miles per day per HH in medium metro
• /- 10 OK on screen lines
• Sign is important

89

• Compute by
• volume group
• facility type
• transit assignments
• time of day

90
(No Transcript)
91
12. TROUBLE SHOOTING
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
(No Transcript)
97
Other Factors Impacting Forecasted Travel Demand

• Can be implied in travel surveys (but not
  explicit)
• Telecommuting
• Flexible work hours
• HB business
• How to account for
• Aging population
• Internet shopping
• Roadway congestion (will it affect generation in
  the future)
• New modes

98
Issues for modeling

• Transferability of parameters
• More research is needed
• Forensic analysis
• How well did the models work?
• Confidence and Credibility
• How to improve
• Official versions vs. what-if models
• Integrity of the model
• Need more transparency, documentation,
  appropriateness of techniques