Micro%20Simulation%20Peer%20Review%20Process

1
Micro Simulation Peer Review Process

• Paramics User Group Meeting
• July 15th, 2008
• Presented by
• Matthew Juckes
• in Association with
• Richard Braidwood
• and
• John Shaw

2
Micro Simulation Peer Review Process

• What is a Peer Review?
• The process of subjecting an author's scholarly
  work, research or ideas to the scrutiny of others
  who are experts in the same field. Peer review
  requires a community of experts in a given (and
  often narrowly defined) field, who are qualified
  and able to perform Impartial review.
  (wikipedia.org)
• Impartial Review
• Underutilized process in transportation modeling.

3
Why the need for Peer Reviews

• Ascertain whether the model is fit for its
  purpose.
• Avoid going down the wrong path
• Identify issues while there is still time to
  react
• Insure model has followed standard practices
• Extra set of eyes, (avoids Tunnel Vision)
• Provides confidence in the model

4
Who conducts the Review

• Internal
• In house check (no direct project involvement)
• External
• External Consultant as chosen by the client at
  the start of a project
• Agencies need to set up standard practice to
  avoid conflict of interest with reviewers
• Client / Local Authority
• Member of the client with experience with the
  software package being used

5
Model Process Base Model
Model/Project Scope
Network Development
Data Collection
Matrix Development
Network Calibration/ Validation
Base Network Completion
6
Model Process Design Year/Do Minimum Model
Base Network
Design Year Matrix
Design Year Growth Rate
Committed Do-Min Project Coding
Calibration Check
Do-Min Network Completion
7
Model Process Design Year/Build Scenarios
Do-Min Network
Calibration Check
Code Scenarios
Compute MOEs and Results
Model Completion and Report
8
Review Process

• Two levels of review
• Report Card / Managerial Review
• Summary of review results
• Less detailed more global (for example Good, Ok,
  bad ratings for different categories in the
  review)
• Lets PMs know the state of the model
• Technical Review
• Tech Memo
• High level of detail
• For use by model team to address issues and when
  needed make changes to the model

9
Review Process Report Card

• Agency Standardized Report Card
• Grading system
• A, B, C, D, E and F
• Green, Yellow, and Red
• Good, Ok, Poor
• Grade for each category from the technical
  reviews
• short summary report
• List of suggested actions needed
• Reviewer sign off on the model

10
Review Process Report Card

• Categories
• Network Coding
• Intersection Traffic Control
• Zone Structure
• O-D Matrices, Demand Profiles and Time Periods
• Core Simulation Parameters and Configuration

11
Review Process Report Card

• Categories
• Closures, Restrictions and Incidents
• Routing/Assignment Parameters
• Visibility and Awareness Distance
• Special Features and Plug-ins

12
Review Process Report Card

• Categories
• Calibration
• Consistency with Related Models
• Documentation and Results
• Conclusion

13
Review Process Report Card

• Sample of Report Card

14
Review Process Technical Process

• Base Model
• Configuration
• Review of configuration set up and configuration
  variables (varies by platform)
• Any changes from Default values should be noted
• Network Coding
• Nodes Placement issues should be noted
• Links Are there any anomalies, inconsistencies?
• Junctions Check Junctions for issues, this may
  range from poorly operating intersections to
  incorrectly coded movements

15
Review Process Technical Process

• Base Model (cont)
• Network Coding
• Stoplines/Nextlanes Insure that stoplines,
  nextlanes have been probably set up to provide
  for proper vehicle transitions over nodes
• Lane Utilization Insure that vehicles are using
  the correct lanes for movements
• Hazards/Sign Posting When hazards are present
  are sign post distances sensibly set?
• General Model Issues when running the model, do
  things visually look correct?

16
Review Process Technical Process

• Base Model (cont)
• Traffic Demand
• Vehicle Types
• Does the vehicle type distribution make sense and
  meet the needs of the project for both the base
  and future models
• Have the levels of familiar drivers and
  perturbation been correctly coded (Paramics based
  issue)
• Demand Matrices
• Do the demands files make sense, Number of
  matrices/trip distribution/no illogical pairs
• Have the matrices been developed using an
  accepted practice
• Have profiles been created and if so do they seem
  logical and match field data

17
Review Process Technical Process

• Base Model (cont)
• Traffic Demand
• Time Periods
• Are the time periods correctly set up to meet the
  needs of the project, has sufficient loading time
  been given
• Advanced Model Apps
• If any plug-ins or actuated signals have been
  used have they been used correctly and do they
  raise any issues?
• Validation
• Have the Validation targets been met?
• General observations and Conclusions

18
Review Process Technical Process

• Do Minimum Model
• Growth Rates
• Were the appropriate growth rates applied to the
  existing matrices
• Design Year Matrices
• Are the design year matrices consistent with the
  base year matrices
• Committed/Do-Min Coding
• Have all committed projects been added?
• Have Do-Min improvements been justified?

19
Review Process Technical Process

• Do Minimum Model
• Configuration
• Are there any changes in the configuration and if
  so are they justified
• Network Coding
• Have any none committed/Do-Min changes been made
  to the network
• General Observations and Conclusions

20
Review Process Technical Process

• Design Year/Build Scenarios
• Design Year Matrices
• Are the matrices the same as the Do-Min
• Configuration
• Are there any changes in the configuration and if
  so are they justified
• Network Coding
• Have scenarios been properly coded
• Are there any changes to the network that are not
  associated to the scenario
• MOEs and Model Conclusions
• Have the MOEs been correctly calculated
• Do the conclusions make sense
• General Observations and Conclusions

21
Review Process Paramics Tools

• Visual Audit
• Queues
• Lane Movements
• Stuck Vehicles

Simulation Run
Next Lanes
22
Review Process Paramics Tools

• Network Audit Tool

By Link
By Node
23
Review Process Paramics Tools

• Demands Routing

Cost Factors
Demands
Routes
24
Review Process Validation Results

• Review of Documentation and Tables

25
Conclusions

• Benefit to both Client and Consultant
• Set up as part of the project schedule
• Impartial Process a Key
• Standardized Process
• Usable comments

26
Further Information

• For further information
• Matthew Juckes
• www.stumphausman.com
• juckes_at_stumphausman.com
• 732.767.0103

27
The End

• Thank you for your time.