Probabilistic%20NAS%20Platform

1
Probabilistic NAS Platform

• George Hunter, Fred WielandBen Boisvert,
  Krishnakumar Ramamoorthy
• Sensis Corporation
• December 10, 2008

2
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

3
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

4
What is PNP?

• An fast-time and flexible NAS-wide simulation
  tool
• Real-time or fast-time modes
• Half-hour runtime on a laptop, to simulate a day
  in the NAS
• Physics-based trajectories computed through
  integrating aerodynamic energy balance equations
  by varying the time-step size
• System uncertainties (weather, security,
  operations )
• Plug-and-play architecture
• Dynamic clients (TFM, DAC, AOC, )
• An ATC community resource
• Formal software development processes in place
• Adaptable to current system or NextGen future
  concepts
• Uses
• Environment in which to design, build and test
  decision support tools
• TFM, DAC, AOC,
• Fast-time, real-time, shadow-mode
• Potential NAS tool
• Service provider, operator, collaborative uses
• Benefits assessment tool
• Fast-time tool to evaluate improved
  infrastructure, technology, procedures
• Evaluates historic and future traffic scenarios
  in weather

5
PNP Architecture
Graphical User Interface Plan View Display
NAS Database
Reports
Flight Data
NAS Simulation
Probabilistic NAS Platform (PNP)
MATLAB Scripting Interface
Weather Data
Performance Data
A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
6
PNP Architecture
Graphical User Interface Plan View Display
NAS Database
Reports
Flight Data
NAS Simulation
Probabilistic NAS Platform (PNP)
MATLAB Scripting Interface
Weather Data
Performance Data
SimObjects
MATLAB Client
Client As Middleware
Java Client
Decision making
Prob-TFM
A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
External Client (Any Language)
7
PNP Client Development

• TFM client development
• ProbTFM (Sensis internal development)
• TFM client integration
• C2 (algorithms from and used with permission of
  Bob Hoffman, Metron)
• Constrained LP (algorithms from and used with
  permission of NASA, Joey Rios) in progress
• DAC client integration
• MxDAC (algorithms from and used with permission
  of Min Xue, NASA/UARC)
• AOC client development
• Gaming behaviors (collaboration with GMU/Lance
  Sherry) in progress

8
Capabilities Summary
v

• Real-time
• Fast-time
• Airport weather impact models
• Airspace weather impact models
• Weather-integrated decision making
• Probabilistic modeling / decision making
• Traffic flow management
• Dynamic airspace configuration
• Surface traffic modeling
• Terminal area modeling
• Super density operations
• Fuel burn modeling
• Emissions modeling
• Trajectory-based operations
• Separation assurance
• Plug-n-play
• Fast run-time

v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
9
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
10
Team and Development History
People Envnment Data Functality Pr
ojects Users
Project
System
System lead
System
Software lead
Software
Software
Matlab
Java/real-time
Web 2.0
11
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
12
Example Project Uses

• JPDO Modeling and Analysis
• NextGen performance evaluation with weather
• FAA NASPAC Weather Modeling
• Convection impact modeling for NASPAC
• NASA Gaming NRA
• Evaluation of NextGen gaming with AOC clients
• NASA MetaSimulation NRA
• Investigation of TFM DAC interactions
• NASA SLDAST RFA
• Evaluation of NextGen TFM concepts and models
• NASA Market-Based TFM NRA
• Evaluation of NextGen market-based TFM concepts

13
NextGen Sensitivity Studies
George Hunter, Fred Wieland " Sensitivity of the
National Airspace System Performance to Weather
Forecast Accuracy," Integrated Communications,
Navigation and Surveillance Conference (ICNS),
Herndon, VA, May, 2008
Kris Ramamoorthy, George Hunter, "Evaluation of
National Airspace System Performance Improvement
With Four Dimensional Trajectories," AIAA Digital
Avionics Systems Conference (DASC), Dallas, TX,
October, 2007
14
Market-Based TFM Studies
George Hunter, et. al., "Toward an Economic Model
to Incentivize Voluntary Optimization of NAS
Traffic Flow," AIAA ATIO Conference, Anchorage,
AK, September, 2008.
15
Dynamic Airspace Configuration
George Hunter, "Preliminary Assessment of
Interactions Between Traffic Flow Management and
Dynamic Airspace Configuration Capabilities,"
AIAA Digital Avionics Systems Conference (DASC),
St. Paul, MN, October, 2008.
16
AOC Dispatch Use Case
17
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
18
Processes and Testing Cycle
19
Project Monitoring and Control

• JIRA is used to track issues
• Project Manager and Lead Software Engineer assign
  task priorities, due dates, and personnel.
• Weekly telecoms keep distributed team apprised of
  PNP and communications open
• Project Manager maintains a master schedule in
  MS-Project

20
Development Tracking

• Software engineers use JIRA to track and status
  development efforts.

21
Branch Configuration Management

• Software Engineers are responsible for creating
  branches from the trunk to develop
  fixes/enhancements.
• The Configuration Management of the software is
  accomplished with Subversion
• Subversion is an open source version control
  system (http//subversion.tigris.org/)

22
Unit and System Testing

• Software Engineers are responsible for creating
  unit tests to verify the correctness of their
  code. The JIRA issue number is to be used
  throughout the code and unit tests for tracking
  purposes.
• Software Engineers are responsible for running
  their own system/function tests to verify their
  software.
• Once testing is validated, code is merged back on
  to the trunk.

23
Trunk Configuration Management

• Once all validated JIRA issues are merged unto
  the trunk, regression testing is performed.

24
Regression Testing

• Regression testing
• Aggregate results
• Total delay
• Total congestion
• Traffic volume
• TFM initiatives
• Runtime
• Different scenarios
• Truncated demand set
• Full demand set
• Weather
• Automated
• Weekly or as required
• Archived
• Graphical quick-look

25
Quantitative Project Management

• Regression testing validation is performed and
  the release letter is updated.
• Release is tagged in Subversion.
• JIRA issues are closed.
• Documentation is updated to reflect changes in
  software.

26
Outline

• What is PNP?
• Team and development history
• Example uses of the model
• Software processes and testing
• Validation

A fast-time physics-based (trajectory-based)
NAS-wide modeling tool
27
System-Level Engineering Validation

• ASPM / ETMS verification tests
• Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field
  observations
• Models
• Trajectory data
• Airport capacities (VMC / IMC)
• Sector capacities in weather
• Aggregate performance
• Mean flight delay
• Sector and airport overloadings
• Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and
  temporal)
• Delays by airport and time of day
• Sector and airport loading by time of day
• Spatial loading patterns
• Light and heavy weather days

v
28
System-Level Software Verification

• Cross check sums
• SFlights SOperations at all airports
• SFlight time SMinutes from sector loads
• SSector load by sector SSector load by time
• SAirport ops SFlights using the airport in
  demand set
• SDelays by flight SDelays by time and
  reroutes
• Weather data checks
• Compare PNP/Metar airport capacity with ASPM
  AAR/ADR
• Compare PNP/Metar airport capacity with ASPM IFR
  periods
• Ensure SEn route convection versus time of day is
  smooth
• Ensure WxMAP MAP for all sector time bins
• Graphical
• Ensure reroutes overlaid on weather make sense
• TFM Performance
• Number of delays per flight, min and max flight
  delay
• Maximum airport and sector overloading (ensure
  are reasonable)

29
System-Level Engineering Validation

• ASPM / ETMS verification tests
• Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field
  observations
• Models
• Trajectory data
• Airport capacities (VMC / IMC)
• Sector capacities in weather
• Aggregate performance
• Mean flight delay
• Sector and airport overloadings
• Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and
  temporal)
• Delays by airport and time of day
• Sector and airport loading by time of day
• Spatial loading patterns
• Light and heavy weather days

v
30
Trajectory Model Validation

• Compared to ETMS flight data (May 2008)

George Hunter, Ben Boisvert, Kris Ramamoorthy,
"Advanced Traffic Flow Management Experiments for
National Airspace Performance Improvement," 2007
Winter Simulation Conference, Washington, DC,
December, 2007
31
ProbTFM Performance

• ASPM / ETMS verification tests
• Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field
  observations
• Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and
  forecasted
• Sector capacities in weather, actual and
  forecasted
• Aggregate performance
• Mean flight delay
• Sector and airport loadings
• Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and
  temporal)
• Delays by airport and time of day
• Sector and airport loading by time of day
• Spatial loading patterns
• Light and heavy weather days

v
32
Compare With Field Observations

• Compare to ETMS/ASPM
• Forecast accuracies, Decision making horizon,
  Delay distribution

33
Verification of Results

• ASPM / ETMS verification tests
• Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field
  observations
• Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and
  forecasted
• Sector capacities in weather, actual and
  forecasted
• Aggregate performance
• Mean flight delay
• Sector and airport loadings
• Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and
  temporal)
• Delays by airport and time of day
• Sector and airport loading by time of day
• Spatial loading patterns
• Light and heavy weather days

v
34
System Loading Patterns

• ProbTFM predicted, 1445 GMT

ETMS Actual, 1445 GMT
ETMSUnderloading Overloading
ProbTFM loading
35
Verification of Results

• ASPM / ETMS verification tests
• Compare ASPM/ETMS data with simulation data
• Calibrate concept to match aggregate field
  observations
• Models
• Trajectory data
• Airport capacities (VMC / IMC), actual and
  forecasted
• Sector capacities in weather, actual and
  forecasted
• Aggregate performance
• Mean flight delay
• Sector and airport loadings
• Detailed performance
• Flight delay by airport and time of day
• Overloading and delay patterns (Spatial and
  temporal)
• Delays by airport and time of day
• Sector and airport loading by time of day
• Spatial loading patterns
• Light and heavy weather days, control days

v
36
Conclusion

• The development of PNP has benefited from lessons
  learned over past two decades in NAS system wide
  modeling
• Plug and play simulation architecture
• Supports both analytical and HITL studies
• Adaptable to simulate current system as well as
  NextGen future concepts
• Fast-time, physics-based
• Formal software development processes in place
• Probabilistic decision making and extensive
  weather modeling explicitly incorporated in tool

37
Publications

1. George Hunter, "Preliminary Assessment of
   Interactions Between Traffic Flow Management and
   Dynamic Airspace Configuration Capabilities,"
   AIAA Digital Avionics Systems Conference (DASC),
   St. Paul, MN, October, 2008.
2. George Hunter, et. al., "Toward an Economic Model
   to Incentivize Voluntary Optimization of NAS
   Traffic Flow," AIAA ATIO Conference, Anchorage,
   AK, September, 2008.
3. George Hunter, Fred Wieland " Sensitivity of the
   National Airspace System Performance to Weather
   Forecast Accuracy," Integrated Communications,
   Navigation and Surveillance Conference (ICNS),
   Herndon, VA, May, 2008.
4. George Hunter, Kris Ramamoorthy, "Integration of
   terminal area probabilistic meteorological
   forecasts in NAS-wide traffic flow management
   decision making," 13th Conference on Aviation,
   Range and Aerospace Meteorology, New Orleans, LA,
   January, 2008.
5. Kris Ramamoorthy, George Hunter, "The Integration
   of Meteorological Data in Air Traffic Management
   Requirements and Sensitivities," 46th AIAA
   Aerospace Sciences Meeting and Exhibit, Reno, NV,
   January, 2008.
6. George Hunter, Ben Boisvert, Kris Ramamoorthy,
   "Advanced Traffic Flow Management Experiments for
   National Airspace Performance Improvement," 2007
   Winter Simulation Conference, Washington, DC,
   December, 2007.
7. Kris Ramamoorthy, George Hunter, "Evaluation of
   National Airspace System Performance Improvement
   With Four Dimensional Trajectories," AIAA Digital
   Avionics Systems Conference (DASC), Dallas, TX,
   October, 2007.
8. Kris Ramamoorthy, Ben Boisvert, George Hunter,
   "Sensitivity of Advanced Traffic Flow Management
   to Different Weather Scenarios," Integrated
   Communications, Navigation and Surveillance
   Conference (ICNS), Herndon, VA, May, 2007.
9. George Hunter, Ben Boisvert, Kris Ramamoorthy,
   "Use of automated aviation weather forecasts in
   future NAS," The 87th American Meteorological
   Society Annual Meeting, San Antonio, TX, January,
   2007.
10. Kris Ramamoorthy, George Hunter, "Probabilistic
    Traffic Flow Management in the Presence of
    Inclement Weather and Other System
    Uncertainties," INFORMS Annual Meeting,
    Pittsburgh, PA, November, 2006.
11. Kris Ramamoorthy, Ben Boisvert, George Hunter, "A
    Real-Time Probabilistic TFM Evaluation Tool,"
    AIAA Digital Avionics Systems Conference (DASC),
    Portland, OR, October, 2006.
12. George Hunter, Kris Ramamoorthy, Alexander Klein
    "Modeling and Performance of NAS in Inclement
    Weather," AIAA Aviation Technology, Integration
    and Operations (ATIO) Forum, Wichita, KS,
    September 2006.
13. Kris Ramamoorthy, George Hunter, "A
    Trajectory-Based Probabilistic TFM Evaluation
    Tool and Experiment," Integrated Communications,
    Navigation and Surveillance Conference (ICNS),
    Baltimore, MD, May, 2006.
14. Kris Ramamoorthy, George Hunter, "Avionics and
    National Airspace Architecture Strategies for
    Future Demand Scenarios in Inclement Weather,"
    AIAA Digital Avionics Systems Conference (DASC),
    Crystal City, VA, October, 2005.
15. George Hunter, Kris Ramamoorthy, Joe Post,
    "Evaluation of the Future National Airspace
    System in Heavy Weather," AIAA Aviation
    Technology, Integration and Operations (ATIO)
    Forum, Arlington, VA, September 2005.
16. James D. Phillips, An Accurate and Flexible
    Trajectory Analysis, World Aviation Congress
    (SAE Paper 975599), Anaheim, CA, October 13-16,
    1997.

38
Questions?
39
Backup
40
PNP Systems Requirements

• System requirements
• PNP is a Java application
• Hardware
• Memory minimum 1GB, preferred 2GB
• CPU Pentium (4) 3.2 GHz or better
• Video card 128MB memory, preferred 256MB
• Software
• Java JDK 6 http//java.sun.com/javase/downloads/in
  dex.jsp
• MySQL Server 5.0 http//dev.mysql.com
• Third party licenses
• Eurocontrol BADA usage license

41
Weather Days

• Ten weather days, two control days

42
Weather Days

• Weather days
• Spectrum of weather days
• Variation in weather type and intensity
• Variation in season
• Support real-world comparison
• Support same sector data
• Variation in traffic demand volume and structure
• Different days of week, holidays
• Control days

43
(No Transcript)
44
NextGen PerformanceSensitivity Analysis
45
En Route and Terminal Area Combined Sensitivities
- 2025
46
(No Transcript)
47
Benefit of ImprovedConvection Forecasts
48
Investment Analysis
49
(No Transcript)
50
Benefit of Using Clear Weather Forecasts
51
Benefit Evaluation
Case 2 No distinction between clear and heavy
weather forecast accuracy
Case 1 Take advantage of improved forecast
accuracy in clear weather
Persistence forecast, 11/16/06
52
(No Transcript)
53
Market-Based TFMValuation of NAS Access
54
Congestion-Delay Relationship

• Unconstrained sector congestion cost (SCC) for
  zero lookahead time (blue) and PNP-ProbTFM
  simulated delay (black) time histories for all en
  route NAS sectors and flights, respectively.

Delay
SCC
55
Aggregate Delay Model

• Hypothesize a first-order lag transfer function

Simulated delay
Modeled delay
56
Aggregate Delay Model

• Hypothesize a second-order transfer function

Simulated delay
Modeled delay
57
Transfer Functions Summary
58
Explicit Cost Model

• Evaluate cost of NAS access by removing the
  flight
• Remove one flight
• 11/16/06, UAL233, A320
• Morning departure from Bradley International
  (KBDL) to Chicago OHare airport (KORD)
• Relatively high cost flight
• 90.02 SCC

59
Remove UAL233

• Delay reduction by time bin in simulation run
• Delay reduction of 8141 minutes

60
(No Transcript)
61
NAS Performance Sensitivity Studies

• Performance sensitivity to
• Delay distribution policy (most important factor)
• TFM system agility
• System forecasts (least important factor)

62
(No Transcript)
63
Dynamic Airspace Configuration
64
NAS Sectorization

• Nov 12, 2006

65
MxDAC Afternoon Sectorization

• Nov 12, 2006, LAT6, Gen20

66
MxDAC Midday Sectorization
Coeff_peak_ac_var0.0 Coeff_avg_ac_var0.0 Coeff_c
rossings0.0 Coeff_transition_time0.0 Coeff_resid
ual_capacity1.0

• Nov 12, 2006, LAT2, Gen40

67
Delay-Congestion Performance
68
(No Transcript)
69
Equity AnalysisCost of Delay Distribution
70
Cost of Distributing Delay

• RMS delay can be reduced by spreading delay to
  more flights
• But at the cost of increased total delay

71
(No Transcript)
72
AOC Dispatch Use Case
73
Dispatcher Successfully Finds a Reroute
74
(No Transcript)
75
Project Monitoring and Control

• JIRA is used to track issues
• Project Manager and Lead Software Engineer assign
  task priorities, due dates, and personnel.
• Weekly telecoms keep distributed team apprised of
  PNP and communications open
• Project Manager maintains a master schedule in
  MS-Project

76
Development Tracking

• Software engineers use JIRA to track and status
  development efforts.

77
Branch Configuration Management

• Software Engineers are responsible for creating
  branches from the trunk to develop
  fixes/enhancements.
• The Configuration Management of the software is
  accomplished with Subversion
• Subversion is an open source version control
  system (http//subversion.tigris.org/)

78
Unit and System Testing

• Software Engineers are responsible for creating
  unit tests to verify the correctness of their
  code. The JIRA issue number is to be used
  throughout the code and unit tests for tracking
  purposes.
• Software Engineers are responsible for running
  their own system/function tests to verify their
  software.
• Once testing is validated, code is merged back on
  to the trunk.

79
Trunk Configuration Management

• Once all validated JIRA issues are merged unto
  the trunk, regression testing is performed.

80
Regression Testing

• Regression testing
• Aggregate results
• Total delay
• Total congestion
• Traffic volume
• TFM initiatives
• Runtime
• Different scenarios
• Truncated demand set
• Full demand set
• Weather
• Automated
• Weekly or as required
• Archived
• Graphical quick-look

81
Quantitative Project Management

• Regression testing validation is performed and
  the release letter is updated.
• Release is tagged in Subversion.
• JIRA issues are closed.
• Documentation is updated to reflect changes in
  software.

82
Risk Management

• Lessons learned analysis
• A wrap up meeting is held to discuss all issues
  on a project in which proactive steps can be
  documented to avoid the same mistakes