Performance Metrics in ATM

1

• Performance Metrics in ATM
• Defining an achievable level of safety and
  setting performance targets
• Tony LICU
• Safety Regulation Unit
• EUROCONTROL

2
Summary

• Current work being undertaken by SRU/SRC
• Harmonised definition of safety performance
  criteria
• Data Gathered from across ECAC
• Conclusions from analysis
• Defining your key indicators
• Recommendations on performance targets

3
EUROCONTROL Institutional structure
Civil/Military Interface Committee
Audit Board
4
Safety Regulation Commission Work Programme
5
Safety Regulation Commission Objectives
TO PRODUCE SAFETY REGULATORY
REQUIREMENTS FOR ATM ACROSS THE ECAC REGION
TO CO-ORDINATE REQUIREMENTS IMPLEMENTATION ACR
OSS THE ECAC REGION
TO ESTABLISH A PROCESS TO MEASURE SAFETY
PERFORMANCE AND IDENTIFY ATM KEY RISK AREAS
6
Proposed Approach

• What do we need Safety Performance Indicators
• targets for?

• How do we set up accurate and effective Safety
• Indicators and safety targets?

• What are the current ATM safety levels in Europe?

7
Objectives Safety Improvement

• To monitor safety levels and trends in ATM

• To improve overall aviation safety

• To assess and monitor changes in ATM

• To assess the use of Good Working practices

• To assess if safety is acceptable(social,politic
  al)

8
EUROCONTROL Performance Review Reports (PRRs)
The lack of consistency and availability of
safety data at the European level has prevented
meaningful conclusions being drawn on the
performance of ATM safety across the ECAC
area. The use of national summary data, in
aggregated and unattributable form, is essential
to future performance assessment and decision
making in the safety area.
9
Experience issues so far in ATM
10
EUROCONTROL Initiatives Safety Measurement
and Improvement Programme (1)
EUROCONTROL Safety Regulatory Requirement
Reporting and Assessment of Safety Occurrences
in ATM (Includes reporting annual statistics to
EUROCONTROL) ESARR 2
Safety Occurrence Severity Classification
Scheme (Guidance Material)
Confidentiality and Publication Policy (Set Of
Principles )

• Non-punitive/just-punitive Environment

11
EUROCONTROL Initiatives Safety Measurement and
Improvement Programme (2)
12
(No Transcript)
13
What criteria or elements would you consider for
the purpose of classifying an occurrence as an
accident ?
1
2
3
4
5
7
6
14
Persons on board at time of occurrence?
15
(No Transcript)
16
(No Transcript)
17
There is a need for detailed agreed definitions
18
Tool 1
Tool 2
SOFIA
Tool 4
Tool 3
Tool 5
Tool 6
Tool 7
19
ESARR 2
IMPLEMENTATION By when?
ACCIDENTS
INCIDENTS

• January 2000
• Accidents/Incidents in terms of near collisions
• January 2001
• Incidents in terms of potential for near
  collisions
• January 2002
• ATM specific occurrences
• WITH ATM contribution/High level causes

ATM Specific OCCURRENCES
2- If another aircraft, collision or near
collision
20
ATM Safety Performance in ECAC
21
ATM Safety Performance in ECAC contd
22
National Safety Levels

• Amendment 40 to ICAO Annex 11
• ECAC safety objective of the ATM 2000 strategy
• to improve safety level by ensuring that the
  number of ATM induced accidents and serious or
  risk-bearing incidents do not increase, where
  possible, decrease
• SRC POL DOC 1 - ECAC Safety Minima
• scope limited to ATM direct contribution to the
  accidents
• reliable ECAC-wide statistics on accidents with
  ATM indirect contribution are essential
• States have the task of developing national ATM
  safety levels/minima to meet provisions of ICAO
  Annex 11 and SRC POL DOC 1

23
EUROCONTROL Initiatives SRC POL DOC 1 ECAC
Safety Minima for ATM
24
ECAC Safety Minima for ATM
(According to ICAO ADREP Statistics)
(According to app. 2 of ATM direct
contribution to accidents)
(According to CFMU data for 1996-1999)
(According to max. STATFOR predicted increase of
traffic volume by 6,7 per year)
25
Relative Minima
Absolute Minima
ECAC Safety Minimum-ATM Absolute Number
4,38 10-8
0,623
ESARR 4- 1,55. 10-8 (Stepped increase assuming
6,7 annual traffic increase )
1,55 10-8
Year
2004
1999
2015
26
CONCLUSIONS (1)

• There is a need to have a better ECAC-wide
  identification of safety issues/key risk areas
• No overall safety performance conclusions at ECAC
  level are yet possible because of a lack of
  safety data, and inconsistencies of safety data
  reports, from States
• The time required at national level for the
  analysis of ATM related incidents, and for
  reporting feed back to those who have reported,
  is too long

27
CONCLUSIONS (2)

• Reliable ECAC-wide accident and incident
  statistics, including ATM contribution, are
  essential in assessing potential safety trends,.
• A limited number of qualified potential safety
  issues could be identified from the SRC
  assessment, the most prominent of which relate to
• an increase in runway incursions
• an increase in collisions on the ground between
  aircraft and vehicle(s) /person(s)/obstacle(s),
  and
• the identification of a high number of
  unauthorised penetrations of airspace, which are
  disproportionately higher for Operational Air
  Traffic (OAT), when compared to total number of
  flights

28
Recommendations on performance targets

• To share key risk areas/remedial actions

• To specify objectives of SIs/PTs

• To secure reliability of safety data

• To adopt common sense approach to scope

• To recognise limits of SIs/PTs

29
SRC/SRU Web Address
http//www.eurocontrol.int/src
30
Summary

• Current work being undertaken by SRU/SRC
• Harmonised definition of safety performance
  criteria
• Data Gathered from across ECAC
• Conclusions from analysis
• Defining your key indicators
• Recommendations on performance targets

31

• Performance Metrics in ATM
• Defining an achievable level of safety and
  setting performance targets
• Tony LICU
• Safety Regulation Unit
• EUROCONTROL

QUESTIONS ?