EUROCONTROL: CARE Innovative Actions

1

• EUROCONTROL CARE Innovative Actions
• Managing Uncertainty between Controllers and
  Pilots the Presentation of Uncertain
  Information
• FINDINGS AND PROPOSALS FOR FURTHER WORK
• David Nicholls, RM Consultants Ltd
• 24th January 2002

2
OUTLINE OF PRESENTATION

• 1 Introduction - the preliminary study
• 2 Overview of proposed Work Packages
• 3 WP1 Presenting uncertainty in the complexity
  of future traffic patterns (TLS)
• 4 WP2 Presenting the uncertainty of conflict
  risk in Free Flight (CARE/ ASAS)
• 5 WP3 Lessons from Military experience for Free
  Flight
• 6 Timescales

3
1. INTRODUCTION - THE PRELIMINARY STUDY

4
Why does uncertainty matter?
5
Main Areas of Uncertainty

• Uncertainty is particularly evident in
• Predictive tools (e.g STCA, TCAS, GPWS)
• Free Flight ... (more decision makers, more
  collaborative decisions)
• The potential for uncertainty is likely to
  increase as these become more widespread, and as
  traffic grows and other new systems are introduced

6
Why present uncertainty?

• Where uncertainty
• cannot / should not be eliminated
• is foreseeable (bounded)
• and matters to safety or efficiency .
• giving pilots and controllers information about
  uncertainty may assist decision-making

7
Preliminary study Objectives Scope

• Objectives
• Identify where uncertainty occurs in ATM
• Develop initial guidance on presentation
• Identify where further research could benefit
  safety or efficiency
• Scope
• Uncertainty is a broad term, covering
  variability, inaccuracy, lack of knowledge,
  likelihood of events, degree of belief ...
• Consider presentation of uncertainty, not
  reduction at source or uncertainty in verbal
  communication

8
Preliminary study Conduct

• Literature surveys, experience from past ATM
  projects in ATM and other domains, incident data,
  anecdotal evidence, structured interviews,
  discussion with pilots
• Partners RMC, University of Glasgow,
  independent associates

9
Preliminary study Findings (1) importance of
uncertainty

• Several areas in which presenting uncertainty
  could improve safety and efficiency
• An appropriate perception of uncertainty is
  essential to good decision making
• Some uncertainty is necessary to maintain
• interest, arousal, attentiveness,
• situation awareness - keeping in the loop
• cognitive function (need to make errors)
• enjoyment, pride in the job

10
Preliminary study Findings (2) - current
presentations

• Uncertainty is rarely presented explicitly
• As with other types of information, only the
  visual channel is extensively used
• Remarkably little research applicable to ATM
• Current ATM approach relies mainly on eliminating
  uncertainty

we dont have uncertainty in ATM
11
Preliminary study Outputs

• High level principles (e.g. allow for individual
  variations in attitudes to uncertainty)
• Tentative model of influences on attitudes and
  cognitive models
• Framework to assist in applying principles
• define the uncertainty
• decide whether, and if so to whom, to present it
• select a presentational form, develop and
  evaluate prototypes
• Catalogue of potential presentational forms

12
Presentational forms categories

• Sensory channels visual/ aural/ tactile/
  combined
• potential for greater use of auditory channel,
  especially if datalink reduces routine R/T
  communication
• Symbology abstract/ iconic/ mathematical/
  semantic
• could the greater expressive range of artistic
  (iconic) forms outweigh their culture-dependence?

13
Presentational forms a few examples
Graphic
Mathematical
95

• Musical
• consonance and dissonance
• cadences - resolved, unresolved
• tonality major/ minor
• tonal quality

Semantic
certain - unsure - very uncertain
14
Need for further work

• Uncertainty is inevitable, and may increase
• Appropriate perceptions of uncertainty are
  essential to safe and efficient decisions
• but ..
• There is little relevant research
• There may be scope to exploit a wider range of
  presentational forms
• so ...
• 11 possible topics for research identified

15
2. OVERVIEW OF PROPOSED STUDIES

16
Selecting the research topics

• Overall objectives address gaps by applying
  guidance to test cases. Hence
• refine guidance
• add value to the test cases
• Topics prioritised with regard to
• Co-operation
• Innovation
• Added value to EUROCONTROL projects
• Addressing Europe-wide safety/ capacity issues
• Tangible benefits rather than long term
  theoretical development (but address some big
  questions)

17
Proposed WPs

• WP1 Presenting uncertainty in the complexity of
  future traffic patterns (Tactical Load Smoother -
  TLS)
• WP2 Presenting the uncertainty of conflict risk
  in Free Flight (CARE/ASAS)
• WP3 Lessons from military experience for Free
  Flight

18
Research Team
risk management consultants
Lead consultant aviation risk management,
uncertainty
With input from
EUROCONTROL Experimental Centre (EEC) Knowledge
of TLS, prototype development and evaluation (WP1)
19
3. WP1 PRESENTING UNCERTAINTY IN THE COMPLEXITY
OF FUTURE TRAFFIC PATTERNS

• Alternative Interfaces for the Tactical Load
  Smoother

20
WP1 Rationale

• The TLS shows a multi-sector planner when and
  where traffic is likely to become excessively
  complex for sector controllers
• The TLS is
• one of the few examples of an ATM interface which
  explicitly deals with uncertainty
• an example of the challenges of time-displaced
  information
• an opportunity to apply the guidance on
  uncertainty presentation to an ongoing
  development for which EUROCONTROL have a need for
  interface development

TLS Complexity Map (from Meckiff et al, 1998)
21
WP1 Objective

• Develop, apply and test the guidance on
  uncertainty presentation in the context of the
  TLS
• In particular
• Devise and evaluate prototype presentations of
  the future traffic pattern which
• are intuitively clear
• but not easily confused with the current traffic
  pattern

22
WP1 Scope

• Feed ideas into EECs own development of the TLS
  HMI, potentially suitable for immediate
  implementation
• Develop alternative, more radical presentations,
  using TLS as a case study
• Prototypes will be developed and evaluated to the
  level at which EUROCONTROL can make an informed
  decision on whether to proceed to integration and
  formal user testing

23
WP1 Overview of Method
Identify and define the uncertainties causes,
effects, dialogues, cognitive models attitudes
Operational context and uses (info from EEC)
For each uncertainty

Decide whether to present uncertainty
Generate and short list ideas for presentation ...
Review point
Develop and evaluate prototypes ...

• Deliverables
• Evaluation of prototype presentations
• Conclusions for uncertainty presentation in
  general

24
decide whether to present each uncertainty

• Some advantages
• more comprehensive picture for decision-making
• improved situation awareness
• better ability to detect when system is straying
  outside the normal limits of uncertainty
• reduced volume of information (trends)
• job satisfaction

• Some disadvantages
• may increase cognitive load
• may increase overall uncertainty in system
  individual judgments and responses may vary
• increased potential for confusion between parties

25
generate and short-list ideas

• Input/ inspiration criticism free processes
• ideas from preliminary study
• concepts from Deep Blues studies
• brainstorming and workshops (Deep Blue methods)
  
• other creativity techniques

• Initial review - consider factors such as
• which sensory channel?
• iconic or abstract?
• alphanumeric vs graphical?
• common or different needs of users?

Divide into ideas for immediate implementation
and more radical, longer term possibilities
Review point - which ideas to develop, how, who by
26
develop and evaluate prototypes
Radical alternatives Scenario-based evaluation
(Deep Blue)
Immediate implementation EEC eDEP, HF lab,
quantitative measures
Complementary approaches

• User observation and activity analysis
• Concept generation, rapid prototyping, mock-ups
  computer animation with interaction
• Evaluation

• Deliverables
• Evaluation of prototype presentations informed
  basis for EUROCONTROL decisions
• Analysis and conclusions for uncertainty
  presentation in general

27
4. WP2 PRESENTING THE UNCERTAINTY OF CONFLICT
RISK IN FREE FLIGHT

28
WP2 Rationale - why Free Flight?

• Collaborative Decision Making - need to share
  mental models

• More decision-makers
• Additional system states (MAS/FFAS) and rules
• Variation in aircraft equipment fits

29
WP2 Rationale (contd) - benefits of the
CARE/ASAS link?

• Presentation and HMI issues are not yet heavily
  constrained
• Air and ground are both involved
• Hazard analyses have identified uncertainty
  issues - in particular the complex relationship
  between time, space and risk .
• Defining Separation Minima requires understanding
  of uncertainty, including the effects of
  presenting it to users

30
time, space, risk and uncertainty
Max risk - still time for aircraft to deviate,
not enough time to intervene
CPA
Risk of collision
Uncertainty
Separationminimum?
Separation
Time
31
WP2 Objective

• Devise and evaluate prototype presentations which
  convey an appropriate awareness of uncertainty in
  the risk of conflict or collision, i.e. which
  enable
• controllers to set appropriate Separation
  Objectives, within the overall Separation Minima
• pilots to make appropriate decisions, within the
  limits on their autonomy

32
WP2 Scope

• Develop evaluate to point at which EUROCONTROL
  can make an informed decision on whether to
  proceed
• Scenarios (based on ASAS definitions) could
  include
• Autonomous Operation
• Partial delegation (CSA) for in-trail sequencing
• Transitions between MAS and FFAS

FFAS
MAS
33
WP2 Method

• As WP1, with changes of emphasis
• use ASAS Hazard Analyses to define causes and
  effects of uncertainty
• importance of deciding who receives information
  about uncertainty pilot(s)/ controller(s)/ all/
  some/ none?

34
WP2 Deliverables

• Evaluated prototype presentations for CARE/ASAS
• Feedback to definition of Separation Minima
• Analysis and conclusions relating to uncertainty
  presentation in general
• Comment on issues relating to job design of FFAS
  controller

35
5. WP3 LESSONS FROM MILITARY EXPERIENCE FOR
UNCERTAINTY MANAGEMENT IN FREE FLIGHT

36
WP3 Rationale, Objective, Scope

• Current Military ATM has some similarities with
  Free Flight
• pilots have more autonomy, locus of control may
  shift dynamically, there is collaboration between
  pilots
• aircraft tend not to follow fixed routes
• operations are inherently more uncertain and
  variable
• So, are there lessons for civil Free Flight?
• small-scale, open-ended study
• emphasis on presentation, but may lead to other
  ideas

37
WP3 Methods

• Define example scenarios for discussion (e.g.
  from WP2)
• Interviews with military/ civil pilots and
  controllers - compare attitudes to uncertainty
• Incident analysis - where has uncertainty
  mattered, how is it reported, where might
  presentation have helped?
• Compare treatment of uncertainty in military/
  civil selection and training

38
WP3 Deliverables

• Identify areas where crossover from military
  practice/ knowledge could benefit civil Free
  Flight
• Refine the model of influences on attitudes and
  cognitive models, as applied specifically to Free
  Flight, and feed back to WP2
• Suggest other work - e.g. experimental comparison
  of civil and military attitudes to uncertainty

39
6. OVERALL PROJECT TIMESCALES

40
Timescale
2002
2003
WP1 (TLS)
WP2 (Free Flight)
WP3 (Military-Civil)
review point