A Glimpse at Archie: The LOSA Archive

1
A Glimpse at Archie The LOSA Archive

• Ashleigh Merritt, Ph.D.
• The University of Texas Human Factors Research
  Project (UT)
  
• Third ICAO/IATA LOSA/TEM Conference
• September 13-14, 2005, Kuala Lumpur

2
What is Archie?

• Archie is the LOSA Archive
• A database of de-identified airline data which is
  maintained and updated by UT
  
• Airlines that do a LOSA with The LOSA
  Collaborative contribute their data to Archie for
  research purposes

3
How big is Archie?

• As of October 2004, Archie had data from
• 15 airlines
• 3,309 flights
• 11,000 threats
• 7,000 errors
• 1,500 undesired aircraft states

4
How old is Archie?

• Five LOSAs in five years to develop the
  methodology
  
• Data in the Archive starts in 2000
• Had to wait for Archie to grow before doing any
  industry-wide analyses.
  
• If we think of Archie as a dog, he would be a
  young bloodhound

5
How big is Archie?

• As of August 2005, Archie has data from
• 20 airlines (including 3 repeat LOSAs)
• 4800 flights
• 4800 flight narratives written by trained
  observers
  
• 4800 ratings of TEM countermeasure performance
• 17,500 threats
• 12,500 errors
• 2,400 undesired aircraft states

6
Archies Attributes 2004

• Archie is loyal and faithful
• Archie alerts of danger
• Archie is a tracker
• Archie plays well with others

7
2005 Update on Archie

• Archie has remained loyal and faithful to its
  aviation master
  
• No airline has been publicly identified
• No individual pilots can be or ever have been
  identified

8
Archie continues to play well with others

• Queries from / data-sharing with
• NOSS (ATC group)
• IATA/ICAO (ITA)
• Boeing
• Airlines
• ASAP
• ATSB

9
2005 Archie is learning to track

• Airline threats
• ATC threats
• Intentional Noncompliance

10
Airline threats Simple Tracking

• Airline threats include aircraft malfunctions,
  time pressure, errors/distractions/interruptions
  from Cabin, Dispatch, Maintenance, Ground, Ramp.
  
• Airline threats occur on 2/3 of all flights,
  adding to overall workload.
  
• Range 55 - 81 across different airlines
• Three-quarters of airline threats occur before
  the aircraft has even left the gate
  (predeparture).

TARGETS FOR ENHANCEMENT?
11
ATC threat chains Complex tracking

• 10 LOSAs 2426 Flights 9450 threats
• 2350 ATC threats
• ¼ of all threats
• about 1 per flight
• 236 mismanaged ATC threats
• 30 of all mismanaged threats
• 10 of all ATC threats were mismanaged

12
Mismanaged ATC threats
2426 Flights 9450 threats
236 mismanaged ATC threats
13
ATC threats - Errors
2426 Flights 9450 threats
280 errors
236 mismanaged ATC threats
Automation errors (20) wrong MCP/FCU altitude
setting dialed
14
ATC threats - Errors - UAS
2426 Flights 9450 threats
2 UAS
280 errors
236 mismanaged ATC threats
58 UAS
56 Automation errors
9 UAS
21 UAS
5 UAS
15
ATC threats - UAS
2426 Flights 9450 threats
Lateral deviation 17
Speed too high 14
Vertical deviation 10
Unstable Approach 10 Continued Landing
9 Incorrect Automation configurati
on 9 Taxiway/runway incursi
on 6
Speed too low 6
Incorrect Aircraft configuration
5 Other
14
280 errors
236 mismanaged ATC threats
106 UAS
Bottom line 4 of flights had a UAS arising fr
om an ATC threat that was mismanaged.
¾ occurred during descent/approach/land
16
Archie tracks intentional noncompliance

• Across 22 LOSAs, the average is 40 of flights
  have one or more noncompliance errors
  
• Range 23 of flights at one airline to 90 (!!)
  at another
  
• Most common noncompliance errors
• checklist performed from memory / nonstandard
  checklist use
  
• failure to cross-verify MCP/FCU altitude alerter
  changes
  
• PF makes own MCP/FCU changes

17
Noncompliance correlates with..

• Based on 22 LOSAs (4800 flts), airlines that have
  more flights with intentional noncompliance
  errors also have more flights with
  
• Mismanaged threats (r .7)
• Mismanaged handling errors (r .9)
• Undesired aircraft states (r .9)
• Mismanaged UASs (r .8)

18
Archies Future
Archie is maturing and getting smarter more
LOSAs allow for more analyses at the airline
level, which is where the greatest differences
are emerging. Soon a sub-Archive for regional
airlines
Archie is getting involved in more industry
projects If LOSA and ASAP overlap, then focus
ing on LOSA targets should reduce ASAP
incidents ATC is a big threat to pilots, are p
ilots a big threat to ATC?
19
Archie is recognizing his strength

• LOSA is a macro approach for Safety Management
  Systems
  
• LOSA detects gross effects strengths and
  weaknesses in the system
  
• Management uses the information
• Self-corrections, training, procedural changes,

If and when certain problems persist, micro
approaches can investigate more closely

20

• The University of Texas
• Human Factors Research Project
• www.psy.utexas.edu/HumanFactors