Pilot Fatigue Prediction System

1
Fly Safe
Pilot Fatigue Prediction System

• Milinda Illesinghe

Supervised by Dr. Thilak Chaminda
2
Overview

• Initial Problem and Scope
• About Fatigue
• Fatigue Overcome methodologies
• Proposed System
• Requirement Analysis
• Scope Refinement
• Project management
• Design and Implementation
• Testing
• Evaluation
• Conclusion
• Future Enhancements

3
Initial Problem

• Pilot fatigue is massive problematic with
  aviation.
• Fatigue related Aviation accidents directs the
  industry insecure.
• Real-time Fatigue Detection or Identification is
  not a appreciate solution for Aircraft pilots

Scope
Application to forecast the probability and the
level of Human Fatigue
4
Fatigue
What is Fatigue?

• What is it?
• How its Happen?
• To Whom?
• Aviation Industry
• Critical of it?

How its Happen?

• Lack of Sleep
• Using high amount of Caffeine and Alcohol content
  drinks
• Physiological and Physiological Conditions

5
Fatigue
To Whom it happen?

• Anyone in the world
• Improper Sleeping pattern maintainers
• Heavy work employees
• Night shift worker
• People who challenging with Circadian Rhythm

• What is it?
• How its Happen?
• To Whom?
• Aviation Industry
• Critical of it?

6
Fatigue
Fatigue Vs. Aviation

• Responsible with numerous passengers
• Value of the properties
• Reputation of the airline

• What is it?
• How its Happen?
• To Whom?
• Aviation Industry
• Critical of it?

Critical of it
UPS Flight 1354 crash in Alabama-USA (14 Aug
2013)
Minnesota crash in 2008
7
Fatigue Overcoming Methodologies

• Education to prevent
• Cockpit napping
• Real-Time Fatigue Detection
• Vision based monitoring
• Behaviour monitoring
• EEG monitoring
• Fatigue Management and Scheduling
• FRMS
• CAP 371 Regulations
• Fatigue Prediction

8
Proposed System
Pilot Fatigue occurrence probability prediction
System by observing the pilots prior day
behaviours with wearable devices
9
Requirement Analysis
- Stakeholders
10
Requirement Analysis
- Elicitation Techniques

• Elicitation Techniques

• Survey Experiences

• Literature Review
• Interview
• Questionnaires

• Targeted Sri Lankan Airline Pilots
• With coordinates of CAA-SL
• Pilot behaviours dataset gathered by 10 pilots
• Overall pilots engage 20 pilots engage to the
  survey

Interviewed Pilots Age groups
11
Requirement Analysis
- Statistics
Choice of Mobile devices brands
Pilots popular internet connectivity devices
Pilots popular internet surfing browser
Citizens in Sri Lankan
Pilots in Sri Lanka
12
Requirement Analysis
- Functional and Nonfunctional

• Functional Requirement

• Non - functional Requirements

• Accuracy
• Usability
• Performance
• Compatibility
• Security

• Predict the Fatigue Probability
• Provide feedback to the System
• Re-train the prediction model
• User authentications to responsibilities.

13
Requirement Analysis
- Use case Diagram
14
Requirement Analysis
- Scope Refinement
15
Project Management

• Development Methodology Agile (scrum)
• Development and Progression updates with
  Supervisor
• Object Oriented Development
• Web Application and Prediction Module
• Worked on planned schedule
• Risk Management was evaluated

16
System Architecture

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

17
Prediction Module - Architecture

• Prediction Stage

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

• Phase A Validate the CAP 371 regulations
• Phase B Fatigue Prediction of Previous Day
  Behaviour
• Phase C Combining Phase A and B

Prediction Architecture
18
Prediction Module - Architecture

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

19
Prediction Module CAP 371 Regulations
(Decision Tree and SVM)

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

CAP 371 Rules
Intersection Value Decider
Risk of the Fatigue
SVM Design
20
Prediction Module Pilot Behaviour
(Multi Layer Perceptron)

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

• Prediction Implementation
• Force Train Algorithm Implementation

True or False Options -D, -R, -I, -C,
-B Recursive Options with Numeric values -N
10, -M 5, -L 10
Optimization of MLP
Selection of Best Model
21
Prediction Module Combining Predictions
(Fuzzy Logic)

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Structure of fuzzy logic
Membership Function
Defuzzification Values
22
Dataset Pre-processing
- Dataset Background

• Dataset creation
• Similar approach attributes
• Sleep pattern of Railroad maintenances Survey
  2006
• Sleep Patterns of Railroad Dispatchers - 2008
• Fatigue Management Program for Canadian Marine
  Pilots - 2002
• Aviation relates suspecting attributes to the
  Fatigue
• Proposed System Dataset
• Number of Attributes - 24
• Number of Classes in the Target- 5 NN, N, A, Y,
  YY
• Number of Instances - 75

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

23
Dataset Pre-processing
- Dataset Content
Attribute Range Description Affect to Target
Age 2,3,4,5,6 Age of the pilot. Eligible Age Constraint is within 20- 69 Middle age limit pilots are much effective than others.
Gender Male, Female Pilots Gender No gender effect on the duties
Marital Status Married, Unmarried Marital Status of the pilot No Marital Status effect on the duties
Children Dependents Numeric Number of children depend on the pilot Risk of the fatigue is depend with this attribute due to the responsibilities
Job type Pilot Flying, No Pilot Flying, Cruise Job duty at the flight. Act as a pilot flying, co-pilot, or extra. Flying pilot has higher fatigue risk than co-pilot. Been extra pilot is least risky
Flight Automation High, Medium, Low Flight operation automation level Flying with Low level automation flight is affect to fatigue.
Flying Sectors Numeric Number of sectors planned to fly consecutive Flying more sectors affect to fatigue
Start Time 24 hours Starting time of the duty Start works on mid night is highest affect to fatigue. Normal daylight makes low risk
Flying Hours 24 hours Duty time period Flying hours correspond equal to fatigue risk
Time-Zone Diff. 24 hours Change of time zone Changes of time zone is effect for the human circadian clock. It affect to fatigue
Sleep location Home, Away Prior day Sleep location Pilot who rest at their home has Low fatigue risk
Bed time 24 hours Time to went for the sleep Out of Standard sleeping schedules affect to the fatigue
Time woke up 24 hours Time of wakeup from the sleep Out of Standard sleeping schedules affect to the fatigue
No times awakened Numeric Number of awaken incident at the last sleep No awakened times correspondently equal to fatigue risk
Quality of sleep 1,2,3,4,5 Sleep Quality listed from 1-5 (1 poor) No times awakened attribute affect to this attribute
Nutrition Fair, Poor Quality of the Prior day Diet Poor diet is higher chance to the fatigue
Drug/Alcohol use Yes, No Taking alcohol based beverages (Wine, Beer, Liquor, etc.) Using alcohol content drugs is popular among this society. Taking drugs previous day is not much affective to fatigue
Caffeine Beverages Yes, No Taking caffeine contain beverages Its highly affective to the fatigue
If yes how many Numeric Number of cups of caffeine beverages Amount of caffeine correspondently equal to fatigue
Health status 1,2,3,4,5 Healthy level of the pilot (1 Poor) Health status correspondently affect to fatigue
Medical treatment Yes, No Use treatment to any disease Taking medical is affect to the fatigue
Psychological Issue Yes, No Having Psychological issues Having Psychological Issue is affect to the fatigue
Stress Yes, No Having Stress Having Stress is affect to the fatigue
Sleep Disorders Yes, No Having Sleeping Disorder Having Sleeping Disorder is affect to the fatigue
Fatigue YY, Y, A, N, NN YY Extreme Fatigue, Y Fatigue, A Average, N No fatigue, NN Never a chance for a Fatigue YY Extreme Fatigue, Y Fatigue, A Average, N No fatigue, NN Never a chance for a Fatigue

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

24
Dataset Pre-processing
- Dataset Balancing

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Average Instances count for Dataset training
25
Dataset Pre-processing
- Sequence diagram of Balancing

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

26
Pilot Behaviour Dataset Component Analysis - PCA
Factor Priorities Effective Attributes for Fatigue
F1 Age, Children Depends, Flying Hours, Sleep Quality
F2 Duty Start Time, Bed Time, Wakeup Time, Nutrition
F3 Job Type, Caffeine used, Stress
F4 Caffeine used, Sleep Location, Job Type

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Factor wise Best representative Attributes
Class Distribution
Variable Distribution
27
Pilot Behaviour Dataset Component Analysis - PCA

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Correlation Matrix
28
Web Application - Architecture

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Static Web Page
Architecture
29
Web Application Screen Shots

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

30
Web Application User Manual
http//flysafe.net46.net/support/

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

31
Web Service Connector

• System Architecture
• Prediction Module
• Dataset Preprocessing
• Web Application
• Web Service Connector

Architecture
32
Testing

• Test case Coverage

33
Testing

• Accuracy Testing -Qualitative

• Compatibility -Qualitative

Browser Compatibility
Resolution Compatibility
Resolution Device login
1366 x 768 Laptop inline
1280 x 768 Personal Computer inline
1280 x 800 Tab inline
480 x 854 Mobile horizontal

• SVM accuracy 100
• MLP accuracy 93.3
• Fuzzy Logic Accuracy 100
• Worst case Accuracy is 93.3

• Performance Testing -Quantitative

Signal Mode Signal Strength Loading Time Test Result
3G Regular 750kbps 4.29s Pass
3G Good 1Mbps 1.83s Pass
4G LTE Regular 4Mbps 1.80 Pass
34
Evaluation

• Qualitative Evaluation

• Quantitative Evaluation

• Evaluators

• Accuracy

• Pilots (Users)
• Aviation Officers (Experts)

• SVM Prediction
• MLP Prediction
• Fuzzy Logic

Name Profession Experience
Capt. Sunil Wettamuni Sri Lankan Airline (Line Pilot) 31 Years
Capt. Lucian Ratnayake Sri Lankan Airline (Line Pilot) 28 Years
Capt. G.A.Fernando Sri Lankan Airline (Line Pilot) 13 Years

• Dataset Analysis

• Dataset Behaviours
• Dataset Balancing approach
• Effect to re-train model

Mr. H.M.C Nimalsiri Director General Authority CAA-SL 2011-present

• Evaluate Subjects

• Approach
• Data Gathering
• Usability
• Drawbacks

35
Evaluation
- Dataset Behaviours

• Quantitative Evaluation

Dataset ID Attributes Available Trained Duration Accuracy
D1 Original Dataset with 24 attributes 21 minutes Accuracy 93.33333 Confusion Matrix 1.70
D2 Factor 1s top 15 effective Attributes 15 minutes Accuracy 95.00000 Confusion Matrix 1.78
D3 Factor 2s top 15 effective Attributes 39 minutes Accuracy 91.66667 Confusion Matrix 1.65
D4 Factor 3s top 15 effective Attributes 22 minutes Accuracy 93.33333 Confusion Matrix 1.78
D5 Factor 4s top 15 effective Attributes 14 minutes Accuracy 87.50000 Confusion Matrix 1.63
D6 Combination of D1 and D2 and filters the top 15 effective Attributes 48 minutes Accuracy 89.16667 Confusion Matrix 1.64
D7 Combination of D1 and D3 and filters the top 15 effective Attributes 16 minutes Accuracy 94.16666 Confusion Matrix 1.82
D8 Combination of D1, D2, D3, and D4 and filters the top 15 effective Attributes 15 minutes Accuracy 92.50000 Confusion Matrix 1.70
Best Accuracy algorithm
Best Confusion Matrix
Accuracy Vs. PCA selections
36
Conclusion

• Problem and Challengers Encountered

• Building the pilot behaviour dataset.
• Monitoring the prior day pilot behaviours
• Time and resource management

• Learning Outcomes

Technologies which Already knew Technologies which Partially Knew and Improved Technologies which Newly Learnt
Web Application Development with PHP Mathematical Graphs and Statistics Web Service and JavaEE developments
Java Programming AJAX and JQuery Data Mining techniques
Database Implementation Bootstrap development Weka Implementation with Java
  Algorithm Designing Component Analysis
    Browser restrictions on SOP
37
Limitations

• This System is restricted to the Aircraft Pilots

• Not for Helicopter Pilots
• Not for other type of transportation operators

• System is not considering the cumulative night
  shifts

• One system Admin should appoint with the
  prediction web service

38
Future Enhancements

• Automated Pilot Behavior Monitoring
• Automated Feedback Providing Module

39
Future Enhancements

• Fatigue Prediction As A Service

40
Thank You