Experience in Deploying Aviation Maintenance Examination Systems

1
Experience in Deploying Aviation Maintenance
Examination Systems
Presentation at EATS 2008, Vienna, November 2008

• Paul J. Nolan
• Director, ASTech

2
Contents

• ASTech introduction
• Work in Examination Systems
• Common User Issues
• Some of the Problems
• The Future Web Technologies
• Concluding Remarks

3
Industry Sectors Product Groups
Edu. Training
4
What ASTech Does in Aviation Industry
ASTech (Atlantic Simulation and Training
Technologies) has been serving industry for over
seven years and specialises in the development of

• Examination Systems (on-line and paper-based MCQ)
• Web-based training and LMS systems
• Multimedia and simulation based training
  components

PART-66 Examination Software Experts
5
Clients include

• ESB Generation (Irl)
• National Grid (Irl)
• Edenderry Power Co. (Irl)
• Group Schneider
• Wyeth Biofarma
• Lufthansa Tech. Training (Hamburg Frankfurt)
• Scandinavian Airlines (Norway, Sweden and
  Denmark)
• TAP Air Portugal
• Irish Aviation Authority
• Greek Civil Aviation

• SRTechnics (Irl, UK Switzerland)
• Shannon Aerospace
• National Flight Centre (Irl)
• European Helicopter Academy
• Open Learning Centre (National University of
  Ireland, Galway)
• FAS Training (Irl)
• Cargolux (Luxembourg)
• Kollmorgan Controls
• Boston Scientific

6
SBT (Simulation Based Training) 2
Studies have shown Current training tools fail
to allow technicians to develop a simple,
accurate mental model upon which to build
knowledge and skills during training and line
operating experience. Technicians need a tool
that can support the long-term acquisition of
automation skills and an accurate mental model.
Simulation allows technicians to
understand HOW procedural skills WHY this
requires an accurate mental model WHEN
automation skills Simulation model is at the
center of a tool that allows the technicians to
understand the WHY, WHEN and HOW (The Boeing
Company, Dr Mumaw, D. Boorman and J. Griffin,
March 2001)
7
Earlier Reference
Web-Based Special Purpose Simulatorsor STD
(Synthetic Training Devices)
Rationale
Confucius (c. 450 BC) said I hear and I
forget, I see and I remember, I do and I
understand. Educators say that typically students
remember - 10 of what they read - 20 of what
they hear - 30 if they see visuals related what
they are hearing - 50 if they watch someone
doing something while explaining it - 90 if they
do it themselves, even if only as a simulation
8
Original Schematic
On right is a scanned image of schematic. ASTech
has vapabilities to transform these into active
schematics see next slide
TOC
9
Boeing 737 Electrical Power System
TOC
10
Log of Previous Actions Displayed
11
Assessment Examination Tools

• Multiple Choice Essay Questions
• Questions randomly generated from large data
  banks (5000 questions)
• Questions selected based on chapter/topic/sect
  and level of difficulty
• Both formal and practice exams
• Used for certification of a/c engineers pilots
• Paper based and on-line examinations
• Auto correction and detailed statistical analysis
• Integrates with SAP, MINT and other proprietary
  databases for personnel records etc.

12
2000
2001
2006
13
2000
2001
2006
14
Publish Exam
15
Reporting Statistics

• Exam Schedule
• Question Paper (Long Form, Short Form etc.)
• Question Listing (including Revisions)
• Individual Trainee Single Exam
• Individual Trainee All Exams
• Single Exam - Item Analysis
• Single Exam Summary
• Client Specific Reports

16
Scanner Ready Answer Paper
17
Some of the Problems/Issues

• Question Import in different formats
• Image Import
• Duplicate Question Identification

18
Duplicate Question Identification

• Description of the Problem
• Approaches to Solution
• Selected Approach
• Algorithm
• Implementation

19
Similar Question Search Results
20
Applications of MC-SQSE

• Identify similar questions in existing
  databanks and use in pruning
• Identify similar questions before they are are
  added to the databank

21
Importing Questions
Media

• Paper Based
• Electronic

Electronic

• Postscript / pdf
• Raw text
• Word processing
• WORD
• Quicksilver
• markup text
• XML
• HTML
• DOCBOOK
• Latex
• csv

22
Electronic Format (contd)

• Spread Sheet e.g. Excel
• Database
• Oracle
• MS SQL Server
• Postgres
• Access

Convert operating systens e.g. Windows, Unix,
Mac Different versions on Windows!
23
Graphic Images

• All usual types including
• BMP
• GIF
• JPG
• PNG
• DXF
• SVG
• PIC

Equations

• Images
• Markup e.g. XML, MathML, Tex, WORQ equations

24
(No Transcript)
25
Integration of LMS Exam Tools
TOC
26
TOC (Table of Contents)
27
Free-Text Search
28
Free-Text Search
29
Tag-based Search
30
Tag-based Search
31
Motivations

• John teaches biology, over the Internet, using
  digital libraries and modern technologies (wikis,
  blogs)
• How to deliver the material just-in-time?
• How to pre-asses students?
• How to automate most of the process?

32
Semantic Web Technologies for Digital Libraries?

• Metadata is the key concept
• the Web does not have metadata
• the idea of a Semantic Web is nice but difficult
  to implement
• many digital libraries do have metadata in place
• we simply must make them available in a machine
  understandable format
• the Semantic Web provides the format RDF

33
EXPOSURE to MCQ Exams

• Formal Examinations (Summative)
• Formative

Lets leverage some of the huge advanced made in
LMS systems such as Moodle, Blackboard, WebCT as
well as Flash
34
SCORM LMS and IETM features

• Generate SCORM manifest for individual course
  modules from Examination Software
• Export SCORM test questions to question bank
• Use examination system Item Analysis in updating
  SCORM course content
• Use modern doc. mgt. features (e.g. Google type
  search) when generating exam questions.

done
done
research
done
35
Summary Key Points

• Examination systems well advanced
• Increasing acceptance of on-line exams
• Web deployment is gaining acceptance
• Common Deployment Problems
• Operating systems / back-end database not an
  issue.
• Importing legacy questions huge task
• Similar question identification and QA critical
• Exams must be run on-line or scanner used for
  full item analysis
• The Future link into LMS and use of intelligent
  document and/or semantic web technologies.

36
Contact Details
ASTech Address Unit 2, Campus Innovation
Centre, Upper Newcastle, Galway, Ireland.
Phone [phone] Fax [phone]
E-Mail contact_at_astech.ie www http//www.astech
.ie
HilCo Address Ewiges Wegli 12, Address 8302
Kloten (Zurich) Switzerland Phone [phone]
1027 Email bhilderbrand_at_flyaway.ch web www.hilco-a
viation.com