VTT Technical Research Centre of Finland

1
3rd INTUITION Workshop VR/VE Industry
Challenges and Opportunities
Augmented Reality Plant Model Services for Mobile
Maintenance Worker Pekka Siltanen, Tommi Karhela,
Charles Woodward, Paula Savioja, Jani
Jäppinen VTT Technical Research Centre of Finland
PLAMOS PROJECT
BACKGROUND
PLAMOS PROTOTYPE
The objective of Plamos project is to increase
the productivity of the maintenance personnel by
providing the users all the information needed in
intuitive, augmented reality user interface. The
plant model information is utilised in a mobile
terminal where an augmented reality view is
generated for a process operation and maintenance
engineer. The mobile terminal is a tablet PC
equipped with a camera and WLAN
connection. Plamos creates new innovative
solutions by combining several state-of-the art
research approaches plant model based standard
information interfaces, visual browsing
techniques, augmented reality and human
technology interaction research. The Plamos user
interfaces for mobile workers are specified on
top of the standard maintenance application
interfaces, e.g. OPC Unified Architecture and
MIMOSA OpenOM
In the first Plamos prototype, the tracking
solution is based on detecting visual tags
attached to the plant environment. To enable
greater flexibility, however, our final aim is to
employ hybrid tracking methods with feature
detection methods to determine the users
orientation in the environment.
Complexity of an industrial plant is so huge that
maintenance and operation personnel of the plant
need advanced technology to access information
related to the equipment maintained. Even though
current maintenance and automation systems enable
storage and browsing of equipment and process
related data, they do not solve many of the
common problems the maintenance personnel are
facing, e.g. how to locate the equipment
needing maintenance, how to get intuitive and
easily understood information about the plant
floor equipment and their state, how to combine
data from several sources without accessing many
different applications with different user
interfaces The problems are expected to come even
more difficult in the future. Since
owner/operators are outsourcing parts of their
operations, new business opportunities occur in
industrial services. While the change creates new
business opportunities, it also creates new
demands for software applications and services
Plamos Augmented Reality GUI
OPC
SVG
VRML
MIMOSA OPEN OM
The system uses a small tablet PC (Sony VAIO with
Windows XP) equipped with a video camera and WLAN
connection. The augmented reality application is
based on ARToolkit, which we have enhanced with
e.g. adaptive thresholding, feature tracking and
Kalman filtering. The application recognizes the
visual tags and maps them to the plant model
objects. Visual tags are also used for monitoring
real-time process data of certain object. In the
prototype process data was simulated with APROS
process simulator, which has a standard OPC
interface for remote access. Furthermore tags can
be used for augmenting 3D model(s) of the plant
over the video stream. The final integrated
prototype will combine the information stored in
maintenance system with AR and 3D views all in
same user interface. Using Plamos the user can
also report problems noticed while performing
maintenance tasks.
Automation system/ Process simulator
PID Schemas
3D Models
Maintenance information
ProCore
Plant model information is normally distributed
in several applications. In the Plamos prototype,
the information is stored in one common data
warehouse, the ProCore ontology modelling
application. ProCore allows creation of links
between different data models describing
different views to the plant model, enabling
information that has been traditionally
represented in separate user interfaces to be
shown in one graphical user interface.
The challenges are related to the application
interoperability, as well as human system
interfaces. While the machinery becomes
increasingly complicated and maintenance is
outsourced or carried out with smaller amount of
onsite maintenance personnel, the need for
intuitive user interfaces for locating the
equipments and viewing the product and process
information is increasing.
Schwabenlandhalle, Fellbach / Stuttgart,
Germany30th of November 1st of December 2006
VTT Technical Research Centre of Finland