Starting to Audit Documents in the Engineering Domain

1
Starting to Audit Documents in the Engineering
Domain
Peter J Wild, Steve Culley, Chris McMahon, Mansur
Darlington, Shaofeng Liu
Innovative Manufacturing Research
Centre Department of Mechanical Engineering
University of Bath
2
Presentation Overview

• Project Context
• Introduction
• Study Setting and Context
• Interim Findings
• Modelling Framework
• Conclusions and Future Work

3
Project Context

• ICID Information Content in Documents
• Three strand project
• Stream A Document Structures Decomposition
  Schemes
• Stream C Computational Methods Mark-up
  Languages

• Stream B Engineers Exploration Behaviour
  Document Use Patterns
• Document Audits
• Diary Studies
• Observational work
• Leading to evaluation / experiments using
• Document structures (Stream A)
• Scenarios of use (Stream B)
• Tools (Stream C)

4
Document Studies and Auditing

• Interest in
• how people use, generate, save, and search, paper
  and electronic based documents
• auditing information flows
• Attempting to pull this into a coherent approach
  to auditing engineering documents
• Starting questions for the auditing process
• What do they have use and generate

5
Audits should provide data about

• Varieties and types of engineering documents
• Strengths and weakness of document manifestations
  
• Patterns of document use within different types
  of engineering company (e.g. SME, design,
  manufacturing)

6
Case Study Setting and Context

• TrollCo
• Engineering Design and Manufacturing company
  based in Wiltshire
• General engineering work
• e.g. nuclear, railway, highways
• Design and manufacture of their own product line
• Culture of pride in the quality of their work
• We are the Rolls Royce of..
• Pressures of the market
• Trouble is people want Rolls Royce at Ford
  prices
• British Standards Institute (BSI) quality
  certified

7
Research Approach

• Engineers would volunteer stories, scenarios,
  information about how things were carried out
• Lacking formal interviews
• Informal interviews, and scenario gathering
• Pen and paper observations
• Iterative discussions with TrollCos engineers
  have taken place
• Audits of physical computer based documents

8
What Do They Have?

• 1000s Documents
• 250 Types
• Reflect a variety of concerns, manifestations and
  life-spans
• Expected and TrollCo specific
• Cannot be easily wedged into a simple
  classification (e.g. ISO 9001)
• We are exploring classification methods

9
Using and Generating

• Use and generate large number of documents.
• Mostly developed by the quality management system
  
• Many generic to product and engineering services
  division
• Distinction between routine and exception
  documents

10
Product Division Documents (Order)
Product Order / Demonstration Build Test / Assess quality Delivery Finance Service / Maintenance
Brochure Demonstration-request Draft Order Draft Quote Final Quote Final Order Order acknowledgement Contract Review Production programme Product stocking sheet Planning sheet Call off sheet Welding build sheet Sub assembly build sheet Time Temp Requirements Electrical options Final inspection check Certificate of conformity Advice Note Maps Invoice Routine Service Customer Feedback form Parts order form
Exceptions Exceptions Exceptions Exceptions Exceptions Exceptions
Credit checks Internal reject note, External reject note, Supplier Quality Approval Record, Corrective action request form and Vendor assessment form Internal reject note, External reject note, Supplier Quality Approval Record, Corrective action request form and Vendor assessment form 2nd Invoice Legal proceedings Breakdown Checklist Call out Service
11
TrollCo Consider Their Document Usage Chaotic!

• Documents have
• needed information
• layout and meaning are comprehensible
• Document chaos appears to stem from
• ambiguity in orders
• dealing with multiple instances of orders
• the variety inherent in the product line and
  customer needs
• Design Documents probably are chaotic

12
An Unused Catalogue Taxonomy

• Files marked with a category and subcategory
• Taxonomy was locally developed and reflected the
  forms of material that the engineers used, and
  still use today
• It was apparent that it was neglected

13
Contemporary storage situation
14
And
15
Why?

• Demise of the ordered catalogue store and
  taxonomy due to
• Time
• Location
• Supplier management strategy

16
Paper vs. Electronic Documents / Files

• Minimal use of bookmarks
• Email client file structure was default
• Naming and hierarchy conventions in files are
  variable and inconsistent between electronic and
  paper manifestations

• Struck by paucity of manifestation, binding and
  container mechanisms in computer based file
  systems

17
Paper vs. Electronic Documents / Files

• In future audits we will investigate further the
  semantics ascribed to document bindings
• More generally we could investigate more
  elaborate binding mechanisms in computer based
  documents (see also Rogers 1996)

18
A Modelling Framework?

• Activity being undertaken with documents is
  essential to understanding why they exist and how
  they are used
• Non-partisan approach to how we model the work
  aspects in analysis of data from document audits
• Analysing the context of documents at the five
  levels of Culture, Flow, Sequence, Artifact, and
  Physical Character have been influential in
  framing our understanding

19
Culture Flow and Sequence
Modelling Issues / Element Have, Use, Generate TrollCo examples
CULTURE Influences, overlap, Standards and policy, power, breakdowns, values, identity, emotions, style, preferences How the ISO culture influences which processes and documents are undertaken The influence of ISO type documents, and the overall concern with maintaining BSI. Pride in the quality of the product. Fiercely competitive and litigious competitor. Work culture, staff culture. Culture of purchasers, (large government, and industrial). Informality and hands on nature of the design culture. Staff and recruitment issues.
FLOW Roles, Agents, Goals, collaboration structures. Central and representative roles Which roles and agents own and archive documents Which are the key documents? Which people are documents passed onto? For general flow see cells in table 1. To-from customer. Flow between, staff in engineering, marketing, finance, production, and quality managers. To-from suppliers. To-from other bodies (tax, quality and financial auditors
SEQUENCE Task and Sequence models What is the sequence of document use? How the documents are generated? ISO processes and documents See individual cells in table one for order
20
Artefact and Physical Properties
Modelling Issues / Element Have, Use, Generate TrollCo examples
ARTEFACT Artefact models, document types and manifestations, document creation and exchange tools. Document proxies, central and representative artefacts. Manifestations, (e.g., email, fax, letter, paper, electronic) Bindings Containers Templates, printers (individual and shared), fax machines (in various locations) photocopier, software artefacts (CAD, word processors, email, web browsers, PDF related tools). Manual tools, staplers, pens, pencils, cabinets, cupboards, files, archives, shelves, desks, paper pads.
PHYSICAL Location of artefacts, office layouts Movement of documents and related artefacts around the site Physical location of documents, archives (multiple), location of tools and artefacts, physical distance between staff in marketing, finance, engineering and production. Physical location of staff and offices, suppliers (UK, and Europe) and customers (UK, Europe, USA, and Australia).
21
Conclusions

• Work in progress
• Interim findings
• Have Use and Generate
• Catalogue Taxonomy and Storage
• Electronic and Physical Manifestations
• Focussing on 3 questions of have, use, and
  generate, and five levels of analysis Culture,
  Flow, Sequence, Artifact, and Physical Location
• Are Have Use and Generate always that
  distinguishable?
• Do engineers think well in these terms?

22
Future Work

• Arranging to audit documents in other companies.
  
• Particular focus in future audits is adding
  methodological rigour to the data gathering
• Also using auditing software tools to
  unobtrusively gather data and speed up some
  analysis, and provide more robust statistical
  comparisons between organisations.

23
Questions and Discussion