Effective Internal Audits

1
Effective Internal Audits Is Outsourcing the
Answer? DENIS J. DEVOS, P.EngDenisDevos_at_sympat
ico.ca [phone] 16th Annual Quality Audit
Division Conference Atlanta Georgia October
11-12 2007
2
Past Practice

• Outsourcing internal audits was discouraged
• Registrars discouraged this abdication of
  management responsibility for QMS
• Companies already pay for external audits, they
  dont want to pay for internal audits, too
• Consultants didnt always work on honing their
  auditing skills
• Internal audit training was widely available, and
  a two-day investment could make anyone an auditor

3
Low Expectations, No Disappointments

• Follow the checklist!
• Dont ask about these other areas, theyre not
  part of our Quality System!
• Tunnel Vision
• Findings were the usual type of document control
  issues, incomplete tags etc.
• Audits didnt propel improvement, but after all,
  these were only internal audits.
• Third-party auditors are not allowed to consult.

4
ISO 90012000The Process Approach

• Process Approach auditing is not yet fully
  matured
• Internal audit courses still focus too much on
  the Standard, instead of critical thinking
• Requirements still not examined within the
  context of RISK
• Abstract concepts such as turtle diagrams are
  counter-intuitive for internal auditors
• The BIG PICTURE is not often understood by
  internal auditors

5
Challenges to Internal Audit

• Audit training cannot teach a Big Picture
  perspective nor impart experience
• Auditors are not always released from their real
  work to perform audits
• Audit teams experience a high rate of attrition
• Findings do not often lead to improved
  performance

6
Survey Methodology

• 155 survey responses from a distribution by Denis
  Devos and two colleagues
• Approximately 90 surveys completed during ASQ
  conferences
• London Ontario monthly meeting
• Canton Ohio monthly meeting
• ASQ Audit Division Conference, Reno 2006
• ASQ Automotive Division Conf. Detroit 2007

7
Survey Sample

• Approximately 75 of surveys from Ontario and
  Michigan and Ohio
• 25 of surveys from throughout the US.
• N155 Automotive 65 or 42

• Aerospace 8.5
• Medical Devices 5.8
• Pharmaceuticals 4.0
• Consumer Goods 4.0
• Engineered Industrial Products 11.7

• Food/Packaging 3.0
• Healthcare 5.0
• Education 1.3
• Govt Services 7.8

8

• When have you used an external auditor
  to perform your internal audit?
• During implementation of QMS
• During upgrade to TS 16949, ISO 90012000 etc.
• Once or Twice as a fill in
• On-going basis internal audits are always
  outsourced

Imp/Upgrade Fill In On-going
9

• Did you have an internal audit team in
  place when you outsourced your audit(s)?
• YES (86 and 76 respectively)
• NO

86
YES NO
10

• Why did you use an external resource?
• Lack of auditors none trained
• Lack of auditors theyre too busy, not
  available
• Lack of auditors they leave the team or company
• We intended to audit, but got too far behind
• Lack of confidence in our internal resources
• A strategic decision we knew we wanted a
  professional

1 2 3 4 5 6
11

• How did you select your internal auditor
• A consultant who assisted with implementation or
  upgrade
• An auditor from an affiliated company or sister
  plant
• Referred by customer, another company, etc.
• Referred by your registrar or their auditor
• Other

1 2 3 4 5
12

• Please rate your experience with the
  quality of the audit services
• Excellent 2. Very Good
• 3. Good 4. Fair
• 5. Poor

Excellent Very Good Good Fair Poor
13

• Why do you anticipate the need for
  external resources?
• Past employee audits have not given satisfactory
  results
• Were getting behind, need a boost to get back
  on track
• Customer/Registrar expectations getting to high
  to meet
• Auditors not trained 5. Too Busy 6.
  Auditors leave us
• 7. A strategic decision we want a
  professional to help us

1 2 3 4 5 6 7
14

• What influenced your decision
  not to outsource?
• Weve never considered outsourcing
• We wouldnt know where to find a competent
  auditor
• We would consider it a failure if we couldnt do
  it ourselves
• 4. We dont consider internal audit important
  enough for that
• 5. Only our own employees know our processes well
  enough
• 6. We dont trust an outsider
• 7. Cost

1 2 3 4 5 6 7
15

• If your organization decides to outsource,
  what is the long-term direction?
• Outsourcing forever a strategic decision
• To fill in the gaps with our internal audits
• To periodically support, coach, mentor our own
  team
• A one-time training or support for our audit team

1 2 3 4
16
What characteristics are most important
in an internal auditor? Industry
or Sector Knowledge Audit experience with
organizations just like yours Work experience
with organizations just like yours Audit
experience in a wide variety of
organizations Work experience in a wide variety
of organizations Track Record, experience,
professional references Certification as CQA by
the ASQ Certification by RAB, IATF
etc. Availability on Short Notice Cost Other

3
2
7 8 9 5 6 10
4
1
----gt tells funny jokes
17

• What are the top 3 characteristics for
  an internal auditor?
• Industry Knowledge
• 2-3 Auditing/Working experience in Similar
  Companies
• 4-5 Auditing/Working experience in a variety of
  Companies
• 6. Track Record and Experience
• 7-8 ASQ Certification RAB/IATF Certification

1 2 3 4 5 6 7-8
18
What is the least important (lowest priority)?
10. Cost 9. Availability on short notice
8. RAB/IATF Certification 7. ASQ CQA
Certification 6. Track Record, experience,
professional references
10 9 8 7 6 Other
19
Cost Influence Once a company decides to
outsource audits, how high a priority is it for
auditor selection?
Top 3 Priority Priority 4-8 Lowest Priority,
9-10
20
Selected Written Comments

• I see greater demand for more outside auditors
  (companies) are frustrated with internal audits
  and need positive conclusions
• A Registrar Auditor in Ontario
• Auditor volunteering can be like pulling teeth
• Medical Device Mfg. in Ontario
• Need knowledge and experience with process and
  cultural changes. We dont need an auditor
  whos been doing the same thing for 10 years.
• Engineered Industrial Products, USA

21
Selected Written Comments
If an audit is outsourced, how can it be
internal? This concept is foreign to
me. Food Safety Regulator - Canada
(outsourced auditor) has very good
understanding of our business and is a good
mentor. We have a strategic relationship w/
several business advisors Electrical
Controls Mfg. in Ontario We brought in a group
to prepare us for FDA and CGA inspection Phar
maceutical Mfg. - USA
22
Automotive Ontario and Michigan
We rely on (auditor) because of a lack of
internal resources. Objectivity, no political
agenda and a value-added approach (Registrar)
was very impressed with the quality of our
internal audits during the last two
visits The biggest plus is the perception of
the company to take it more seriously. Audits
are more effective and more cost effective.
23
Automotive Ontario and Michigan
an eye-opener to the management team. It
forced them to deal with some outstanding,
long-term issues. External Auditors seem to
heighten the importance of the audit and the QMS
as a whole. A good idea. We never have
enough people to do our auditing
24
Conclusions

• Many companies (70) report that theyve never
  considered outsourcing internal audits. This
  trend will likely diminish as word spreads that
  this is now an accepted alternative
• Less than 30 of respondents felt each of the
  following
• Outsourcing internal audits is a failure of
  their team
• Only full-time employees know their QMS well
  enough
• Which suggests that a majority of companies are
  open to considering outsourcing as a strategy

1
2
25
Conclusions
Although cost is a moderate influence on whether
or not to outsource internal audits, it does not
weigh heavily in the decision of which auditor to
use. Better experiences with audits result in
more likeliness to continue.
Automotive 68 high satisfaction, 72 will use
again Non-auto 40 high
satisfaction, 53 will use again
3
4
26
Conclusions
A majority of companies who have outsourced
their audits at least once had a positive
experience and would do it again. Companies
may not outsource their audits forever, but
professional auditors can fill in the gaps and
provide coaching and mentoring to existing
audit teams.
5
6