How Does the ECA Assess Member States

1
How Does the ECA Assess Member States Internal
Control Systems? Alan Findlay (CEAD
Directorate B - Audit and Audit Supervision)
Workshop on Audit/Evaluation of Public
Internal Financial Control Systems
(PIFC)Ankara, 8-9 July 2008
European Court of Auditors
2
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

3
The Assurance Model (1)
4
The Assurance Model (2)

•  Assurance Model
• as a tool to manage audit risk
• Preliminary assessment of inherent and control
  risk (combined risk assessment) to determine
• Reasonable expectations concerning level of
  confidence (controls assurance) that can be drawn
  from supervisory and control systems
• Resulting extent and scope of substantive testing
  to be carried out, aiming to reach required
  overall level of confidence (95)
• Revision and / or final confirmation of
  preliminary assessment on the basis of audit
  results achieved

5
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

6
DAS audit process
PERFORM AUDIT
P
LAN AUDIT
AND EVALUATE
PROCEDURES
RESULTS
Audit
Audit
Environment
Evidence
UNDERSTAND
F
ORM AUDIT
THE AUDIT
International
CONCLUSIONS AND

Audit
ENVIRONMENT
Legal Basis

AUDIT OPINION
Standards
THE DAS

• International

Art. 248 of the Treaty
Standards of

• Reliability

Auditing (ISA)

• Legality

published by IFAC
regularity

• INTOSAI auditing

(Art.
129(4) of Financial
standards
Regulation)

• COSO Framework

• published with

financial
statements
7
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

8
Audit environment

• Collect /update and analyse information about
  the audit environment
• Legal framework
• Design and functioning of the supervisory and
  control systems
• Nature of payments
• Audit standards to be applied, etc.

Gain understanding of the audit environment
9
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

10
Audit procedures
Plan audit procedures
Plan audit procedures allowing to obtain
reasonable assurance
11
Audit procedures (2)

• Determine materiality
• Errors are material if, individually or
  aggregated with other errors, they would
  reasonably affect decisions of addressees of
  opinion
• General practice between 0,5 and 2
• Possibility to set different level(s) taking into
  account addressees requirements (tolerable or
  residual risk)
• Issues material by context/nature to be disclosed
• Materiality rules not (always) directly
  applicable to systems weaknesses (often only
  risk)
• Qualitative aspects (seriousness of deficiency or
  frequency)
• Quantitative aspects (possible financial impact)

12
Audit procedures (3)

• Preliminary assessment of inherent risk
• Economic and technical environment
• Type and operating of activities, programmes,
  etc.
• Characteristics of beneficiaries
• Complexity of rules
• Management policies and practices
• Other information to be taken into account
• Previous work and knowledge / experience
• Annual activity reports and declarations of
  Commissions Directors-general and their
  Synthesis
• Other obligatory reports of different services of
  Commission and Member States
• Relevant other information (National
  declarations, national certificates, ...)
• Reports of other auditors

13
Audit procedures (4)

• Preliminary assessment of
• quality of supervisory and control systems
• Central question Are supervisory and control
  systems adequate to manage the risk insofar as
  compliance with relevant laws, regulations,
  contracts, etc. is concerned?
• Strong link between inherent risk and control
  risk
• Different implementation levels to be considered

14
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

15
Performing the audit

• Perform audit procedures
• Examination of supervisory and control systems
• Substantive testing of payments
• Analysis of management representation (National
  Declarations Summaries, Annual Activity Reports,
  etc.)
• Possibly work on other sources of audit evidence
  (e.g. other auditors)

Perform audit procedures and evaluate results

• Evaluate audit results
• (Re-)examine / (Re-)assess preliminary
  expectations
• Modify initial planning on the basis of
  preliminary audit results, if needed
• Perform supplementary audit work, if necessary

16
Audit objectives for supervisory and control
systems

• Assessment of their capacity to
• Supply management with information needed to
  ensure that legal, regulatory and contractual
  provisions have been complied with and, where
  applicable, corrective action is taken
• Give reasonable assurance concerning compliance
  of revenue and expenditure with laws,
  regulations, contracts, etc.

17
Main steps of evaluatingsupervisory and control
Systems

• Auditors should first identify the management
  departments, national/regional/local authorities
  or other third parties having a role of
  supervision or control
• Preparation of a programme of tests to check
  systems compliance with three key control
  objectives
• Design according to regulation or other
  provisions
• Transposition by parties concerned
• Continuous and effective functioning

18
The role of substantive testing in evaluating
systems

• Substantive testing samples may also be used for
  tests of control to limit number of on-the-spot
  checks
• Examination of payments affected by errors
• Identification of systems which should have
  prevented or identified and corrected them
• Analysis of control deficiencies at their origin
• Overall analysis of the results
• Assessment of level of assurance obtained

19
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion

20
Forming audit conclusions opinions
Form audit conclusions and audit opinion
Form audit opinion
21
Forming audit conclusions opinions

• Straightforward forming of audit conclusions
  requires that
• Actual audit results confirm reasonable
  expectations formed on the basis of preliminary
  combined risk assessment
• Findings for different sources are compatible
  with each other

22
Form audit opinion
Examination of work of other auditors
Analysis of annual activity reports and
declarations
Evaluation of supervisory and control systems
Substantive testing

• Professional judgement and materiality
• Qualitative evaluation of results on work on
  systems
• Quantitative evaluation of results of substantive
  testing
• Analysis of coherence of audit results

Audit conclusions
Audit conclusions
Audit conclusions
Audit opinion the DAS
23
Agenda

• The Courts Assurance Model
• The DAS audit process
• Understanding the environment
• Planning audit procedures
• Performing the audit evaluating results
• Forming audit opinion
• Questions / Discussion