INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS

1
INTRODUCTION TO NONSTATISTICAL SAMPLING FOR
AUDITORS
Jeanne H. Yamamura CPA, MIM, PhD
2
SITUATION

• You are auditing the Dept. of Admissions
  Records for Micronesia College.
• One of your objectives is to verify that student
  records are being updated correctly and timely.
• You decide to select a sample of grades posted
  from the most recent semester completed.

3
SITUATION

• What would you normally document about this
  sample?
• Sample size
• Selection method
• Population
• Procedures to be performed
• Purpose of test
• What kind of test is this?

4
OBJECTIVES

• Review of sampling concepts
• Types of sampling - overview
• Nonstatistical attribute sampling
• Steps in applying
• Additional coverage of
• Sampling methods
• Compliance auditing

5
Applicable Professional Standards

• SAS 39 Audit Sampling
• SAS 111 Amendment to SAS 39 Audit Sampling
• ISA 530 Audit Sampling

6
AUDIT SAMPLING

• Application of an audit procedure to less than
  100 of the items in a population
• Account balance
• Class of transactions
• Examination on a test basis
• Key Sample is intended to be representative of
  the population.
• Objective To reach a conclusion about the
  population based on the sample items tested.

7
SAMPLING RISK

• Possibility that the sample is NOT representative
  of the population
• As a result, auditor will reach WRONG conclusion
• Decision errors
• Type I Risk of incorrect rejection
• Type II Risk of incorrect acceptance

8
TYPE I RISK OF INCORRECT REJECTION

• Internal control Risk that sample supports
  conclusion that control is NOT operating
  effectively when it really is
• AKA Risk of underreliance, risk of assessing
  control risk too high
• Substantive testing Risk that sample supports
  conclusion that balance is NOT properly stated
  when it really is

9
TYPE II RISK OF INCORRECT ACCEPTANCE

• Internal control Risk that sample supports
  conclusion that control is operating effectively
  when it really isnt
• AKA Risk of overreliance, risk of assessing
  control risk too low
• Substantive testing Risk that sample supports
  conclusion that balance is properly stated when
  it really isnt

10
WHICH RISK POSES THE GREATER DANGER TO AN AUDITOR?

• Type I - Risk of incorrect rejection
• Efficiency
• Type II - Risk of incorrect acceptance
• Effectiveness
• Auditor focus on Type II
• Also provides coverage for Type I

11
NONSAMPLING RISK

• Risk of auditor error
• Sample wrong population
• Fail to detect a misstatement when applying audit
  procedure
• Misinterpret audit result
• Controlled through
• Adequate training
• Proper planning
• Effective supervision

12
SAMPLE SIZE FACTORS

• Desired level of assurance (confidence level)
• Acceptable defect rate (tolerable error)
• Historical defect rate (expected error)

13
CONFIDENCE LEVEL

• Complement of sampling risk
• 5 sampling risk, 95 confidence level
• How much reliance will be placed on test results
• The greater the reliance and the more severe the
  consequences of Type II error, the higher the
  confidence level needed
• Sample size increases with confidence level
  (decreases with sampling risk)

14
TOLERABLE ERROR AND EXPECTED ERROR

• Precision the gap between tolerable error and
  expected error
• Expected population error rate 1
• Auditors tolerable error rate 3
• AKA Allowance for sampling risk
• Sample size increases as precision decreases

15
WHEN DO YOU SAMPLE?

• Inspection of tangible assets, e.g., inventory
  observation
• Inspection of records or documents, e.g.,
  internal control testing
• Reperformance, e.g., internal control testing
• Confirmation, e.g., verification of AR balances

16
WHEN IS SAMPLING INAPPROPRIATE?

• Selection of all items with a particular
  characteristic, e.g., all disbursements gt
  100,000
• Testing only one or a few items, e.g., automated
  IT controls, walk throughs
• Analytical procedures
• Scanning
• Inquiry
• Observation

17
WALKTHROUGHS

• Designed to provide evidence regarding the design
  and implementation of controls
• Can provide some assurance of operating
  effectiveness BUT
• Depends on nature of control (automated or
  manual)
• Depends on nature of auditors procedures to test
  control (also includes inquiry and observation
  combined with strong control environment and
  adequate monitoring)
• Walkthough sample of 1

18
STATISTICAL VS NONSTATISTICAL SAMPLING

• Statistical sampling
• Statistical computation of sample size
• Statistical evaluation of results
• Nonstatistical sampling
• Sample sizes should be approximately the same (AU
  350.22)
• Sample sizes must be sufficient to support
  reliance on controls and assertions being tested

19
WHEN IS SAMPLING NONSTATISTICAL?

• If sample size determined judgmentally
• If sample selected haphazardly
• If sample results evaluated judgmentally

20
TYPES OF SAMPLING

• Attribute sampling
• Monetary unit sampling
• Classical variables sampling

21
ATTRIBUTE SAMPLING

• Used to estimate proportion of a population that
  possesses a specific characteristic
• Most commonly used for T of C
• Can also be used for dual purpose testing (T of C
  and Substantive T of T)

22
MONETARY-UNIT SAMPLING

• AKA probability proportional to size (PPS)
  sampling, cumulative monetary unit sampling
• Used to estimate dollar amount of misstatement

23
CLASSICAL VARIABLES SAMPLING

• Uses normal distribution theory to identify
  amount of misstatement
• Useful when large number of differences expected
• Smaller sample size than MUS
• Effective for both overstatements and
  understatements
• Can easily incorporate zero balances

24
STEPS IN NONSTATISTICAL ATTRIBUTE SAMPLING
APPLICATION

• Planning
• Determine the test objectives
• Define the population characteristics
• Determine the sample size
• Performance
• Select sample items
• Perform the auditing procedures
• Evaluation
• Calculate the results
• Draw conclusions

25
STEP 1 DETERMINE THE TEST OBJECTIVES

• Objective for T of C To determine the operating
  effectiveness of the internal control
• Support control risk assessment below maximum (FS
  audit)
• Identify controls to be tested and understand why
  they are to be tested

26
TESTS OF CONTROLS

• Concerned primarily with
• Were the necessary controls performed?
• How were they performed?
• By whom were they performed?
• Appropriate when documentary evidence of
  performance exists

27
SUBSTANTIVE TEST OF TRANSACTIONS

• Objective for S T of T To determine whether the
  transactions contain monetary misstatements
• Alternatively, to determine whether the system is
  operating as designed
• Identify transactions to be tested and understand
  why they are to be tested

28
STEP 2 DEFINE THE POPULATION CHARACTERISTICS

• Define the sampling population
• Can be defined however desired BUT must include
  entire population as defined
• Test population for completeness
• Define the sampling unit
• Determined by available records
• Based on definition of population and audit
  objective
• Define the control deviation conditions

29
STEP 3 DETERMINE THE SAMPLE SIZE

• Consider desired confidence level, tolerable
  deviation rate, and expected population deviation
  rate
• Judgmentally determine sample size
• NOTE Check against statistical sample size
  tables to verify adequacy

30
TOLERABLE RATE GUIDELINES
Significance of the transactions and related account balances that the IC are intended to affect Significance of the transactions and related account balances that the IC are intended to affect
Highly significant balances Tolerable Rate of 4
Significant balances Tolerable Rate of 5
Less significant balances Tolerable Rate of 6
Preliminary Assessment of CR Tolerable Rate
Low lt 5
Moderate lt 10
High Do not test controls
31
TOLERABLE RATE GUIDELINES
Assessed importance of the control Tolerable Rate
Highly important 3 - 5
Moderately important 6 10
32
ESTIMATE OF POPULATION ERROR RATE

• Prior year results
• Preliminary sample
• Should be low 0, 1
• Higher rates increase sample size

33
STEP 3 DETERMINE THE SAMPLE SIZE

• Guidelines for nonstatistical sample sizes for
  tests of controls
• If any errors found, increase sample size or
  increase control risk (Probably not applicable to
  Public Auditor)

Desired level of controls reliance (how important is the control/process) Sample size
Low 15-20
Moderate 25-35
High 40-60
34
SMALL POPULATIONS AND INFREQUENTLY OPERATING
CONTROLS
Small Population Sample Size Table Small Population Sample Size Table
Control Frequency and Population Size Sample Size
Quarterly (4) 2
Monthly (12) 2-4
Semimonthly (24) 3-8
Weekly (52) 5-9
35
STEP 4 SELECT SAMPLE ITEMS

• Random sample
• Systematic sample (with random start)
• Haphazard selection

36
RANDOM SELECTION

• Every possible combination of population items
  has an equal chance of being included in the
  sample
• Random number tables
• Computer generation of random numbers

37
SYSTEMATIC SELECTION

• Interval calculated and items selected based on
  size of interval
• Interval Population / Desired Sample Size
• Starting point is random number within interval
• Need to consider if bias present due to patterns
  in data

38
HAPHAZARD SELECTION

• Selection by auditor without any conscious bias
• If you select large, risky, or unusual items, it
  is NOT haphazard selection and it is NOT audit
  sampling. Instead targeted or directed
  selection
• Still desire representative sample
• Avoid unusual, large, first or last
• Useful for certain situations
• Example Tracing credits from AR to CR/other
  sources looking for fictitious credits
• Less costly and simpler

39
STEP 5 PERFORM THE AUDITING PROCEDURES

• Conduct planned audit procedures
• What if?
• Voided documents - if properly voided, not a
  deviation replace with new sample item
• Unused or inapplicable documents replace with
  new sample item
• Inability to examine sample item deviation
• Stopping test before completion large number of
  deviations detected

40
STEP 5 PERFORM THE AUDITING PROCEDURES

• Deviations observed
• Investigate nature, cause, and consequence of
  every exception
• Unintentional error? Or fraud?
• Monetary misstatement resulted?
• Cause misunderstanding of instructions?
  Carelessness?
• Effect on other areas?

41
STEP 6 CALCULATE THE RESULTS

• No computed upper deviation rate (per table in
  statistical sampling)
• Compute Calculated Sampling Error Tolerable
  Error Rate Sample Error Rate.

42
STEP 7 DRAW CONCLUSIONS

• Control not effective (system not working as
  designed) if
• Calculated Sampling Error too small
• Depends on sample size used
• Sample Error Rate gt Tolerable Error Rate
• Sample Error Rate gt Expected Population Error Rate

43
COMPLIANCE AUDITING

• Performance of auditing procedures to determine
  whether an entity is complying with specific
  requirements of laws, regulations, or agreements
• Governmental entities and other recipients of
  governmental financial assistance
• Compliance with laws and regulations that
  materially affect each major federal assistance
  program

44
COMPLIANCE AUDITING OF FEDERAL ASSISTANCE PROGRAMS

• Definition of population for testing of an
  internal control procedure that applies to more
  than one program
• Define items from each major program as a
  separate population, OR
• Define all items to which control is applicable
  as a single population
• Second choice usually more efficient

45
COMPLIANCE AUDITING - EXAMPLE

• Federal financial assistance for Island City
• Three major federal financial assistance programs
• Four nonmajor programs
• Control Transaction review to ensure that only
  legally allowable costs are charged to each
  program

46
COMPLIANCE AUDITING - EXAMPLE

• More efficient to select one sample from
  population of all transactions (major and
  nonmajor programs)
• Confidence level 95
• Tolerable deviation rate 9
• Expected population deviation rate 1
• Sample size 51
• 1 allowable deviation

47
T of C versus S T of T

• Test of Control
• Verifies that a control is operating effectively
• Substantive Test of Transactions
• Verified that a transaction does not contain a
  misstatement

48
ASSERTIONS FOR CLASSES OF TRANSACTIONS

• Occurrence Transaction actually occurred and
  pertains to the entity (existence/validity)
• Completeness All transactions have been
  recorded
• Accuracy Amounts and other data have been
  recorded correctly

49
ASSERTIONS FOR CLASSES OF TRANSACTIONS

• Cutoff Transactions have been recorded in the
  correct accounting period
• Classification Transactions have been recorded
  in the proper accounts

50
CALCULATED SAMPLING ERROR

• Tolerable error rate Sample error rate
  Calculated sampling error
• Sample error rate Population error rate
• due to sampling error
• Auditor must evaluate calculated sampling error
  to see if it is big enough (sufficiently large to
  allow for sampling error in population)

51
CALCULATED SAMPLING ERROR

• If Sample error rate gt Tolerable error rate
  REJECT CONTROL NOT WORKING or PROCEDURE NOT
  BEING FOLLOWED
• If Sample error rate gt Expected population error
  rate, REJECT CONTROL NOT WORKING OR PROCEDURE
  NOT BEING FOLLOWED

52
STEPS IN NONSTATISTICAL SUBSTANTIVE SAMPLING
APPLICATION

• Planning
• Determine the test objectives
• Define the population characteristics
• Determine the sample size
• Performance
• Select sample items
• Perform the auditing procedures
• Evaluation
• Calculate the results
• Draw conclusions

53
STEP 2 DEFINE THE POPULATION CHARACTERISTICS

• Identify individually significant items
• Some items too risky, must be audited, OR
• Easier to pull out and test large items
• Stratify population
• Divide population into homogeneous units
• For example, all items gt 10,000
• Items tested 100 are not part of the sample

54
STEP 2 DEFINE THE POPULATION CHARACTERISTICS

• Define the sampling population
• Consists of an account balance or class of
  transactions
• Will project sample results to population
• Must be sure to adequately identify population
• For example Accounts Receivable could be defined
  as
• All accounts
• Accounts with zero balances
• Accounts with debit balances
• Accounts with credit balances

55
STEP 2 DEFINE THE POPULATION CHARACTERISTICS

• Define the sampling unit
• Any item in the defined population
• Could be an account or a transaction

56
STEP 3 DETERMINE THE SAMPLE SIZE

• Subjective determination OK
• Factors to consider
• Amounts of individual items
• Accounting populations usually include a few very
  large items, a number of moderately large
  amounts, and a large number of small amounts
• If not stratified, will need larger sample
• Variability and size of population
• The greater the variability, the larger the
  sample size needed
• Population size little effect on sample size so
  usually ignored

57
STEP 3 DETERMINE THE SAMPLE SIZE

• Factors to consider
• Risk of incorrect acceptance (RIA)
• As RIA increased, sample size decreases
• If controls good, can accept larger RIA for
  substantive testing
• Tolerable misstatement and expected misstatement
• Larger tolerable misstatement, smaller sample
  size
• Larger expected misstatement, larger sample size

58
STEP 4 SELECT SAMPLE ITEMS

• Any method that will result in representative
  sample
• Random sample
• Systematic sample (with random start)
• Haphazard selection

59
STEP 5 PERFORM THE AUDITING PROCEDURES

• Deviations observed
• Investigate nature, cause, and consequence of
  every exception
• Unintentional error? Or fraud?
• Monetary misstatement resulted?
• Cause misunderstanding? Carelessness?
• Effect on other areas?

60
STEP 6 CALCULATE THE RESULTS

• Compute sample error amount or sample error rate
• Project to population
• Projected misstatement
• Error number of items in population
• Error rate dollar population value

61
STEP 7 DRAW CONCLUSIONS

• Compare projected misstatement to tolerable
  misstatement
• If projected misstatement lt tolerable
  misstatement, population OK
• If projected misstatement gt tolerable
  misstatement, population misstated

62
STEP 7 DRAW CONCLUSIONS

• Consider sampling risk
• If projected misstatement lt expected
  misstatement, probably safe to conclude that
  population is OK (i.e., there is an acceptably
  LOW risk that the true misstatement exceeds the
• tolerable misstatement)
• If projected misstatement gt expected
  misstatement, greater risk present (i.e., there
  is an UNACCEPTABLY HIGH risk that the true
  misstatement exceeds the tolerable
• misstatement).

63
STEP 7 DRAW CONCLUSIONS

• If recorded amount believed to be misstated, need
  more work!
• Investigate misstatements
• Adjust recorded amounts

64
RESOURCES

• Audit Sampling An Introduction, 3rd Edition,
  Guy, Carmichael Whittington
• Audit Guide Audit Sampling, New Edition as of
  May 1, 2008, AICPA
• Auditing Assurance Services, 6th Edition,
  Messier, Glover, Prawitt
• Auditing Assurance Services, 12th Edition,
  Arens, Elder Beasley

65
QUESTIONS?