Chris Nyce, FCAS, MAAA

1
Research on the Effectiveness of a Trend Test in
the Property/Casualty RBC Formula
AAA Updated Research on the NAIC Risk Based
Capital (RBC) Formula

• Chris Nyce, FCAS, MAAA
• KPMG Senior Manager

2
Disclaimer

• These results are based on research conducted by
  a subgroup of the American Academy P/C RBC
  Committee
• Views expressed today are based on the research,
  but do not necessarily reflect the views of the
  Academy, CAS, KPMG, or the NAIC who of course
  makes all decisions about changes to the RBC
  formula
• Examples used are illustrative, and not a
  reference to any specific company
• Anyone who says otherwise is not only wrong, but
  is itching for a fight

3
The Mission of the AAA RBC Committee

• Began the research with a charge
• NAIC expressed concern that RBC was becoming more
  of a lagging indicator, not useful in predicting
  companies that become weaker
• Given the use of a trend test in the life RBC
  formula, is the application of a trend test in
  the Property/Casualty RBC formula a good idea?
• Our interpretation-
• Not a Yes/No question
• Instead-What is the most effective way of
  differentiating between companies above the
  Company Action Level that are likely to fall
  below it, and those that are likely to remain
  above it.
• We approached this with a one year future time
  horizon, i.e. based on observable data this year,
  what will happen next year

4
Status of the Work

• These ideas were presented in the Spring 2004 CAS
  meeting, prior to release of the Academy report
• Academy issued a report on August 26, currently
  featured on the casualty cover page of the
  Academy website
• The Academy report did not recommend
  incorporating the finding in RBC formulas,
  instead concluded the one certain approach is a
  best predictor of those examined
• NAIC PC RBC subgroup did recommend on August 31,
  2004 that this trend test be incorporated into
  the RBC formula, and a company being flagged
  results in CAL
• Exposure period ends November 11, 2004
• NAIC considers comments, votes at December
  meeting
• Could vote yes, no, or further deliberation
• If yes, likely implementation in 2005

5
Background

• Life test currently uses a trend test
• Applies to companies with RBC between 250 and
  the company action level (CAL) of 200
• RBC ratio is the ratio of capital to RBC required
  capital
• Life test looks at past changes in RBC
• Max of last year and the three year average
  decline in margin for each company
• Apply that change to the current RBC position
• If below 190, company is deemed to be at the CAL
• Note that even before our work, the feeling of
  committee members was that the life trend test
  did not work well for P/C companies
• We quickly confirmed this to be true

6
Our Approach

• Basic question-What is the most effective
  predictor of decline in capital adequacy?
• In general terms, used Hypothesis Testing
• Examined specific cases of past company failures
• Formulated hypotheses on the causes of RBC
  decline
• Tested the hypotheses using statistical tests on
  annual statement data
• Conducted additional tests by examining the
  effectiveness retrospectively
• Measured the results using a specific set of
  metrics
• Selected one approach that produced the best
  metrics

7
Boundaries of our Study

• Did not constrain ourselves to examining the life
  formula
• Based on publicly filed data from the NAIC blank
• And used company level data, due to conclusions
  in the work leading to the original formula that
  this was appropriate
• Outcome has to be intuitively correct, and simple
• All research also from public data sources
• For NAIC data, company names remained
  confidential
• Outcome had to be based on empirical data, not on
  our preconceived opinions

8
Data Considerations

• For micro analysis we used public data sources
  such as AM BEST and press reports
• NAIC provided 5 year history of all requested
  data elements
• Confidential as to company identifier
• About 2400 companies
• Used data through 2002 for statistical tests,
  updated through 2003 for retrospective test
• We scrubbed the data, in general separately for
  each test to maximize data points utilized
• Screened out invalid entries and extreme values

9
Micro Results-Initial Hypotheses

• Companies we examined could be characterized as
  experiencing trouble due to various causes, such
  as
• High levels of reinsurance recoverables, causing
  high leverage in estimating reserves, and
  exposure to disputed balances
• High leverage of premiums and reserves to surplus
• Reserve inadequacies coming to roost
• Poor operating results
• Fraud and misrepresentation
• Ill-liquid or incorrectly valued assets
• Under-funded pensions (usually a contributor,
  not a cause)

10
What is the Best Early Indicator of Future
Capital Declines?
Lack of Liquidity
Bad Assets
Past Capital Declines
Poor Profitability
11
Overall Macro Approach

• Performed statistical tests on the NAIC database
• Explored the basic relationships behind each
  hypothesis
• Performed retrospective tests on characteristics
  of companies just prior to falling to the CAL
• Set up metrics to evaluate the outcome of the
  retrospective test
• Determined recommendations based on all of the
  above

12
Statistical Tests

• Explored relationships between hypothesized
  variables
• Performed tests on the NAIC database of 2400
  companies for 5 years ending 2002
• Looked for statistical tendencies
• Generally used correlation and regression
  analysis
• Examined the percentage of variation explained
• Calculated the measures of significance
• Used to corroborate and explain retrospective
  result
• Note that a poor result in our tests does not
  necessarily mean that the measure is not good for
  IRIS or other financial evaluations
• And high correlations dont necessarily mean the
  hypothesis would form a good trend test

13
Statistical Test of Life Type Trend Test

• Does a simple life type of trend test work?
• Correlation between year to year changes in RBC
  ratio for all companies -23 (wrong sign)
• For only companies near the CAL 1
• In 2001 and 2002, the direction of the change in
  subsequent years was only the same 41 of the
  time
• Changes in market asset valuations dominated any
  characteristics of companies themselves
• Implication Life type of trend test is worse
  than random guessing for P/C Companies

14
What About Underwriting Results and Reserve
Runoff?

• Underwriting Results
• Correlation between subsequent year combined
  ratios 25 to 34 between 2000-2002
• For only companies near CAL correlation is 33 to
  75 (highly significant)
• Reserve Runoff
• Correlation between subsequent year runoff
  ratios33 to 37 between 2000 and 2002
• For only companies near CAL correlation is 29 to
  35
• This is good and bad news
• Statistical relationship is strong
• But still only predicts a portion of the
  subsequent year outcome

15
What is the Predictive Power of Leverage?

• Gross Leverage
• Correlation between gross leverage and subsequent
  year RBC ratio change -1 to 1 between
  2000-2002
• For only companies near CAL correlation is 5 to
  3 (not significant)
• Net Leverage
• Correlation between net leverage and subsequent
  year RBC ratio change 3 to 4 between 2000-2002
• For only companies near CAL correlation is 1 to
  16 (wrong sign)
• This is not a good outcome
• Statistical relationship is weak and sign is
  sometimes wrong

16
Well then it must be Liquidity?

• Correlation between liquid assets to surplus and
  subsequent RBC change is 4 to 1 over 2001 to
  2002
• Depending on sample, relationship is not
  significant, or sign is wrong

17
In 2002 and 2003, Portion of Companies Falling to
CAL
RBC Ratio in Prior Year Total Companies in Sample Number of Companies Falling to CAL Percentage Falling to CAL
200 to 300 314 30 9.6
300 to 350 166 9 5.4
350 to 400 205 4 2.0
400 to 450 176 3 1.7
Greater than 200 3582 55 1.5
18
Retrospective Tests

• Performed on NAIC database of 2400 companies
  ending 2003
• Generally Yes/No
• Measured whether the hypothesis accurately
  predicted the subsequent year outcome, or not
• Therefore, scrubs were oriented toward invalids,
  but not toward extremes
• Measured on three metrics
• Effectiveness-Percentage of overall correct
  predictions
• False alarms-Percentage of companies flagged that
  did not deteriorate to CAL
• Failing Companies Flagged-Percentage of companies
  that subsequently declined to the CAL that were
  correctly flagged

19
Retrospective Approach

• Started by setting a threshold such as leverage
  above industry average, or combined ratio above
  110, etc.
• Based on the threshold, companies were flagged
  or not flagged
• Allowed for mixed approaches
• Leverage, reserve runoff, and combined ratio
• Reserve runoff and combined ratio
• Three year tests of reserve runoff and combined
  ratio
• Adjusted the threshold to optimize the metrics
• Based on trial and error
• Understand, this test tells us not what causes
  RBC decline, but what best predicts it
• Although the implication for the cause is pretty
  clear

20
Retrospective Metrics
21
An Effective Approach Based on Tests
22
Further Metrics from the Combined Ratio Trend Test
23
Why not 100 Effective

• Formula approach doesnt account for capital
  changes (contributed, dividend)
• Financial statements can always be subject to
  restatement
• RBC ratio decline could involve fraud, or an
  other wise solid looking asset losing value
• Or pension funding
• The statistical relationship is strong, but is
  not 100 predictive of direction and magnitude
• Need to keep the test simple

24
Could a PC RBC Trend Test be Appropriate?