Performance Based Studies Research Group

1
Performance Based Studies Research Group
State of Oklahoma Risk Management Training
www.pbsrg.com
2
State of Oklahoma Strategic Plan

• Compete based on value (proven performance and
  expertise)
• Transfer risk and control to vendor
• Minimize client decision making, directives, and
  control
• Vendor writes majority of the contract
• Vendor manages contract
• Measurement, measurement, measurement

3
State Vendor Expectations

• Proactive and Accountable
• Vendor has control
• Focus on risk in the seams
• Risk Plans to minimize risk impact customer
  satisfaction
• Performance measurement uses dominate
  information
• Performance information/risk identification
  disengages the bureaucracy
• Win-Win
• Vendors success is just as important as States
  success
• Goal alignment
• No contract Vendor writes their own
• Vendor has control
• Contract is a guide
• If we have to go to the contract then everyone
  has already lost
• Should never have to go the contract

4
2009 State of Oklahoma Consultant Selection and
Risk Management Process
Filter 1 Registration Past Perf. Info.
Filter 2 Scope RAVA plan
Filter 4 Best Value Prioritization
Filter 5 Pre-planning Phase WR and Scope
Filter 6 Weekly Report Post-Rating
Filter 3 Interviews
High
Criteria Interview RAVA PPI Qual.
Criteria RAVA Tech. PPI Qual.
Award
Shortlist
Identify BV
Low
Time
5
Final Rating

• After the project is complete, the client will
  evaluate the project.
• The Final Rating will be heavily impacted by
• Change orders
• Project delays
• Poor quality
• Owner surprises
• Complaints
• Ability to submit accurate and timely weekly
  reports
• In Best-Value Procurement, the vendors goal is to
  get a high rating in order to stay competitive.

5
6
50 Modification
6
7
State of Oklahoma Risk Management Process

• Vendor takes into account all client requirements
  and constraints
• Vendor creates a baseline (cost, schedule, scope,
  quality)
• Vendor identifies all risks that they do not
  control that could impact baseline plan and shows
  a plan to minimize them.
• Vendor takes into account all client concerns and
  shows how they will be minimized.
• If any deviation occurs during the project it is
  documented and tracked on the weekly report.
• Vendor will be rated by the client on their
  deviation from the baseline.

8
The Weekly Report

• An excel document that tracks pre-identified
  risks/concerns and actual risks/concerns on a
  project.
• Pre-Identified Risks/Concerns
• Any concerns of the client at any point in the
  project
• Any risks identified before the project begins
• Actual Risks/Concerns
• Risks that are unforeseen
• Risks that the contractor feels has a good
  probability of occurring
• All information on the weekly report must be in
  Dominant fashion.
• Dominant means
• Simple, clear, concise, accurate..
• Easy to understand without a lot of information.

9
Weekly Report Schedule

• Inserted in the ScheduleBudget tab of the
  weekly
• The schedule should go from task order award till
  construction close-out (even if contractor was
  only awarded till the design/workplan phase)
• The schedule items should include the following
  types of activities
• Major milestones
• High risk activities
• The schedule should be updated every week.
• It should have incorporated impact of risks
• It should accurately reflect the current status
  of the project

10
Reporting Pre-Identified Risks

• Before a project begins the vendor should have
  done the following
• Talked with all client parties and documented
  their concerns
• Identified any potential risks that the vendor
  does not control
• Created a plan to minimize all concerns and risks
• Placed all concerns/risks and minimization plans
  in the weekly report in the RMP tab
• Throughout the course of a project if anyone has
  concerns or identifies more potential risks the
  vendor should add them to the RMP tab with plans
  to minimize them.
• If a pre-identified risk becomes a potential
  problem on the project it is then documented on
  the Risks tab as well.

11
Reporting Actual Risks

• Actual risks should be identified on the RISKS
  tab of the weekly report
• When an actual risk is identified the following
  should documented
• How it became a risk
• If it was not pre-identified, an explanation of
  why it was not
• A dated history of actions performed to minimize
  the risk
• Impact to cost and time
• Owner rating of risk
• Until a risk is resolved the following must be
  updated
• Estimated impact to time and cost
• Actions performed to minimize the risk
• Owner rating of the risk
• Every resolved risk with an impact, must clearly
  be linked to a modification on the
  ScheduleBudget tab.

12
Industry Structure
High
II. Value Based
III. Negotiated-Bid
Owner selects vendor Negotiates with
vendor Vendor performs
Best Value (Performance and price
measurements) Quality control
Best Value is best predictable performance
Performance
I. Price Based
IV. Unstable Market
Specifications, standards and qualification
based Management Inspection
Award is based on subjective minimum which has
to be enforced
Competition
High
Low
13
Problem with Priced Based Systems proliferates
decision making
Owners The lowest possible quality that I
want
Contractors The highest possible value that you
will get
Maximum
Minimum
14
Impact of Minimum StandardsBlind vs Visionary
High
Low
Risk
Performance
Contractor 1 Contractor 2 Contractor 3 Contractor
4
Low
High
Decision making what is the minimum standard,
and do all contractors meet the minimum standards
15
Traditional PM vs. New PM Model
16
Accurate perception leads to less decision making
and expectations
Initial conditions
Final conditions
Laws
Laws
Time
17
Best Value Selection
18
Industry performance and capability
Vendor X
Customers
Highly Trained
Outsourcing Owner
Partnering Owner
Medium Trained
Minimal Experience
Price Based
19
Impact on Professionalism

• Best Value Environment
• More accountability
• More expertise
• More professionalism
• Price Based Environment
• Less accountability
• Less expertise
• Less professionalism
• Less perceived value added
• Movements to make designer work for contractors
  due to lack of value added

20
Dominant information

• What does the most qualified, experienced people
  in a firm do?
• Get work, or do work?
• Why is this?
• It is more important in a destabilized industry
  to get work then to do work.
• Downside.what happens to our professionalism????

21
ASTM Standard Setting
22
ASTM Standard Setting
23
What Will Manufacturer C Do?
Standards motivate manufacturers to lower quality
24
There is something wrong with the delivery of
services..
No one knows how bad the problem really is..
Entire system is broken. Requires more
management. Performance/value is
decreasing. Relationships are more important
than results.
25
As management, control, and direction become more
important..
Management
.it becomes less important to be skilled,
accountable, and able to minimize risk
26
Managers Code The movement of risk.....when
making decisions
Is It Working?
NO
YES
Dont Mess With It!
Did You Mess With It?
YES
YOU IDIOT!
NO
Anyone Else Knows?
Will it Blow Up In Your Hands?
YES
Youre SCREWED!
YES
Can You Blame Someone Else?
NO
NO
Look The Other Way
NO
Hide It
Yes
NO PROBLEM!
27
Impact on Designers Value and Professionalism

• Best Value Environment
• More accountability
• More expertise
• More professionalism
• Dominantly clear that the designer can add the
  most value
• Price Based Environment
• Less accountability
• Less expertise
• Less professionalism
• Less perceived value added
• Movements to make designer work for contractors
  due to lack of value added

28
What is the difference?Do designers spend more
time in reacting to bureaucracy or doing a good
technical job?
2
1
3
4
29
Traditional PM vs. New PM Model
30
Evolution of Delivery Processes
Design-Bid-Build (DBB)
Design
Build
Design-Build (DB)
Design-Build
Construction Manager At Risk (CMAR)
Design
Build
31
Where is value added?????
Design-Bid-Build (DBB)
Design
Build
Design-Build (DB)
Design-Build
Construction Manager At Risk (CMAR)
Design
Build
32
Who should be the center of the Universe in
construction? In design?
High Performing Vendor
Outsourcing
Client
Low Performing Vendor
Direction, control, review
33
CMAR Best Practices
0
30
60
100
Design Intent
Baseline
Final Design
Designer
1
3
5
Contractor
2
4
Scope Cost Est.
Re-Scope w/Modifications
Bonding capacity
34
Self Regulating Loop(Six Sigma DMAIC Generated)

• Actions
• Minimize data flow
• Minimize analysis
• Minimize control

Scope Risk Assessment/Value Added Plan
M
R
Interview Key Personnel
M
R
Identify value (PPI, RA, Interview)
V
M
M
Preplanning, Quality Control Plan
R
M
50
50
M
M
Measure again
M
R
35
2009 State of Oklahoma Consultant Selection
Process
Filter 1 Registration Past Perf. Info.
Filter 2 Scope RAVA plan
Filter 4 Best Value Prioritization
Filter 5 Pre-planning Phase WR and Scope
Filter 6 Weekly Report Post-Rating
Filter 3 Interviews
High
Criteria Interview RAVA PPI Qual.
Criteria RAVA Tech. PPI Qual.
Award
Shortlist
Identify BV
Low
Time
36
Past Performance Surveys/ Past Performance Sheet
Design Firm
Design Firm compiles and sends surveys and past
performance sheet back to State
Mail survey back to Design Firm
State of Oklahoma
Past Client
37
Past Performance InformationForm 254

• Design firms
• Critical sub elements
• Lead project managers

38
Traditional Management
Initial conditions
Final conditions
D
MC
Laws
Laws
Time
D
Risk is deviation from expected measurements
39
When do we determine the expectation so we can
measure performance?
Initial conditions
Final conditions
M
M
Laws
Laws
M
Time
M
Risk is deviation from expected measurements
40
Scope

• Will be rated in relative fashion
• Conservative plan on how the work will be done
• If other options are available, they will be
  considered along with the risk
• 30,000 foot level description
• Final design description
• Milestone schedule

41
Risk Assessment and Value Added Plan (RAVA)

• The Risk Assessment and Value Added (RAVA) Plan
  consists of two pieces
• The Risk Assessment
• The Value Added
• The Risk Assessment section is used to identify
  high performing vendors that can
• Identify and minimize risk before the service has
  started
• Deliver plan to minimize risk during the life of
  the service.
• The RA Plan should clearly address the following
  items
• List and prioritize major risk items (areas that
  may cause the service not to meet the
  expectations of the Government).
• Each vendor should focus on risks it does not
  control and
• Explain how the vendor will minimize the risk.

42
Risk Assessment Plan Steps

• What events cause risk
• How will the design firm minimize the risk from
  happening
• If the risk happens, what will the designer do
• Transfers control and the opportunity to minimize
  risk to the design firm

43
RAVA Plan Rated Blind

• The RAVA Plan must NOT exceed 2 pages.
• The RAVA Plan must not have ANY NAMES (such as
  vendor or manufacturer names, personnel names,
  project names, product names, or company
  letterhead)
• Do NOT include brochures, marketing information,
  or product names!
• The RAVA Plans will be evaluated blind in order
  to minimize any bias.

44
Value Added Differentiation

• Answer the questions What value do I bring that
  differentiates me from my competitors?
• The Value Added Differentiation section of the
  RAVA Plan is to allow a vendor to
• Show how it will add value (that its competitors
  cannot)
• The impact of that value in simple, provable
  terms
• How the value added will be measured must have
  an impact on dollars, time, and/or satisfaction
  of the Government
• Marketing material is worthless and will
  negatively impact scores
• Value Added is where vendors provide great
  ideas

45
How The Submittal Process Works
Average Score
Evaluation Members
46
Interviews

• The following individual will be interviewed
• Project Manager/Lead Designer
• All interviews will be individual
• The same individuals used for the past
  performance survey will be the same individuals
  interviewed and will be the same individuals that
  will be assigned to the project
• No bait and switch

47
Sample Interview Questions

• Why were you selected for this project?
• How many similar projects have you worked on?
• What is different about this projects from other
  clients that you have worked for?
• What are you doing differently on this project
  than you have done before?
• Identify, prioritize, and how you will minimize
  the risks of this project.
• What risks dont you control? How will you
  minimize those risks?
• What are your personal goals on this project?
• How are you going to measure your performance?
• What value do you bring to the project in terms
  of differences based on dollars, quality, or
  time?
• How will you measure performance for this
  project?

48
Best Value Selection

• Past Performance
• Risk Assessment (dont control) / Value Added
• Scope/Technical Proposal (2 Page
  proposal/milestone schedule)
• Qualifications
• Interview

49
Rating Scheme (1-10, 1(not qualified, 5(cant
differentiate/dont know), 10 (dominantly better)
Filter 1 Registration Past Perf. Info.
Filter 2 Selection RAVA plan
Filter 4 Best Value Prioritization
Filter 5 Pre-planning Phase WR and Scope
Filter 6 Weekly Report Post-Rating
Filter 3 Interviews
High
Criteria Interview RAVA PPI Qual.
Criteria RAVA Tech. PPI Qual.
Award
Shortlist
Identify BV
Low
Time
50
Weekly Report
Unforeseen Risks
51
What are the measurements of a Designer

• Scope and cost
• Plan with minimal risk
• Plan to minimize risk
• Knowing this with minimal effort

• Has the designer done this before
• Do they know what they dont know on a future
  project
• Can they clearly articulate their vision

52
Traditional PM Model Influence and Control
(micromanagement)

• Reactive
• Focus on changing people
• Followers are the constraint
• Requires maximum amount of resources
• Relieves management from accountability of
  nonperformance
• Inefficient and nonperforming

53
New PM model Leadership Model(measurements)

• Proactive
• Alignment (measurement)
• Requires Understanding (acceptance of
  measurements)
• Leader/manager and system is the constraint
  (blindness)
• Focus is on setting up a quality controlled
  system based on performance measurements, best
  value requirements, and dominant information
• Efficient (measurement)

54
State of OklahomaBest Value Process for
Designers and Contractors
55
Differences in new system

• Provide past performance information (PPI) from
  references
• PPI on organization and critical components
• Submittals (scope and RAVA plan) will be two
  pages each
• One additional page for milestone schedule
• Interview will be with the project manager
• How do you know you can do it
• How are you doing it
• How are you taking control of the project and
  minimizing risk and decision making of client
• How did you pick your engineering subvendors

56
Requirements on Competing Firms

• More or less time invested on a project?
• More or less detail?
• Risk they control or do not control?

• Dominant differential or technical
  qualifications?
• Risk minimization done when?
• More risky approach or conservative control?

57
Designer should separate the requirement from the
risk

• Propose the vision that they can do with
  minimial risk
• Propose to immediately analyze any potentially
  risky options in the beginning of the design
  contract
• Only pursue risky options if there is dominant
  information that justifies the perceived risk
• Cost should be based on vision and
  requirements
• Makes all other parties accountable and gives
  control of the design back to the design
  professional

58
Delivery of Design Services
Status Quo

• Control, manage, direct, and inspect
• Increase the flow of information
• Inefficient, ineffective
• Maximizes technical issues

X
M
New Design Delivery Model

• Align
• Transfer control to the contractor
• Preplanning
• Quality Control
• Measure
• Minimize flow of information
• Minimizes technical issues

M
M
M
M