Consensus Week Process

1
Consensus Week Process
Jan 08
2
Logistics Information

• Building Hours
• Normal hours 8 to 5
• Building will be locked at 5 but teams can work
  later
• Rooms
• We will use rooms B, C, H, and I for
  teams
• Food and coaches will be in room F upstairs
• Each room will have an SCF projector and laptop
  that is networked into a printer
• Each room will have a shred basket that will be
  shredded at the end of each day

3
Logistics Information (cont)

• Meals
• Continental Breakfast served at 8
• Lunch served at 12
• Food will be located in room F

4
Desired Objectives

• Understand the Consensus Week and Site Visit
  processes
• Become familiar with the contents of the Examiner
  Process Handbook
• Understand key factors and their use in
  evaluating an organization
• Understand how to identify key themes
• Gather input on process improvements and best
  practices

5
Desired Objectives (cont.)

• Learn how to write understandable, actionable
  feedback comments including so whats
• Process comments
• Results comments including use of the Category 7
  Results Matrix
• Learn how to synthesize independent review
  comments into a consensus comment
• Learn how to analyze results and write feedback
• Learn how to score an application

6
Desired Outcomes (cont.)

• Examiners feel prepared for the consensus and
  site visit processes
• Examiners feel Consensus Week was worth the time
  and effort
• Resources are used effectively

7
Introductions

• Your name
• Your employer
• Your job title
• One ah-ha learned so far

8
Consensus Week Process Review
9
Consensus Week

• Just-in-time and Just-enough training will be
  delivered to all Examiners
• Each examiner will be assigned as lead for a
  Category and/or a group of Items
• During Consensus Week, the application team will
• clarify and resolve differences in strengths,
  opportunities for improvement, and scoring
• reach consensus on feedback comments
• reach consensus on a score
• learn about the site visit process

10
What is Consensus?

• A decision reached by a group
• May not be the first choice of all team members
• Each team member agrees he or she had an
  opportunity to express his or her opinion
• Alternatives have been discussed
• Each team member agrees to support the decision
  of the group

11
Consensus WeekProcess, Comments Scoring

• Open Process Handbook
• to Consensus Week Section

12
Consensus Week Process
Receive Criteria Overview
Revise Based on Feedback from Coach
Submit Comments Score to Coach
Identify Key Factors for Item
Review Key Themes
Identify Strength OFI Topics
Reach Consensus on Score
Identify IR Comments Related to Topic
Reach Consensus on Comments
Narrow Topics to 6-10 Per Item
Synthesize Comments Related to Topics
13
What is a Strength?

• The applicant answers the criteria question
• The applicant describes a systematic approach
• The applicant provides information about the
  extent of deployment
• The applicant identifies measures used to
  determine the effectiveness of the approach
• The applicant describes cycles of improvement to
  the approach

14
What is an Opportunity for Improvement?

• The applicant does not answer the criteria
  question
• The applicant provides examples, but does not
  describe a systematic approach
• The applicant does not provide information on who
  is involved in a process or when it happens
• The applicant has a process that is in the early
  stages of development or deployment
• The applicant lacks measures or cycles of
  improvement
• The applicant uses the future tense

15
What is a Systematic Process?

• Repeatable
• Uses data and information so learning is possible
• Builds in the opportunity for evaluation,
  improvement and sharing
• Inputs are described
• Key steps are described
• Key performers are described
• Outputs are described
• Measures are described

16
Checklist for Comments

• 6 to 10 Comments
• Comment for each sub-area
• Each comment ties to the criteria
• No conflicting strengths and OFIs
• OFIs will help the applicant move to the next
  level
• OFIs have solid so-whats showing negative
  impact
• Comments follow other content style guidelines

17
Consensus WeekScoring
18
The Scoring System

• Each Item is scored using an anchored rating
  scale
• - Provides a link between the Criteria language
  and Scoring Guidelines
• The Scoring Guidelines focus on two dimensions
• - Process
• - Results
• The process dimension focuses on four elements
• - Approach - Learning
• - Deployment - Integration
• The results dimension focuses on four elements
• - Performance Level - Comparisons
• - Trend - Importance

19
The Scoring System

• Review comments
• Minor gaps?
• Major gaps?
• No right answer - calls for rich discussion
• Consideration is given to what is important to
  the organization as described in
• Organizational profile
• Key strategies
• Coach will help score

20
Basic, Overall, and Multiple

• Basic Item requirements Item title
• Overall Item requirements Item description
• Specific topics applicants need to address
• Multiple Item requirements Areas to address
• Individual criteria questions

21
Give Benefit of the Doubt

• Dont discount an approach because
• it didnt work in your organization
• other approaches are used more effectively in
  your organization
• Ask what else would I need to see to move to the
  next higher scoring range?
• If its not significant, move it up
• If the team decides to not give benefit of the
  doubt
• OFI should be specific about what is unclear
• OFI should specify what SIGNIFICANT results or
  information are missing

22
Use Best Fit

• Score using an overall, holistic sense of the
  scoring range descriptors
• Dont use the individual bullets of the scoring
  range as a checklist
• An organization doesnt need to satisfy all
  elements of all of the bullets in order to fit
  into a scoring range
• Best fit is NOT the highest scoring band where
  you can check all the bullets

23
(No Transcript)
24
(No Transcript)
25
Check Uniformity

• If you are getting the same score for most or all
  of the Items, check your scoring again
• Its unlikely that an organization will be
  uniform across the board

26
Factors that Negatively Impact Scores

• Seeing 50 as mature and highly evolved
• Using the criteria as a checklist
• Expecting too much evidence
• Not using the Key Factors to determine importance
• Placing more weight on opportunities for
  improvement
• More acceptable to look tough

27
Factors that Positively Impact Scores

• Accepting vague statements as sufficient evidence
• Accepting examples of the what as evidence of
  the how
• Giving credit based on an organizations
  reputation

28
Consensus Week Ground Rules
29
Ground Rules for the Week

• YOU MUST BE PRESENT TO LEARN!
• You must be here for all of Consensus Week
• Always be on time in the morning, after lunch,
  and after breaks
• Dont leave early
• THIS IS NOT A 900 - 500 TYPE OF TRAINING
• Be prepared to work late each evening
• The team will decide what hours to work

30
Ground Rules for the Week (cont.)

• Dont be too harsh on the applicant
• Focus feedback and scoring discussions on the
  criteria requirements and scoring guidelines
• Remember that the purpose of consensus is to get
  the best collective thinking of the team
• Stick to the overall schedule of events - call a
  process check or ELMO when team gets off track
• Actively manage the time schedule
• Listen to other points of view
• Encourage participation by all team members

31
Ground Rules for the Week (cont.)

• Strive for consensus
• Consensus means
• Each person has been heard
• Each person can and will support the decision of
  the team

32
Consensus Week Coach Responsibilities

• Mentor and assist application team throughout the
  process
• Provide feedback on consensus comments
• Provide feedback on consensus scores
• Monitor adherence to schedule
• Ensure Consensus Week is rewarding and successful

USE YOUR COACH!
33
Key Success Factors

• Meet all time commitments - the team is depending
  on you!!
• Maintain thorough documentation
• Be open to the opinions of other team members
• Learn about the applicant, the criteria, and your
  fellow examiners

34
Parking Lot Questions