Certification Maintenance Credit Portfolio,

1
Evaluation Results and Lessons Learned,

• Certification Maintenance Credit Portfolio,
• One Year Later!

James McGrath, P.Dt, CDE Credit Portfolio
Evaluation Committee Thursday, October 19th, 2006
2
Presentation Objectives

• At the end of this session, participants should
  be
• able to
• State and report to others the 2006
  certification maintenance (CM) credit portfolio
  (CP) survey results.
• Participate in future credit portfolio programs
  using the knowledge gained from the 2006
  evaluation process.
• Describe revisions and clarifications that have
  been made to the credit portfolio option as a
  result of the evaluation process.

3
Overview of Presentation

• Review key components of the CP Program
• Summarize the CP Evaluation Plan
• Overview of CDE Participants Use of the 2006 CP
  
• Program
• Share results and feedback from the CP
  evaluation
• surveys
• Describe revisions and modifications being made
  to
• enhance the CP program in 2007

4

• Key Components of A Certification Maintenance
• Credit Portfolio
• Program

5
What Is The Certification Maintenance Credit
Portfolio (CP) Program?

• A new method diabetes educators in Canada can now
  use to maintain their certification as a
  certified diabetes educator (CDE)
• It is a customized, structured process that
  allows individual CDEs to document their own
  professional development activities
• Each activity has been assigned a Credit Value
  that is based on a variety of indicators audit,
  knowledge and professional

6
What Is The Certification Maintenance Credit
Portfolio (CP) Program?

• Participants can build a Portfolio of
  activities totalling 250 credits over a 5 year
  period
• The CP process is based on a system of credits,
  not hours
• To be eligible, a diabetes educator at the
• time of application must be a current CDE
• Program was launched in 2006, and was first made
  available to those CDEs who certified or
  recertified in 2001

7
Credit Portfolio ProgramPrimary Objective

• To provide a voluntary, accessible, valid and
• relevant process for certification maintenance
• that recognizes the multidisciplinary diabetes
• education specialists who are engaged in
• promoting excellence of care for individuals
• and communities affected by diabetes.

8
6 Major Categories of Learning Activities

• 1. Practice Review and Self Assessment
• 2. Organized Learning
• 3. Personally Designed Learning
• 4. Educational/Developmental or Teaching
• 5. Publications, Research and Quality Improvement
• 6. Professional Leadership

9
Total Credits Needed? 250 over 5 years

• Can spread credits over 1 to 5 years
• No particular number of credits per year
• Eg. If you take a leave of absence for one year,
  you still have four years to collect credits

250 Is The Minimum Number of Credits To Submit
10
Capping of Activities

• No more than 50 of credits can be claimed from
  any one of the six categories
• Credits must be from at least two (2) of the 6
  categories
• No limit or restriction on the number of
  categories you may use
• You do not need to collect credits in each
  activity category

11
Collecting Certification Maintenance Credits
The How To Guide

• Provides you with all the
• basic information
• tips and tools
• application forms
• Allows you to successfully use the credit
  portfolio process
• Can be downloaded from www.cdecb.ca or contact
  CDECB for a hard copy - (905)
  838-4898 -
  cdecb_at_sympatico.ca

12
Grandfathering Period

• The CP option will be phased in over four years
  (2006 2009)
• CDEs, beginning February 2006, needed to obtain
  only 20 of the 250 total credits normally
  required

13
Grandfathering Process

14
Credit Portfolio Process
In A Nutshell

• 1. Current CDEs participate in activities to
  obtain the required number of credits
• 2. A CDE maintains his/her own records and
  documentation
• 3. Submit your application form, Summary Form C,
  and payment by annual due date February 1st
• 4. Submit all required documentation to CDECB
  only if audited

15
Certification MaintenanceCredit Portfolio
Evaluation Plan
16
Certification Maintenance CP Program
Committee Structure
Certified Diabetes Educators
17
CP Evaluation Committee Members

• Nola Kornder (Saskatchewan)
• Joyce Wardle (Ontario)
• Kathy Younker (Texas/PEI)
• Linda Patrick (Ontario)
• James McGrath (Nova Scotia)

With support from Credit Portfolio Framework
Audit Committees, CDECB Board Members Joanne
Currie, CDECB Coordinator
18
Why Evaluate The CM Credit Portfolio Program?

• To ensure that the new CP program for
  certification maintenance is meeting its program
  objective's and that it is acceptable to those
  participants using the program.

19
Credit Portfolio Program Evaluation Plan
Accessible

• CP Evaluation committee was struck in Spring 2003
• Developed and implemented 5 year evaluation
  framework
• 5 CP program evaluation indicators will be
  evaluated over the short, intermediate, and long
  term

Valid
Multidisciplinary
Evaluation Focus Years 1 2
20
Evaluation Indicator 1Accessibility

• How many eligible CDEs chose the CP option in
  2006?
• Is the CP option being used by all eligible
  health professions?
• What are the reasons CDEs choose Credit
  Portfolio or the Written exam?
• What was the average number of credits achieved
  by CDEs in Year 1?
• Can most educators reasonably achieve the
  required credits?
• What barriers exist for CDEs who want to
  participate in the CP process? (i.e. cost,
  ability to access continuing education, cap of no
  more than 50 of credits from a single category,
  etc.)

21
Evaluation Indicator 2Sustainability

• Is the CP option revenue neutral for CDECB?
• Is the CP option affordable for the participants?
• How satisfied were CDEs with the new CP option?
  The handbook? Communication processes?
• What was the satisfaction level among those CDEs
  who were audited?
• Will CDEs from Year 1 choose the CP option the
  next time they recertify?

22
Evaluating The 2006 Credit Portfolio Program
CDE's
23
Evaluating CDEs Participation With The Credit
Portfolio Option In 2006
24
Credit Portfolio Summary Form

•  
•  
•  
•  
•       

25
CDEs Embraced The Credit Portfolio Program in
2006

• 155 CDEs from 2001 chose to maintain
  Certification
• 131 CDEs chose CM
• by Credit Portfolio
• 24 CDEs chose CM by
• Examination

84 Participation Rate In Year 1
26
Why Did CDEs Choose The Credit Portfolio Option
Over The Written Exam Method?
Q 18 - Taken From The 2006 Credit Portfolio
Satisfaction Survey

• Involved in a number of employment related
  activities
• Already recording professional activities
• Able to focus learning in a specialty area
• Exam results do not accurately reflect ability as
  a CDE
• Recognition for independent learning
• Do not like writing the exam
• Do not like preparing for the exam

83.7 69.6 45.7 40.2 39.1 38.0 33.7
27
So, Why Did CDEs Choose The Written Exam Over
The New Credit Portfolio Option?
Q 13 - Taken From The May 2006 CDE Post Exam
Survey

• Did not have time to review documents / records
  by the February 1 deadline
• Did not have all of the necessary records or
  documents
• Felt confident with writing the exam.
• The Credit Portfolio option seemed too
  complicated to learn and complete.
• Not aware that there was a new Credit Portfolio
  option

32 30 25 25 23
28
2006 Credit Portfolio Program Who
participated?
Percentage
Nurses
Dietitians
Pharmacists
29
CDEs From All Across Canada Used Credit
Portfolio In 2006
Number of CP Applicants
Province
30
Average Number of Final Credits Submitted
Credit Value
Nurse
Average
Dietitian
Pharmacist
31
Comparison of Final Credits Submitted
89.5 of CDEs Submitted More Than The Lowest
Number of Credits (50 credits) Required in 2006.
Credit Value
32
Did CDEs have a wide variety of learning
activities? Yes
33
Was there a difference in use of activity
categories among health professions? Not Overly!
2
4
3
5
6
1
RNs and RDs participated fairly equally in
activities. Pharmacists were not as active in
using categories 4, 5 or 6, but did favor 3!
34
Results and Feedback From Credit Portfolio
Evaluation Surveys
35
Survey 1Credit Portfolio Satisfaction Survey

• Purpose
• 1) To gather feedback from CDEs about their
  experiences and satisfaction with participating
  in the new Credit Portfolio option.
• 2) Use the data collected to improve the
  function and delivery of the program so that it
  will further meet the CP program objectives and
  be acceptable to those participants using the
  program.

Accessibility
Sustainability
36
Satisfaction Survey Results
All of the questionnaire results for the CP
Satisfaction Survey will be posted on the CDECB
website!
37
Q1 - How did you learn about the Certification
Maintenance Credit Portfolio option?
38
Q8 - Completing the application and credit
portfolio summary form was easy.

• 54.3 agreed or strongly agreed
• CDEs expressed feelings of gratitude,
  happiness, frustration, and confusion.
• CP Summary Form C document has been revised and
  improved

39
Q 8 - Comments Feedback
I really liked it once I got organized. It
will be easier next time!
First time through is always difficult
As a new concept, it required careful studying
at first to become familiar with it
Lots of reading to understand what was
required.
Credit portfolio was less stressful
I simply appreciate the fact there is a credit
portfolio available. I am very stressed by exams
and tend to not think clearly.
A lot of work especially because it was the first
time, from now on I know what is needed and will
prepare all along.
40
Q9 Finding information about the Credit
Portfolio option on the CDECB web page was easy.

• 81.6 of CDEs agreed or strongly agreed.
• CDECB web page is regularly updated by
  administrative support staff.

41
Q10 I found the Frequently Asked Questions
section of the web page useful.

• 66.3 of CDEs agreed or strongly agreed.
• FAQ s is an effective communication management
  tool and time saver when asking general questions.

42
Q11 In the How To Guide, the process for
completing the credit portfolio was adequately
explained.

• 68.5 thought the process was adequately
  explained.
• Concerns - size of how to guide, not enough
  examples, and vagueness of directions on
• some forms.
• How To Guide now contains clearer
  explanations, more examples, and updated forms.

43
Q15 When I contacted the CDECB office regarding
the CP option, my answers were answered in a
timely fashion.

• 41.3 answered N/A
• 60.0 agreed/strongly agreed
• Mixed feedback Question answered within 48
  hrs, thanks Phone not good, email good
• Technical questions go to CP Development
  Committee (volunteers) Some questions require
  research to give the best answer.
• Questions re CP should be submitted by email or
  fax (Activities For Certification Maintenance
  Request Form) prior to January 15

44
Q18 Overall, I was satisfied with this method
of maintaining my certification.

• 85.9 agreed or
• strongly agreed
• and were satisfied
• with the CP method.
• 5.4 of participants were not satisfied (i.e.
  preferred the exam)
• 8.7 responded neutral

45
Feedback On Credit Portfolio From 2006
Credit portfolio is easier
Overall the process was easy b/c I am
committed to education in a variety of venues.
Preparing the portfolio allows me to reflect
on my own practice.
CP recognizes the vastness of diabetes educators
in their communities
Great Job!
It can be done over a longer period of time and
be individualized to my personal work place
opportunities and needs.
After rereading, it was easy to come up with
the 50 credits.
I was delighted with the CP option. Ive
already started to collect for my next recert in
2011.
46
Q19 After this experience,

• 88.0 would recertify again by CP
• 6.5 will not
• recertify again (i.e.
• retirement)
• 4.3 will recertify next time by exam. (i.e. felt
  CP was more work for them, exam is a good chance
  to review
• diabetes, not organized enough to do CP,
  etc.)

47
Credit Portfolio Audit Process Survey
48
What Is The Credit Portfolio Audit Process?

• Another method used to evaluate the
• CP Program. The audit process
• verifies that participants using the CP
• program for certification maintenance
• have followed the appropriate
• guidelines used in the development
• and delivery of the CP program.

49
The 2006 Audit Process

• A predetermined number of Portfolios are
  randomly selected for
• audit in early February
• Written notification requesting CDE to forward a
  triplicate copy of
• their portfolios postmarked no later than
  April 10
• Participants have about 6 weeks to submit their
  portfolios
• Each portfolio is audited by 2 separate
  auditors, 3rd audit, if
• necessary
• Auditors job is to ensure CDEs correct
  participation in the CP
• process. (i.e. a minimum of 50 credits was
  submitted, learning
• activities fit submission criteria and are
  matched correctly to each
• learning category, etc.)

50
2006 Audit Findings

• 50 credits (20 of 250 credits) were required
  in 2006
• 87.5 of audited CPs over submitted the min of
  50 credits.
• The percentage of credits miscategorized and
  moved to another
• category was minimal.
• 91 of portfolios audited didnt have any
  credits moved to
• another category.
• An example of a common error picked up during
  audit
• 2A short term learning events
• - Incorrect calculation of enough
  credits, should be 12 credits
• - Some people exceeded the allowable
  credit value and the credit
• value was lowered to 24
  credits
• - Several people did not submit
  receipts

51
Survey 2Audit Survey Results

• The purpose of the Audit Survey was to evaluate
  CDEs who were selected for audit and determine
  their satisfaction and any issues they may have
  encountered with the audit process.

52
Q2 The amount of time for me to gather and
submit my Credit Portfolio package to the CDECB
office was sufficient.

• 54 agreed or strongly agreed and thought there
  was sufficient time.
• 31 disagreed and felt they did not have enough
  time
• Issues Getting signatures from managers, and
  not already having a Credit Portfolio package
  ready for audit.

Prepare for the Credit Portfolio as if you were
going to be audited!
53
Q4 When I contacted the CDECB office regarding
the Credit Portfolio Audit process, my questions
were answered to my satisfaction.

• 61 checked N/A
• 58.5 agreed or strongly agreed and were
  satisfied.
• Issues Length of time for a response was too
  long and response too close to the Audit
  submission deadline.

• FAQs for Audit now on
• CDECB website
• Technical questions may
• need extra time for an
• answer! Ask audit
• questions early on.

54
General Comments From The Credit Portfolio Audit
Survey
Because it was new,
there were some growing pains but I think
the next time will be easier already collecting
info now.
It was a good process
in that I will now be more organized in filing
and getting a paper trail for what I attended.
Knowing that you might be
audited ensures that your portfolio is organized.
55
Friendly Audit Reminders

• FAQs are listed on the CDECB website
• Read all directions and know what documentation
  to include in your package
• Ask questions before Jan 15th by email/fax
• Ensure all forms are completed
• Dont submit documents in plastic sleeves
• If chosen, it is mandatory to participate in the
  audit
• Submit all documentation in triplicate (3s)
• If there is a deficiency in documentation, the
  CDE candidate will not be notified and credit may
  not be given
• Ensure forms have appropriate signatures
• If you have more than the required credits, (100,
  150, 200, 250) we encourage you to submit extra
  in case some are not accepted.
• The credit portfolio Summary Form C has been
  updated to include a category for credits from
  the CP pilot project

56
Summary of CDE Credit Portfolio For
Certification Maintenance2007
57
Collecting Certification Maintenance Credits
The How To Guide 2007 Edition

• Updated How To Guide (2007) - Directions and
  explanations for completing the CP application
  process, more examples of acceptable and non
  acceptable activities, updated forms, etc.
• Can be downloaded from www.cdecb.ca or contact
  CDECB for a hard copy - (905)
  838-4898 -
  cdecb_at_sympatico.ca

58
Summary Form C - The How To Guide
59
Organized Learning Activities

• 2A - Short Term
• Learning Events
• A variety are available
• Conferences / Workshops
• Seminars / Lectures
• Sponsors may include
• CDA, ADA, AADE
• college or university
• health care institution
• professional association
• Self report on form 2A

• lt 4 hrs, no docs needed, 1 credit/hr
  or part of an hr, to a maximum of 4
  credits/event.
• gt4 hrs, doc needed, 1 credit/hr, max 6
  credits/day
• Insulin pump training retain certificate, 4
  credits, count 1 every 5 yrs (Printing mistake in
  05/05, should 5 credits)

60
Organized Learning Activities

• 2A - Short Term
• Learning Events
• For events that include partial hrs, round up to
  the next full credit. (i.e. 1.5 hrs - count as 2
  credits
• Max 6 credits/day, total of 18 credits with
  documents
• Preconference - ½ day 2 credits, no docs, Full
  day 4 credits max, no docs

• Short courses - Not part of a formal academic
  program (i.e. insulin pump training short self
  study courses provided on-line)
• DRI modules have moved
• from 2B to 2A
• Questions Email CDECB if uncertain about course
  eligibility
• Credits - 5 credits / course or module (counted
  1/5 yrs)
• Proof of completion - Retain for audit

Short Courses Eligible For Credit Has Expanded
Clarification on Credits
Credits For CDA Conference
61
Personally Designed Learning Activities

• ½ credit / article read, CD, video or audiotape
• 4 credits / book read
• 1 credit per article with post test (Retain
  results of post test for articles, as applicable)

• 3B - Self Study
• journal articles, books, videos, CDs or
  audiotapes
• journal articles must be from a peer-reviewed
  journal
• Canadian Journal of Diabetes
• Diabetes Care
• The Diabetes Educator
• Document self study items on form 3B

62
Personally Designed Learning Activities

• 3B - Self Study
• Clarification CJD with the CPG, Vol 27, Supp 2,
  Dec, 2003
• Count ½ credit / article. If the whole issue is
  read, it will be 12.5 credits. Dont round!
• 2006 Some CDEs counted the CPG issue as a book
  (4 credits), others as an article (1/2 credit)
• Diabetes Communicator 1 credit / issue due to
  more short referenced articles larger issue.
• Websites Added in How To Guide - Referencing peer
  reviewed journals

63
Educational / Developmental or Teaching
Activities

• 4E - Short Presentations
• or Teaching
• Topic must be diabetes related
• Audience
• Other health professional practitioners
• Students
• Public presentations external to diabetes program
• Document activities on form 4E
• Several examples given
• 5 credits / presentation or teaching

Questions From CDEs in 2006. Activity
4E rewritten.
64
Educational / Developmental or Teaching
Activities

• 4E - Short Presentations
• or Teaching
• Eligibility- Preparation or teaching cannot be
  usual diabetes educator practice and must involve
  new learning for CDE
• Examples - 2007 How To Guide
• A new form has been developed.
• Already using the 2006 form, continue until you
  need a new page, then switch to 2007 form.
• Audited Send only form 4E!! Do not include
  presentations / program brochures

Example (Acceptable) Updating peers following
CDA conference (i.e. requires CDE to understand
and explain information and perhaps relate to
clinical situations)
Example (Not Acceptable) TV or radio appearance
to explain risk factors of Type 2 diabetes.
65
Educational / Developmental or Teaching
Activities

• 4F Presentation As Part Of A Professional
  Conference
• Title Change- The word Professional has been
  added to the title to help CDEs determine
  whether or not their presentation qualifies.
• Conference must be sponsored by a professional
  body academic or healthcare organization, or
  industry.
• 10 credits per conference presentation.
• Examples now added to the How To Guide.
• If uncertain if a presentation qualifies, contact
  CDECB before Jan 15.

• Example (Acceptable)
• Presentation of a session at
• a diabetes conference
• sponsored by a provincial
• diabetes program.
• Poster session at the
• CDA/DES annual conference

Example (Not Acceptable) Presentation at CDA
sponsored public forum or expo
66
Publications, Research Quality Improvement
Activities

• 5B - Non-peer reviewed publications and peer
  reviewed case studies or letters
• Includes preparation of papers, pamphlets,
  internet articles, and book chapters.
• Case studies or letters (even if in a peer
  reviewed journal)
• Examples - Diabetes Quarterly (Diabetes
  Communicator) and Diabetes Dialogue
• Audited Send form 5B and copy of letter of
  acceptance for publication
• 5 credits / publication

The Diabetes Communicator If the article is
original writing with references and not a
synopsis of something previously written, a
review of another author, or a review of
websites, books or conferences, it will receive 8
credits.
67
Professional Leadership Activities

• 6A Professional
• Leadership
• CDE guides or directs the decision making process
  of a group, thus enhancing their diabetes
  education knowledge or practice
• Examples
• DES, CDA boards
• Regional, provincial or federal advisory/task
  forces
• 20 credits/ group / year
• Letter of appointment

Reminder / Clarification The CDE must
demonstrate both professional leadership and be
able to describe specific outcomes. The activity
should not be a part of a CDEs regular work.
(i.e. facilitating a DECs strategic plan.)
68
Professional Leadership Activities

• 6B - CDE Acting as an
• Expert / Advisor
• CDE appointed to a professional committee or task
  force
• Acts as an expert on topic's) related to diabetes
  education/care
• evaluate a teaching tool
• hypertension working group
• Document on form 6B
• Letter of appointment
• 10 credits / group / year

Additional Activity Added! Membership on the DES
Standards Review Program Committee has been added
to this activity for credit.
69
Summary

• The credit portfolio program for certification
  maintenance was overwhelmingly successful in Year
  1.
• 84 of eligible CDEs chose the Credit Portfolio
  option over the Exam method.
• 88 of CDEs from 2006 plan to use the CP option
  again as a method of maintaining their CDE
  certification
• The Credit Portfolio project has met the
  evaluation criteria of Accessibility and
  Sustainability
• Most of the Bugs with the credit portfolio
  option have been worked out!
• CDECB and the Credit Portfolio sub-committees
  (Framework, Audit and Evaluation) are committed
  to ensuring the CP option is meeting its program
  objectives and that CDEs are satisfied with the
  CP option.

70
Questions

• Contact information
• 1. Email - creditportfolio.cdecb_at_sympatico.ca
• (Activities For Certification Maintenance
  Request Form)
• 2. FAQs _at_ www.cdecb.ca
• 3. Phone [phone]
• 4. Fax - 905-838-4899
• (Activities For Certification Maintenance
  Request Form)

71

• Questions and Answers!