ADVANCED MASTERY OF THE e-PORTFOLIO -A COMPETENCE BASED FRAMEWORK

1
ADVANCED MASTERY OF THE e-PORTFOLIO-A COMPETENCE
BASED FRAMEWORK

• TRAINERS WORKSHOP
• 12.10.10
• RIPON SPA

2
SET OBJECTIVES

• WHAT DO YOU WANT TO GET OUT OF TODAY?

3
Programme

• 930 Agenda setting
• 945 Update in e-portfolio v5 /Clustering the
  RDMP model
• 1000 How do we fill in an ES report.
  Difficulties.Hot tips.
• 1030 The journey from ES to panels
• 1100-1115 Coffee
• 1115 Validation and reflection-exercises in
  groups
• 1230 Update from Deanery
• 1245-1330 Lunch
• 1330 Trainees in Difficulty
• 1500-1515 Tea
• 1515 E-portfolio practice and validity tasks.

4
An exercise in Observation

• http//www.youtube.com/watch?vRbmai1b__i4

5
UPDATE Eportfolio v5

• RCGP update-Positive ideas
• Expanded rating scales-relate to stage-
• NFD Below Expectations NFD Meets
  Expectations NFD Above Expectations
• Competent for Licensing
• Excellent
• Each part is on a separate web page - helps to
  concentrate on each before a summary in the
  seventh part. Each can be saved
• All the evidence for each section is provided on
  same page.
• Eg the curriculum coverage now provides a
  curriculum map which can delve into before making
  comments.
• ES rating scales - all the evidence has been
  pooled together quite nicely. No more clicking
  out of the 'review' section and having to delve
  into other menus and options first.
• Ensures you sign the educational work contract.
  You can't do a review without signing it. Means
  that we are now less likely to be crucified for
  not having signed the ed. work contract when it
  comes to ARCP time.

6
EDUCATIONAL SUPERVISON

• What do we find hard?
• Preparation STs
• Preparation ES
• Documentation-ES report
• Ensure that judgments are fully evidence based
  and referenced.
• Controversies

7
ES Controversies

• Time
• Pay
• Validity
• Lack of Guidance
• Do we need to spend as much time for all? Or
  concentrate on those with difficulties?
• Giving feedback.
• Who reads? CS or ES?
• Validation and linkage
• Reflection

8
STs-PRE ES meeting

• Arrange meeting!
• log and share an e-portfolio entry entitled Ed
  Sup Rev current date. Attach
• COT CBD competency mapping
• HDR spreadsheet and sick leave/all leave
  spreadsheet. List complaints.
• Ensure CSR report done.
• Ensure self rating assessment and PDP up to date.
• Ensure compulsory assessments (inc MSF in modular
  posts)?include NOE
• Ensure last objectives achieved.

9
ES pre meeting

• How much do we look at ?
• How long should we spend?
• How to gain the evidence for competences? See
  appendix
• ESR report when looking at evidence or after
  meeting?
• Ensuring evidence is referenced not feelings

10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Competence Area MSF PSQ COT CbD CEX CSR
Communication and consultation skills x x x   x x
Practising holistically   x x x   x
Data gathering and interpretation x   x x x x
Making a diagnosis/decisions x   x x x x
Clinical management x   x x x x
Managing medical complexity       x x x
Primary care admin and IMT       x    x
Working with colleagues and in teams x     x   x
Community orientation       x   x
Maintaining performance, learning and teaching x       x x
Maintaining an ethical approach x     x   x
Fitness to practise  x     x   x
11
Deanery Guidelines ES

• How Many ES Meetings and When? 
• ST1 2 meetings in first post, 1 meeting in
  second  (i.e. 3 for that year 1 informal 2
  formal)
• ST2 1 per 6m post (ie 2 for that year both
  formal)
• ST3 1 per 6m post (ie 2 for that year both
  formal)
•  So, especially during the period Feb-Aug of
  every year (as that is when most trainees will
  move onto the next ST stage) make sure you have
  had your second ES meeting before the end of May

12
Deanery Guidelines ES

• The deanery expects all trainers to be an ES so
  we need to make sure it is achievable
• read entries, link these to competencies and
  provide feedback
• provide other feedback re number of log entries/
  whether getting behind on OOH requirements,
  formal assessments numbers of entries/ style of
  entries etc.
• assess the Naturally Occurring Evidence at the
  times of the ESRs
• There should be a high level of supervision
  throughout the final year as the year will be
  entirely in GP

13
Deanery expectations ES

• Educate trainees about the quality and
  reflectiveness of the log entries from ST1
• You need to be more vigilant at the ST2 stage
  when in hospital often to ensure do not lose
  sight of attaining GP competencies .
• There is a move to expect more virtual
  supervision of e portfolio by the ES more
  reading and responding to log diary entries, more
  use of 'educators notes' (to indicate progress ).
  we do not see doing the ESR meetings as whole
  role of the ES but that these meetings are
  punctuations on a continuous ( if intermittent)
  interaction between ES and GPSTR!
• Those who are behind/ under performing will need
  more meetings , as well as more continuous
  checking and reading and feeding back etc.

14
After ES meetings

• MUST generate an "Educational  Supervisor's
  Report" (ESR).
• This report will highlight any difficulties -
  either personal, educational or with the post..
  Educational supervision is not about disciplinary
  procedures but more about helping you overcome or
  see you through the difficulties.
• The report has an "agreed action plan" to help
  your training journey become smoother.
• The report must be uploaded onto the e-portfolio.
   

15
ESR suggested framework

• Create a review-in advance-reasons
• So that the Trainee's evidence appears in the
  correct Review period
• For an Educational Supervisor to release the PSQ
  and MSF results to the Trainee, they need to have
  the current review set. The system will not allow
  these to be released if this is not currently
  set.
• Under the new system, in order to make the
  Trainee Self Rating Review specific, a Review
  needs to exist to give the trainee this option.
• View and comment on Curriculum Coverage
• View and comment on Skills Log
• Review and comment on the PDP
• Review the Trainees Self Assessment of Competence

16
ESR Review 2

• Review each of the 12 areas of Competence
• Make an "evidence-based" judgement about the
  progress of the trainee in each of the twelve
  areas, looking at evidence from the linked forms
  and learning log entries.
• Decide on a rating
• Give evidence to support this rating
• Decide on specific actions to help with
  progression before the next review
• The trainee is required to go through the same
  process in their self assessment of competence
• You will not be able to continue to the next
  stage of the review process until all sections
  have been completed
• Finish Review
• Review the summary of all comments and actions
• Comment on the quality of the evidence provided
• Make a Recommendation
• Agree a Learning Plan
• The review details can then be edited, saved and
  edited later, or the review can be completed and
  submitted.

17
ESR

• How do we do it?
• Suggestions re framework-RAM idea
• a) click on learning log and review look at the
  NOE (Audit, SEA, reflection of post)b) click on
  evidence - review CBDs, COTs, CEXs - minimum
  numbers, competence mappingc) stay on evidence -
  review PSQ, MSF and CSR - look for themesd)
  click on the 'Educators' Notes' and see what
  themes there are theree) Now click on 'create
  review' and go through the ESR form in order
  using the evidence made available to you on each
  page and bringing in other stuff noted from steps
  a)-d).

18
Post coffee

• http//www.youtube.com/watch?v6EElqrgk4N0feature
  related

19
Log entries

• Log entries should on average show
• evidence of critical thinking analysis,
  describing own thought processes
• self awareness demonstrating openness and honesty
  about performance and some consideration of
  feelings generated
• evidence of learning, appropriately describing
  what needs to be learned, why how
• appropriate linkage to curriculum
• demonstration of behaviour that allows linkage to
  one or more competency areas

20
Gibbs Reflective Cycle
21
Reflective Writing role and functions

• To maximise the effectiveness of experiential
• learning
• To evaluate ones practice
• To promote critical thinking
• To facilitate the integration of theory with
  practice
• To generate theory
• To evaluate a learning activity
• To demonstrate that learning has taken place

22
Reflection

• How do I decide which competence area to attach?
• You may select more than one area if you believe
  the entry merits this.
• Trainees will be expected to demonstrate
  progression in all 12-competency areas (and hence
  to have a corresponding spread of validated
  entries).
• How do I decide whether the entry shows
  meaningful reflection?
• The trainee should demonstrate the impact on
  their practice of the experience they describe.
  This may relate to new knowledge and skills or
  a0n effect on insight and attitude.
• The quality of the reflection should be judged
  against the learners stage of training with
  sophisticated reflection as progress.
• How should I comment on the log diary entry?
• Your comments vital feedback and should respond
  to their reflection.
• If you have decided to validate an entry against
  a competence area, you should describe in your
  view why it has addressed the area you have
  chosen (and this may include your suggestions for
  further development)

23
Reflection template
24
Validation

• Why does validating entries matter?
• The learning log helps to balance the educational
  portfolio and provides additional evidence of
  learning and progression, capturing evidence from
  learning opportunities in the workplace.
• There is no limit to the number or quality of
  entries that trainees can make in their
  eportfolios,
• but not all of them can or should be validated.
  For example, attendance at
• VTS seminars. Entries which cannot be validated
  may still be useful for curriculum coverage.
• Once validated by you, each entry then forms part
  of the trainees evidence of progression.
• Entries are validated against the 12 areas of the
  competency framework.

25
Validation

• What does validating an entry mean?
• It means that you, in your role as trainer or
  educational supervisor, are confident that the
  entry fulfils the following two requirements
• a) It addresses one or more of the 12 competence
  areas
• b) It demonstrates meaningful reflection
• By validating a log entry you are confirming that
  this is valid evidence of learning in an
  appropriate competency area. You are not making a
  judgement about whether that competence has been
  achieved.

26
Reflection and validation exercises

• Eportfolio j smith2
• Password jsmith2
• Dr Pauline example
• Task 1- first 3 entries 13.2-7.4.08
• Appropriate no of entries?
• Critically appraise the entry and justify
• Curriculum linkage-acceptable
• Advice re further entries -reflection,linkage etc

27
Task 2

• Log entries 7.6.08-1.7.08
• How many log entries do we read? Does it cahange
  at different stages of training.
• ES comments
• Try making your own pre read comments
• What is function?/how often?
• Comment on quality presented and how may improve
• Validation
• Appropriate?
• Alternatives?
• How many validated entries per week/?

28
Task 3

• Last 2 log entries 4.7.08 9.7.08
• How to validate
• Comments?
• Plenary? Anything to ask or share?

29
Lessons from ARCP panels

• Best composition of panel-TPD(Chair)/Trainer who
  is and ES/lay member /Deanery rep
• Evidence given by somebody who does not know ST
  well
• The progress to certification view on the
  portfolio is effectively the Educational
  Supervisors report. The reviewers job prior to
  the panel is to check that the ESs judgement is
  backed by evidence in the portfolio and is
  appropriate.
• A Note on OOP-Those trainees Out of Programme
  (eg maternity, sick or abroad), still need an ES
  review. If there is substantial evidence in that
  review period, it needs commenting on. If not,
  mark the review as refer to panel

30
Lessons from ARCP panels 2

• For CCT-four critical items attainment of
  Competences, AKT, CSA, and the must-dos of OOH
  and NOE . If on track ,evidence appropriately
  backed up-minimal discussion quick look at
  learning log.
• If less straightfoirward illustrative not
  exhaustive, evidence based not anecdotal
• Curriculum coverage /PDP more for ongoing
  supervision than satisfactory progression.

31
ARCP PANEL OUTCOMES

• SATISFACTORY FOR PROGRESSION OR CCT
• OUT OF PROGRAMME
• MAINLY SATISFACTORY-ARCP NOT COMPLETEDFEEDBACK
  TO TRAINEE TO OBTAIN MISSING EVIDENCE-PANEL CHAIR
  REVIEWS EVIDENCE 2W LATER.
• POSSIBLE OR LIKELY UNSATISFACTORY-ARCP NOT
  COMPLETED WRITTEN EVIDENCE TO DEANERY RE REASONS
• ONLY ONE OUTCOME PER TRAINEE IN EACH ARCP CYCLE

32
(No Transcript)
33
WHY do STs get referrred to central Deanery panel

• Majority incomplete evidence eg
• out of hours sessions,
• patient satisfaction questionnaires and
• other workplace based assessment tools.
• NOE(Naturally Occurring Evidence)
• Confusion modular posts /LTFTT
• Clustering minimal evidence.

34
OOH

• There is the service commitment to out of hours
  work that is specified for each training post.
  Not attending OOH sessions is a probity issue.
• In an Innovative Training Post (ITP) most ITPs
  will have the same monthly (6 hour) session of
  OOH work as normal GP training posts. Some will
  have on call commitments to the modular component
  of their post eg on labour ward or hospice.
  make clear in the portfolio. If no OOH sessions
  logged panels will find the portfolio
  unsatisfactory. (PSQ also due in modular posts)
• Documentation of learning in OOH sessions
  -linking that to chapter 7 of the GP curriculum
  Care of the Acutely Ill.
• One trainee documented 2 OOH sessions in two
  months prior to panel. A total of only 3 patients
  had been seen in these two sessions. This is
  considered to be unsatisfactory.
• Clustering/demand

35
OOH cont

• Advise STs to document for each OOH
• The type of session telephone triage, visiting
  doctor, base doctor
• The number of patients seen.
• A selection of the most interesting patients
• The significant learning points and,
• Link these to the curriculum(esp care acurtely
  ill)

36
NOE-Naturally Occurring Evidence

• Deanery gives advice/ guidance as to evidence
  that occurs naturally or Records experience
  that should be submitted annually
• Despite multiple emails commonest area that Sts
  fall down on.
• Is it legally enforceable as a local Yorkshire
  criteria?

37
Naturally Occurring Evidence

• 1) Significant Event Analysis 3 per 6 month
  post file under Significant Event Analysis
• 2) Reflection on key learning points from each
  post file in Reading expected length 1 side
  A4
• 3) Audit or QoF review or NPMS Project x1 in 3
  year training file in Audit/ Project

38
NOE (cont)

• 4) Case study 2 per year file in Audit
• 5) Statement of Total Leave Taken file in
  courses/certificates
• 6)Attendance Record at VTS teaching supplied by
  VTS administrator
• 7) Complaints and adverse incident reports if
  any. File in Professional Conversations

39
Yes No N/A Satisfactory Progress/Comments
Competence Progression (if ST3 all competent or excellent)
Curriculum Coverage
Skills log DOPS
WPBA correct number for ST year?
CBD
COT/MiniCEX
MSF
PSQ
CSR
PDP
Log diary (at least minimum number of entries spread over time, evidence of reflection and development, appropriate links to curriculum)
OOH
Records of NOE (Naturally Occurring Evidence) Access full details of NOE requirements via the link opposite Access full details of NOE requirements via the link opposite Access full details of NOE requirements via the link opposite Access full details of NOE requirements via the link opposite
Significant Event Analysis (6)
Reflection on key learning points from each post
Audit (during GP attachment) (1) or Reflection on QOF (1) or Project fitting the NPMS criteria (1)
40
E-portfolio

• Post tea
• What happened next?
• http//www.youtube.com/watch?vmhRxyGlU2mQ

41
Using the e-portfolio

• In small groups
• Practice using new ES report set review own
  ES-and look at how presented and changes.
• Decide on a framework to present back for the
  most efficient way to assess evidence/ESR rv.
• Look at examples of ES descriptors-use sheets
  and own examples.
• Find alternatives
• Give examples of good and weak evidence in ES
  reports (use appendix criteria)

42
Table 1. Criteria used for assessing
EducationalSupervisors Reports.
Not acceptable Acceptable
The basis for judgements is not clear, i.e. they are not referenced to the evidence. I.e. they are not linked to evidence or there is a substantial lack of evidence to support the judgments made by the educational supervisor. Where the judgements can be evaluated, they do not appear to be justifiable. I.e. where evidence has been cited for any judgments, the accuracy or robustness of the linked evidence is questionable. No comment is made on the current state and the progression of competence. I.e. there are gaps in the content of the report or a current ESR is absent. Suggestions for trainee development are inadequate in number and/or quality. I.e. constructive suggestions for how the candidate might progress are lacking. Judgements are generally referenced to the available evidence Judgements appear to be justifiable The current state and the progression of competence are made clear Suggestions for trainee development are routinely made and appear to be appropriate
43
PLENARY

• ANY LEARNING TO SHARE
• FEEDBACK FORMS
• FUTURE LEARNING NEEDS

44
CLOSE
45
Reflective Writing role and functions

• To maximise the effectiveness of experiential
• learning
• To evaluate ones practice
• To promote critical thinking
• To facilitate the integration of theory with
  practice
• To generate theory
• To evaluate a learning activity
• To demonstrate that learning has taken place

46
Reflective writing description

• What were the significant background factors to
  this experience?
• Describe the experience
• Sequence of events
• Actions
• Observations
• What essential factors contributed to the
  experience?

47
Reflective Writing analysis

• What were the consequences of my actions?
• How do I feel about the experience?
• What factors influenced my decision and
  actions?
• What knowledge influenced my decision and
  actions?

48
Reflective Writing evaluation

• What went well what went badly?
• Could I have dealt better with the situation?
• What other choices did I have?
• What would have been the consequences of
• acting on these other choices?

49
Reflective Writing new perspectives

• What have I learnt from this experience?
• How has this experience affected my thinking?

50
Reflective Writing action plan

• How should I change my practice?
• Behaviour
• Standards, procedures
• Should I suggest changes in policy?
• What constraints may exist?
• Review changes and their effects!

51
Feedback for ES using RCGP ESR descriptors

• Not acceptable
• The basis for judgements is not clear, ie they
  are not referenced
• to the evidence
• Where the judgements can be evaluated, they do
  not appear to be justifiable
• No comment is made on the current state and the
  progression of
• competence
• Suggestions for trainee development are
  inadequate in number
• and/or quality
• Acceptable
• Judgements are generally referenced to the
  available evidence
• Judgements appear to be justifiable
• The current state and the progression of
  competence are made clear
• Suggestions for trainee development are routinely
  made and appear to be appropriate
• Highlights
• Suggestions for Improvement

52
Competences

• How do we demonstrate
• Are there any other ways
• What are the difficult competences?
• Why?
• What can we do to help?

53
COMPETENCES THAT WE STRUGGLE WITH

• FITNESS TO PRACTICE
• http//www.gmc-uk.org/guidance/case_studies.asp
• GMC interactive videos
• Community Orientation
• Managing Complexity

54
CBD

• How do we make it more relevant
• Using as a tool to get the difficult domains eg
• medical complexity
• Community orientation
• Fitness to practice
• Use examples

55
Potential Agenda Items

• ePortfolio update
• Preparing for ES and ARCP for STs and ES
• Reflection and validation of log entries
• Why do Sts struggle at ARCP
• Naturally occurring evidence
• How to enable STs to achieve difficult
  competences
• How to improve our entries as ES
• Curriculum v competences
• Competences/CBD
• ID problem STs-triangulation with portfolio