Electronic Portfolio Approaches to Support Development of

1

• Electronic Portfolio Approaches to Support
  Development of
• Contract Research Staff

Simon Cotterill, Lesley Heseltine, Paul
Drummond, Tony McDonald

• Faculty of Medical Sciences
• University of Newcastle upon Tyne
• School of Medical Education Development
• Career Development Coordinator,
• Faculty Research Strategy Group

Newcastle University
2
Overview

1. Electronic portfolios
2. Contract Research Staff (CRS)
3. Electronic Portfolio Approaches for CRS
4. Feasibility Study, Evaluation Future Plans
5. Questions / Discussion !

3
What are Portfolios ?

• Unstructured
• Formative
• Sample
• Best work
• Factual / Quantitative
• Learner Owned

Structured Summative All work Representative R
eflective / Interpretive Employer Owned
4
What is an Electronic Portfolio ?

1. A computer application which stores the users
   information on a local drive.
2. A static Web page eg. CV on the Internet
3. A dynamic database-driven system that is accessed
   over a network.

5
10 value added features of an online approach
to portfolios

• Highly customisable
• Multi-purpose eg. formative summative
  reduced duplication
• Multiple structures / views
• Easier cross-referencing
• Sharable
• Secure access from a range of locations
• Transportable
• Searchable
• Reduced admin
• Not left on the bus !

6
Managed Environments for Portfolio-based
Reflective Learning Integrated Support for
Evidencing Outcomes. 

• An FDTL-4 project
• Newcastle University (lead site)
• Leeds University
• Sheffield University
• Dundee University (for consultancy)
• Developing Web based portfolios to support
  reflective approaches for evidencing the
  attainment of programme outcomes in undergraduate
  Medicine.
• Closely integrated with on-line curricula and
  study guides to become an integral component of
  managed learning environments for Medicine.

7

• Aims of the FDTL-4 project
• To develop an on-line portfolio system to support
  a reflective approach to evidencing the
  attainment of programme outcomes.
• To promote the development of the reflective
  capabilities of medical students, giving greater
  responsibility for managing their own learning
  and preparing for aspects of work-based and
  lifelong learning.
• To develop strategies to facilitate assessment of
  curriculum outcomes that are not amenable to
  traditional instruments of assessment.

8
Managed Environments for Portfolio-based
Reflective Learning - Integrated Support for
Evidencing Outcomes. 
FDTL-4 Newcastle development work

• Generic ePortfolio
• Stand-alone
• Portfolio framework
• Customisation
• Component selection
• Outcomes / skills sets
• Terminology

• MBBS ePortfolio
• Integrated into the VLE
• Piloting in 2003/4
• Years 1-2 diary / log-book
• Year 3 rotation in Child Health
• Year 4 placements (SSCs)

9
Current Research and Evaluation Studies

• Evaluation of an electronic portfolio to
  facilitate reflective learning in stage 4 medical
  students during their student selected
  components.
• Cotterill SJ, McDonald AM, Bradley P, Robinson R,
  Hammond GR

• -Questionnaires (before and after using the
  ePortfolio)
• change in awareness of intended learning outcomes
  ?
• factors influencing use of the ePortfolio
• - Continuing Learning Inventory (Oddi, 1986)
• - prior reflective practice
• - attitudes to computers in education
• - demographic data
• attitudes and perceptions of using the electronic
  portfolio
• perception of impact of using the electronic
  portfolio
• evaluation of technical features and usability
• -Focus groups
• - Supervisors Questionnaire (random sample)

10
Current Research and Evaluation Studies (cont.)

• 2. Evaluation of a Phase I Portfolio/Log Book
  (paper and electronic versions).
• Cotterill SJ, Sarma S, McDonald AM, Bradley P

• -Focus groups
• -Questionnaire

3. Feasibility Study of an Electronic Portfolio
to Support Faculty Contract Research Staff
(CPD) Cotterill SJ, Heseltine L, Drummond P,
McDonald AM
-Piloting -Structured telephone interviews
11
Using the ePortfolio in Other Contexts

• Dentistry
• A national consortium of undergraduate dental
  schools and postgraduate VDTs
• Postgraduate Medicine
• Building on the MyPIMD project
• Contract Research Staff
• Feasibility study
• Biosciences at Newcastle
• A mini-project funded by LTSN-BS

12
2. Contract Research Staff (CRS)
13
Key Problems for CRS

• Insecurity in employment and unpredictability
  of future funding
• Lack of obvious, visible career paths
• Lack of career development opportunities
• Feelings of being a second class citizen with
  little recognition or status
• Lack of consistency in terms and conditions of
  service

14
Supporting CRS within the Faculty of Medical
Sciences at Newcastle

• A scheme was developed in early 2003 designed to
  enhance career progression and job security of
  research staff on fixed term contracts.
• The scheme adheres to the principals of the
  Concordat and has three strands
• Active Career Management
• Continuity of Employment
• Knowledge Exchange

15
Supporting CRS within the Faculty of Medical
Sciences at Newcastle 2

• First steps
• Knowing who our CRS are
• Views of our CRS
• Contract Research On-line Survey (CROS)
• Contract Research Staff Working Party
• Effective communication with CRS

16
Supporting CRS within the Faculty of Medical
Sciences at Newcastle 3

• Project Launched May 2003
• Active Career Management
• Induction of new CRS
• Performance and Development Review
• Encouragement of Independence
• Continuity of Employment
• Bridging Funding
• Support for Fellowship Applications
• Knowledge Exchange
• Programme of Workshops
• CRS Web site

electronic portfolios ?
17
3. Electronic Portfolio Approaches for CRS
18
Research Career Builder - Skills Matrix

• http//www.shef.ac.uk/gmpcrs/rcb.html
• An FDTL project including input from Nigel
  Beacham, Crispin Coombs, Abigail Davis,
  Stephen Hirst, Ursula Iliffe, Karen Kellard,
  Margaret Parkin, Claire Palmer, Jenny Park,
  Sheila Price, Joan Stenson, Patrick Webb
• This is a Skills Matrix with skills in 3
  different areas
• Core Research Skills (8)
• Personal and Professional Skills (8)
• Specialist Skills

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
4. Feasibility Study, Evaluation Future Plans

• Small scale piloting with volunteers (n11)
• Brief instructions on using the portfolio
• please also try and think about how this might
  be useful from the viewpoint of new / junior CRS.
  Please also consider the following issues were
  the tools easy to use, was navigation easy, did
  you experience any particular problems ?
• Structured telephone interviews (n7 at time of
  analysis) Email feedback

27
Usability / Navigability

• All gave positive feedback on usability /
  navigability
• The beauty of this tool is that, being
  web-based, it is easily accessible. Once
  familiar with the sections within the tool I
  found it easy to use and simple to navigate.
• However
• It appears very helpful, but a bit daunting when
  considering what can be filled in to get an up to
  date record...

28

• The only other comment I had was that it was
  sometimes a bit tricky to get back to the main
  page-and I had to look for the eportfolio link. 
  A link back to that page in the main body of the
  page may be helpful for IT phobes like myself.
• CV
• Qualifications very British, very difficult to
  fill in for non-native researchers with different
  educational histories..
• Good but adding qualifications is tedious need
  a facility to add multiple qualifications on a
  single screen

29

• Being a Post-Doc researcher myself this is a
  good idea as it would enable me to quickly
  reference courses and information which would
  otherwise be lounging in a file somewhere. I have
  no doubt that the eportfolio will be useful when
  CRS are applying for new positions, especially
  the CV option.
• When I got to the PDF view of my CV and My
  Skills and Achievements I discovered loads of
  grammatical and spelling mistakes.  Is there a
  way to have a spelling grammar check running as
  you enter the information?

30
Cross-referencing records of courses and
presentations with the Skills matrix

• 6/7 found this straight forward.
• 1 person found this difficult
• I had difficulty understanding what was
  happening with cross referencing and some
  instruction in this would be helpful. Perhaps it
  would be useful to add examples of inputs for
  each section, or better still, have a model
  eportfolio which could be looked at.

31

• I particularly liked the way it records
  conferences and courses-as I can never remember
  what I have done and when.  The way it links to
  research skills is really good and I can see
  benefits of it for my own personal and
  professional development and as a resource-e.g..
  if applying for other jobs and need to articulate
  strengths and weaknesses and provide evidence. 
  The great thing about it is you can just enter a
  short amount of information at a time and come
  back to it later.

32
Conclusions

• The volunteers were able to use the ePortfolio
  for CRS
• The volunteers generally found it straight
  forward but did provide feedback on how
  navigability of specific parts of the ePortfolio
  could be improved.
• The FDTL4 ePortfolio framework developed for
  medicine has proved applicable in another context.

33
Limitations

• Volunteers were self-selected and included more
  senior CRS (mostly from science backgrounds)
• Small numbers more large scale piloting and
  systematic evaluation is required
• Electronic portfolio approaches may also be
  applied to staff in HE for a range of purposes.

34
Future Plans

• Automatically include records of course
  attendance (where booked and confirmed via the
  CRS Website).
• Make the skills records more sophisticated eg.
  recording levels of competency.
• Link into myProfiles data for publications
  and other information (a small minority of CRS
  may use the myProfiles system to record
  publications and other PDR RAE information).

35
Future Plans 2

• Explore additional linkages with MIS
  information eg. HR data and other course
  attendance information recorded by Staff
  Development and other units in the University.
• Incorporate other eportfolio developments from
  out FDTL4 ePortfolios project (http//www.eportfol
  ios.ac.uk)
• Research and evaluation of using ePortfolios
  for CRS

36
further information http//www.eportfolios.ac.u
k http//medical.faculty.ncl.ac.uk/crs Simon
Cotterill S.J.Cotterill_at_ncl.ac.uk
37
The Personal Profile at Newcastle

• RAE focus. Records
• Teaching and Learning (UG and PG teaching and
  teaching development)
• Research (Research programmes, publications,
  awards, staff and students)
• Third Strand (spinouts, outreach work and
  industrial collaborations)
• Roles (administrative duties, training and
  support roles)
• Integrates centrally held data sets
• RIS awards data under "Research Grants held"
• Graduate School records of PG student supervision
  under "Research Students".

38
Portfolios for HE Staff

• Baume D, Yorke M. The reliability of assessment
  by portfolio on a course to develop and acredit
  teachers in higher education. Studies in higher
  education. 2002 27 7-25
• HE Staff may also benefit from electronic
  portfolio approaches including personal profiles
  and research plans which tie in to management
  information systems and networked learning
  environments.