Implementation of FETAC Assessment Policy

1
Implementation of FETAC Assessment Policy

• Provider Briefing
• April 2008

2
Agenda

• Introduction/purpose
• Update on key functions
• Implementation of Assessment policy
• WCES (Web Candidate Entry System)
• FETAC Business System (FBS)
• Summary
• Q A

3
Key Functions Update

• Provider Quality Assurance
• Awards and Standards
• Levels 1 and 2
• Programme Validation
• Monitoring
• Assessment

4

• Quality Assurance
• Current registered providers 640
• Range of centres
• VECs, FÁS, Fáilte Ireland, Teagasc,
• BIM, HEIs Centres 1,362
• Other
• second level schools 65
• Hospitals 23
• Private Providers 136
• Special Schools 43
• Training (Disability) 127
• Community/Voluntary 173

5

• Awards and Standards
• Key task Migrate existing suite of 430 major and
  associate minor awards into common awards system
• Common format accessible to all providers
• Activities
• Convert (single award)
• Harmonise (more than one named award)
• Review (existing standards)
• Partnership with former awarding bodies and IVEA
• Timeframe Over next three years

6

• Levels 1 and 2
• New standards available
• Programmes designed by registered providers and
  being validated
• New assessment arrangements implemented - internal
  ly verified
• - externally authenticated
• Certificates to issue to first cohort of learners
• Formal presentation ceremony

7

• Validation
• New Function
• New (common) awards standards published
• Programme designed to enable the learners to
  reach the standard
• Developed ? Evaluated ? Approved ?
• Validated ? Delivered
• Consultation with key stakeholders
• Guidelines available
• Operational at Levels 1 and 2

8

• Monitoring
• Monitoring pilot complete 12 centres
• Documentation and tools being revised
• Monitors recruitment process complete
• Monitors commencing May 2008
• Desk review ? Monitoring visits ? Reporting

9

• Review of FETAC 2007
• Self-evaluation involving stakeholders
• External Panel (Irish/European)
• Strategic planning towards 2011
• Panels report positive recommendations

10

• Quality Assuring Assessment
• Qualifications (Education) and Training Act 1999
• National and international best practice
• Policy agreed March 2006
• Guidelines published May 2007
• Implementation 1st July 2008
• Dissemination of information Range of
  Briefings

11
Context

• Meeting sectors needs
• Diversity of provision
• Separate and distinct roles of provider and FETAC
• Internal and external quality dimensions
• Quality - Assessment Framework

12
Assessment Framework

• Publication of nationally developed awards,
  including valid and reliable assessment
  techniques
• Publication of assessment guidelines
• Providers quality assured assessment process
• Validation by FETAC of assessment arrangements
  for the programme
• Providers own internal monitoring and
  self-evaluation
• National monitoring

13
Key Documents

• Quality Assuring Assessment Policy
• Quality Assuring Assessment Guidelines for
  Providers
• Available from May 2008
• Guidelines for Internal Verifiers
• Guidelines for External Authenticators
• Down load on www.fetac.ie/assessment

14
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication
• Internal Verification
• External Authentication
• Results Approval
• Processing Appeals
• Requesting Certification

15
Providers Assessment Process

• agreed procedures through quality assurance
• may need to be reviewed to ensure consistency
  with assessment guidelines
• procedures required for new processes
• Authentication
• Results Approval
• Learner Appeals
• Requesting certification
• evidence available for Monitoring

16
1. Assessment Process

• Process of judging learner achievement against
  the standards of knowledge, skill or
  competence for the purpose of attaining an award
  
• (Summative Assessment)
• Fair and consistent
• Learners informed of what is expected of them
• Guidelines on establishing procedures for fair
  and consistent assessment Section 3.2.
  Assessment Guidelines for Providers

17
Assessment Process

• Key parties
• - The Provider responsible for establishing
  procedures to ensure fairness and consistency
• - The Assessor responsible for judging
  learning achievement and making the assessment
  decision

18
Provider

• implementing best practice assessment policies
  and procedures (agreed through QA)
• ensure consistency with FETAC requirements
• establish
• An Authentication Process
• A Results Approval Process
• An Appeals Process
• ensure auditable trail available for FETAC
  Monitoring

19
Assessor

• implement providers assessment procedures
• plan learner assessment
• devise assessment instruments and marking schemes
• evaluate learner evidence
• make the assessment decision/judgement
• provide information and feedback to learners

20
Assessment Procedures

• Guidelines on best practice Section 3.2
  Assessment Guidelines for Providers
• Range of procedures for example
• Assessment planning
• Information for learners
• Reasonable accommodation
• Devising assessment
• Assessment techniques
• Reflects best practice

21
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication Process
• Internal Verification
• External Authentication
• Results Approval
• Processing Appeals
• Requesting Certification

22
2. Authentication Process

• Internal Verification
• External Authentication including process for
  assignment of an External Authenticator

23
Internal Verification

• Process involves two key elements
• verifying that the providers assessment
  procedures have been applied across the range of
  assessment activities
• confirming assessment results by checking learner
  evidence exists and marks and grades are recorded
  accurately

24
Internal Verification

• may take place on a sampling basis
• outcome of the process is an Internal
  Verification Report
• Report available to
• External Authenticator
• Results Approval Panel and
• FETAC Monitor

25
Internal Verification Process

• process appropriate to provider context and size
• large centre may have a number of internal
• verifiers for different award areas covering a
  number of learner groups
• small centre may have single tutor/internal
  verifier for one learner group

26
Devising an Internal Verification Process

• Consider
• number of assessors, number of learner groups,
  
• range and diversity of programmes being offered
  and awards for which learners are entered
• Devise procedures
• Assign responsibility
• Review and monitor

27
Assign Responsibility

• Procedures must indicate who is responsible
• Clear responsibility for process assigned to
  appropriate staff/tutors
• Identify who carries out the activities/role

28
Role/Activities Internal Verifier

• Check adherence to the providers quality assured
  assessment procedures (sample)
• Check results are recorded for all learners being
  entered for the award
• Confirm the accuracy of results on a sample of
  learner evidence
• Identify and address any irregularities,
  following internal procedures
• Produce an internal verification report

29
I. Check adherence to the providers
quality assured assessment procedures (sample
basis)

• What needs to be done?
• On a Sample of learner evidence check that
• assessment information has been provided to
  learners i.e. submission dates are available,
  guidelines have been issued
• appropriate documentation has been issued
• appropriate documentation/systems have been used
  to record learner evidence
• assessment techniques meet requirements of the
  award specification

30
II. Check results are recorded for all
learners being entered for the award

• What needs to be done?
• For all learners being entered, check
• results are recorded
• evidence is available
• For example
• 85 learners are being entered for the award and
  seeking
• Certification from FETAC, check provisional
  results
• are listed/reported and evidence is available for
  85
• Learners

31
III. Confirm the accuracy of assessment results
on a sample basis

• On a sample of learner evidence
• Check marks are totally and percentage marks
  calculated correctly
• Marks and grades are recorded and transferred
  correct
• Percentage marks and grades are consistent with
  FETAC grade

32
IV. Identify and address any irregularities,
following internal procedures

• Take corrective action as per internal procedures
• May involve for example
• correcting simple errors and notifying assessor
• checking with assessor in missing evidence
• bringing issue to attention of director/principal

33
V. Produce an Internal Verification Report

• Forms evidence that the process has taken place
• Auditable trail
• Report must be signed and made available on
  request to
• External Authenticator
• Results Approval Panel
• FETAC Monitor

34
External Authentication

• Provides independent confirmation of fair and
  consistent assessment in accordance with national
  standards (Sample basis)
• Undertaken through the assignment by the provider
  of an external authenticator

35
Who is an External Authenticator?

• Individual with technical/subject matter
  expertise
• Moderate assessment results in accordance with
  the award standards
• Sample learner evidence
• Selected and assigned by the provider
• Independent of the centre to which they are
  assigned

36
Devising an External Authentication Process

• appropriate to size and context
• ascertain number of major/special
  purpose/supplemental awards being offered within
  provision
• ascertain number of learner groups for each award
  
• consider basis for sampling i.e. across learner
  groups/centres
• select and assign one or more external
  authenticators for each major/special
  purpose/supplemental awards (FETAC Panel)
• ensure learner evidence will be available to
  sample

37
Sampling

• Mechanism by which learner evidence is selected
  i.e. sampled for purposes of Internal
  Verification and External Authentication
  processes
• Basis on which sample is selected is determined
  by the provider Sampling Strategy
• Appropriate to size and context
• Identify learner population from which sample
  will be selected
• Sample must be sufficient to enable sound
  judgments to be made

38
Sampling

• Sample selected may be based on
• A sample of evidence from a specific learner
  group
• A sample of evidence from a number of learner
  groups
• A sample of evidence from a number of learner
  groups across a number of programmes
• Sample of evidence from a number of learner
  groups from a number of centres

39
Sampling Strategy

• Determine the total learner population from which
  the sample will be taken i.e. from 4 specific
  learner groups
• Determine how the sample is to be selected
• Percentage of learners numbers (population) i.e.
  20/25
• Formula vn1 where n is total population of
  learners
• Other scientific formula/methodology
• Minimum sample size guideline 12
• Consider guidelines and criteria section 4.4
  Assessment Guidelines for Providers

40
External Authenticator - Role

• Moderate assessment results
• Confirm fair and consistent assessment of
  learners consistent with providers procedures
  and FETAC requirements
• Sample learner evidence
• Produce an external authentication report and
  sign-off on results
• Participate in provider results approval process
  as required

41
Key activities

• External Authenticator Panel to be established by
  FETAC
• Ongoing discussions with key provider
  stakeholders and DES
• Guidelines on remuneration TBA
• Assessment Task Group FESS and IVEA developing
  website, supporting tools, templates

42
External Authenticator Panel

• Will be made available to registered providers
• Resource to support the process
• List of external authenticators by sub-fields of
  learning

43
External Authenticator Panel

• Established through
• Invitation to external examiners of former
  awarding bodies and NCVA
• Advertisement and selection process commencing
  end of April
• Nominations from key providers own selection
  processes
• Authenticators on the panel agree to work within
  FETAC guidelines (i.e. remuneration guidelines)
• Panel maintained for two years by FETAC

44
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication
• Internal Verification
• External Authentication
• Results Approval
• Processing Appeals
• Requesting Certification

45
3. Results Approval

• Purpose to ensure results are fully quality
• assured and signed-off by the provider
• A Results Approval Panel is established

46
Results Approval Panel

• Membership appropriate to context
• May include senior staff, heads of
  departments/sections, assessors
• Internal verification and external authenticator
  reports are considered by panel
• Results approved and formally signed off

47
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication
• Internal Verification
• External Authentication
• Results Approval
• Processing Appeals
• Requesting Certification

48
4. Appeals Process

• Enables learner to appeal
• the assessment process, if they perceive any
  irregularities/inequality in implementation
• the assessment results
• Only approved results can be formally appealed
• Results approved through results approval process

49
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication
• Internal Verification
• External Authentication
• Results Approval
• Appeals Process
• Requesting Certification

50
Appeals Process

• Provider should
• approve all results through results approval
  process
• Inform learners of the outcome of the process
  i.e. make results available to learners
• allow sufficient time to enable learners to lodge
  an appeal 14 days minimum
• Submit results to FETAC, flag those under appeal

51
National Appeals process

• National appeals policy and process to be
  published Summer 2008
• Learners appeal the outcome of the providers
  assessment and appeals process only not the
  assessment result
• Application through provider

52
5. Requesting Certification

• Final approved results submitted to FETAC
• Candidate web entry system (current)
• Learner information entered i.e. name, PPSN etc
• Provider undertaking that results are approved
• FETAC will issue certificates to all learners
• Following notification of the outcome of
• the providers appeals process, a new
  certificate will be reissued if required

53
Providers Assessment Process

• 5 Key Stages
• Assessment process
• Authentication
• Internal Verification
• External Authentication
• Results Approval
• Processing Appeals
• Requesting Certification

54
WCES ReportsWeb Candidate Entry System
55
WCES Reports

• Web Candidate Entry System range of reports
  available which may facilitate the processes
• For example
• Learner group lists
• Results by learner goup
• Results by learner

56
Names, PPSN, dob, major, minors
57
Results by group
D22222 Outdoor Activities
58
Results by learner
59
Sign off

• our agreed assessment procedures have been
  adhered to
• all elements of the authentication process have
  been adhered to for these results
• these results have been approved and signed off
  by the Results Approval Panel and are being made
  available to learners

60
Sign off
61
FBS - FETAC Business System

• Pilot - Early 2009
• New public website
• More reports available to providers/centres
• Uploading data from other systems

62
Summary

• Roles and context
• 5 Stages
• Best practice
• 1st July 2008
• Use of WCES Recording/Reports/Submit results
• Auditable trail for Monitoring

63
Monitoring

• Auditable trail of evidence available for Monitor
• Assessment Results
• Reports
• Internal Verification
• External Authentication
• Results Approval
• Web Entry System Statistics/reports

64
Next Steps

• External Authenticator Panel available by June
  2008
• Guidelines documents available May 2008
• Internal Verifier guidelines
• External Authenticator guidelines
• Ongoing briefings
• External Authenticator briefings
• New opportunities for Certification to be offered
  4 times per year