Moderation of the FSA Best Practice Linda Meyer

1
Moderation of the FSA Best PracticeLinda
Meyer
2
INTEGRATED FSA

• The FSA for the qualification will focus on the
  extent to which a learner can demonstrate applied
  competence.
• Applied competence, in terms of the NQF is
  evidenced through learners ability to integrate
  concepts, ideas and actions in authentic,
  real-life contexts and is expressed as practical,
  foundational and reflexive competence

3
PURPOSE

• The final integrated summative assessment refers
  to the process of making judgments about
  achievement. This is carried out when a learner
  is ready to be assessed at the end of a
  learnership or full qualification. The FSA will
  be set by a registered assessor appointed by the
  Certification/Outsourced Partner after
  notification from the provider/assessor that the
  candidates are ready.

4
FSA

• Practical competence - demonstrated ability to
  perform a set of tasks and actions in authentic
  contexts
• Foundational competence - demonstrated
  understanding of what we are doing and why we are
  doing it
• Reflexive competence - demonstrated ability to
  integrate our performances with our understanding
  so that we are able to adapt to changed
  circumstances and explain the reason behind these
  adaptations.

5
ASSESSMENT METHODS

• In setting the FSA, SETQAA would like
  Certification/ Outsourced Partners and/or
  assessors to take note of the different
  assessment methods and encourage the use of a
  wide variety of methods.

6
SETTING OF FINAL INTEGRATED SUMMATIVE ASSESSMENT
(FSA)

• Function and role of the Certification
  Partner/Outsourced Partners
• The Certification/Outsourced Partner is
  responsible for setting of the FSA at NQF Level
  4, with a memorandum of model answers and a grid
  showing its linkages with the assessment criteria
  of the qualification.

7
FSA

• All FSA tools and memoranda, even in draft form,
  are the property of the Services SETA ETQA and
  may not be revealed to, or discussed with any
  unauthorized person. The highly confidential
  nature of these assessment instruments must be
  safeguarded at all times.

8
FSA

• If the assessment instrument and memoranda are
  stored on computer, thorough security measures
  must be taken to prevent any illegal access to
  these files.
• The contents of the assessment instrument must
  reflect the relevant unit standards and
  qualification and must not be biased towards any
  particular textbook or learning material.

9
FSA

• A second assessment instrument should be set
  as standby assessment instrument. SETQAA have the
  right to propose the use of either assessment
  instruments for the FSA, if necessary.

10
FSA

• The Certification/Outsourced Partner must ensure
  that the assessment instruments are not
  identical.
• The Certification/Outsourced Partner must also
  ensure that neither of these assessment
  instruments are the same as any previous
  assessment instruments.

11
MODERATION CRITERIA

• The moderator must ensure that the assessment
  instrument is aligned with the purpose and
  rationale of the qualification and that it
  complies with the applicable SAQA level
  descriptor.
• Draft assessment instruments must fall within the
  ambit and in accordance with the guideline
  document.

12
Functions and role of the moderator

• After the Certification/Outsourced Partner has
  set the FSA instrument, with a memorandum of
  model answers, an internal moderator, appointed
  by the Certification/Outsourced Partner, must
  moderate all draft assessment instruments and
  memoranda for the assessment instruments under
  his/her jurisdiction.

13
Functions and role of the moderator

• During the moderation, the assessment instrument
  and memoranda are to be objectively assessed by
  the moderator with regard to the following

14
Functions and role of the moderator cont.

• The moderator must verify that papers set for the
  FSAs are not identical.
• The moderator must verify that the Unit Standards
  are appropriate to the registered qualification.

15
Functions and role of the moderator

• The moderator must ensure that the final
  summative integrated assessment instrument is
  aligned with the purpose and rationale of the
  qualification and that it complies with the
  applicable SAQA level descriptor.

16
Functions and role of the moderator

• The moderator must ensure that the final
  integrated assessment instruments must fall
  within the ambit and in accordance with this
  guideline document.
• The extent to which the prescribed Unit Standards
  are covered as well as the assurance that nothing
  is asked that is outside the scope of the Unit
  Standard/Qualification must be verified.

17
Functions and role of the moderator

• The spread and number of marks allocated
  according to the weighting and credits as per the
  registered learnership/qualification must be
  assessed.

18
Functions and role of the moderator

• The effectiveness of the type of assessment
  methods used, written questions, essays,
  paragraphs, case studies, must be assessed.

19
Functions and role of the moderator

• The questions, terminology, numbering, spelling,
  punctuation, sentence construction, units, and
  symbols must be clear, unambiguous and
  grammatically correct so that candidates will
  know exactly what is expected of them.

20
Functions and role of the moderator

• The length of the assessment instrument must take
  into account the time allocated as well as
  reading time.
• The draft assessment instrument must be of such a
  length that a well-prepared learner will be able
  to answer it comfortably within the time
  allocated, with reasonable time remaining for
  revision.

21
Functions and role of the moderator

• It should be borne in mind that it takes
  considerable time to read through certain
  assessment instruments.
• The memorandum must ensure that model answers
  give enough detail for the assessor of the FSA to
  conduct an accurate and fair assessment.

22
Criteria for moderation of the FSA instrument

• Standard of the paper (validity, reliability and
  fairness)
• Questions of various types e.g. Multiple Choice
  Questions, Case studies, paragraphs, data
  response, essay, etc
• Questions from which candidates are to choose
  are they of equal difficulty level
• Correct distribution in terms of cognitive levels
  (Blooms Taxonomy)

23
Blooms Taxonomy

• In 1956, Benjamin Bloom, a professor at the
  University of Chicago, shared his famous
  "Taxonomy of Educational Objectives". Bloom had
  identified six levels of cognitive complexity
  that have been used over the past 40 years to
  make sure that instruction stimulates and
  develops learners' higher-order thinking skills.
  The Levels are Knowledge Rote memory skills
  (facts, terms, procedures, classification
  systems) Comprehension The ability to
  translate, paraphrase, interpret or extrapolate
  material. Application The capacity to transfer
  knowledge from one setting to another. Analysis
  The ability to discover and differentiate the
  component parts of a larger whole. Synthesis
  The ability to weave component parts into a
  coherent whole. Evaluation The ability to judge
  the value or use of information using a set of
  standards. 

24
Blooms Taxonomy

• 1. Knowledge remembering of previously learned
  material recall (facts or whole theories)
  bringing to mind.
• Terms defines, describes, identifies, lists,
  matches, names.
• 2. Comprehension grasping the meaning of
  material interpreting (explaining or
  summarizing) predicting outcome and effects
  (estimating future trends).
• Terms convert, defend, distinguish, estimate,
  explain, generalize, rewrite.
• 3. Application ability to use learned material
  in a new situation apply rules, laws, methods,
  theories.
• Terms changes, computes, demonstrates, operates,
  shows, uses, solves.
• 4. Analysis breaking down into parts
  understanding organization, clarifying,
  concluding.
• Identify parts See Related Order Relationships
  Clarify.
• Terms distinguish, diagrams, outlines, relates,
  breaks down, discriminates, subdivides.
• 5. Synthesis ability to put parts together to
  form a new whole unique communication set of
  abstract relations.
• Terms combines, complies, composes, creates,
  designs, rearranges.
• 6. Evaluation ability to judge value for
  purpose base on criteria support judgment with
  reason. (No guessing).
• Terms appraises, criticizes, compares, supports,
  concludes, discriminates, contrasts, summarizes,
  explains.

25
Blooms Taxonomy

• .Knowledge (finding out)a. Use - records, films,
  videos, models, events, media, diagrams,
  books...b. observed behavior - ask match,
  discover, locate, observe, listen.
• 2. Comprehension (understanding)a. Use - trends,
  consequences, tables, cartoons....b. observed
  behavior - chart, associate, contrast, interpret,
  compare.
• 3. Application (making use of the knowledge)a.
  use - collection, diary, photographs, sculpture,
  illustration.b. observed behavior - list,
  construct, teach, paint, manipulate, report.
• 4. Analysis questions (taking apart the known)a.
  use - graph, survey, diagram, chart,
  questionnaire, report....b. observed behavior -
  classify, categorize, dissect, advertise, survey.
•  5. Synthesis (putting things together in another
  way)a. use - article, radio show, video, puppet
  show, inventions, poetry, short story...b.
  observed behavior - combine, invent, compose,
  hypothesis, create, produce, write.
•  6. Evaluation (judging outcomes)a. use -
  letters, group with discussion panel, court
  trial, survey, self-evaluation, value,
  allusions...b. observed behavior - judge,
  debate, evaluating, editorialize, recommend.

26
Criteria for moderation of the FSA instrument

• Overall how does the standard of the assessment
  instrument compare in relation to other
  qualifications/s assessment instruments and
  previous assessment instruments.

27
Criteria for moderation of the FSA instrument

• Intellectually challenging and allowing for
  creative responses from candidates
• Suitability of examples and illustrations
• Relationship between mark allocation, level of
  difficulty and time allocation

28
Criteria for moderation of the FSA instrument

• FSA do not count more than continuous assessment
  of providers the objective for the FSA is to
  check the standard across providers and across
  qualifications - STANDARDISATION

29
Technical Criteria

• Cover page with all relevant details such as
  time, unit standards and instructions to
  candidates
• Clarity of instructions to candidates
• Lay out learner friendly
• Correct numbering

30
Technical Criteria

• Mark/weighting allocation clearly indicated
• Quality of illustrations, graphs, tables etc must
  be print ready
• Complete memorandum with model answers with
  mark/weight allocation and provision for
  alternatives

31
Registered Unit Standards and Qualifications

• Relevance to registered Qualification, i.e. unit
  standards, specific outcomes, assessment criteria
• Levels of questions
• Coverage of unit standards
• Weighting and spread of contents

32
History of tool

• Full history of FSA (when it will be conducted,
  who will be assessed, when and where) with all
  drafts, internal moderators comments etc. must
  accompany the assessment instrument each time it
  is submitted to the external moderator (SETQAA)

33
Cognitive skills

• What conceptual constructs of the unit standards
  does the assessment instrument deal with?
• e.g. reasoning ability
• ability to communicate
• ability to translate from verbal to symbolic
• ability to compare and contrast
• ability to see causal relationship
• ability to express an argument clearly

34
Cognitive skills

• Are these constructs representative of the best
  and latest developments in the training of this
  knowledge field?
• Are the questions challenging and allowing for
  creative responses from candidates?
• Suggested application of cognitive levels for an
  NQF Level 4 qualification is
• 10 knowledge
• 20 comprehension
• 40 application
• 30 analysis, synthesis, evaluation

35
Language and bias

• Correct terminology
• Appropriate language register for the level of
  the learner
• Avoidance of gender, race, cultural, provincial
  bias
• Clear and unambiguous specification of
  instructions within questions e.g. list, describe

36
Time

• Relationship between difficulty level of
  questions and time
• Relationship between time and mark allocation and
  answer requirements

37
Internal Moderation

• Is there evidence that the paper has been
  internally moderated?
• Quality, standard and relevance of input from
  internal moderator.

38
Competency of Assessor

• Knowledge of assessor setting the FSA with regard
  to qualification.
• Knowledge, experience, expertise in assessment

39
Marking Memorandum

• Correctness of memorandum
• Correspondence with questions.
• Alternative answers provided
• Does it facilitate assessment?

40
Alignment Grid

• Did they use the grid showing the alignment of
  the instrument with the purpose, rational,
  assessment criteria and NQF level descriptor of
  the qualification as issued by the SSETA used?

41
Overall impression of the paper

• Fairness of the assessment instrument as whole
• Will the assessment instrument as a whole assess
  the achievement of the purpose of the
  qualification and the exit level outcomes
• Recommendations for improvement or maintenance of
  qualification/unit standards.
• final acceptance/rejection of individual
  assessment questions and whole assessment
  instrument needs to be substantiated

42
REFERENCES OR RECORDS

• ETQA-F 005 Final Integrated Summative Assessment
  Instrument

43

• Only after written approval from the external
  moderator, SETQAA will approve and sign off the
  final assessment instrument and notify the
  Certification/Outsourced Partner to proceed with
  the distribution of the assessment instrument and
  conduct the assessment.

44
CONTINUOUS ASSESSMENT VS FINAL SUMMATIVE
ASSESSMENT
SETQAA DOC
45
FSA Moderation process

• APPLICATION FOR MODERATION OF FINAL INTEGRATED
  SUMMATIVE ASSESSMENT INSTRUMENT RECEIVED BY
  SSETQA
• EXTERNAL MODERATOR APPOINTED BY SSETQA
• NO COMMUNICATION PERMITTED BEWEEN DEVELOPER AND
  MODERATOR
• MODERATION OF FINAL INTEGRATED SUMMATIVE
  ASSESSMENT INSTRUMENT / RPL TOOL

46
FSA Moderation process

• PRELIMINARY REPORT PREPARED BY MODERATOR AFTER
  THE FSA/RPL TOOL HAS BEEN MODERATED -
  REMEDIATION IDENTIFIED
• TOOL SENT BACK TO DEVELOPER FOR REMEDIATION TO BE
  ADDRESSED
• THE DEVELOPER MAY REMEDIATE / REQUEST ADDITIONAL
  INFORMATION OR APPEAL THE MODERATION DECISION
  THROUGH THE SSETA APPEALS PROCEDURE

47
FSA Moderation process

• REMEDIATION COMPLETED
• SENT TO EXTERAL MODERATOR TO CONFIRM REMEDIATION
  FINAL REPORT SUBMITTED
• ELECTRONIC VERSIONS SENT
• FSA/RPL TOOL REGISTERED ON DATABASE

48

• The Certification/Outsourced Partner assures the
  rectification is done and sends the final draft
  together with the Moderators report to SETQAA for
  external moderation.

49

• The Certification/Outsourced Partner assures the
  rectification is done and sends the final draft
  together with the Moderators report to SETQAA for
  external moderation.

50

• Only after written approval from the external
  moderator, SETQAA will approve and sign off the
  final assessment instrument and notify the
  Certification Partner to proceed with the
  distribution of the assessment instrument and
  conduct the assessment.

51

• Thank you