THE INTO EUROPE SERIES IN IN-SERVICE TEACHER TRAINING

1
THE INTO EUROPE SERIESIN IN-SERVICE TEACHER
TRAINING
Presentation by Györgyi Együd
Into Europe - European Standards in Language
Assessment Conference Budapest, February 9-10,
2006
2

• WASHBACK
• What is the washback effect? What studies have
  been conducted?
• Are there preparatory materials?
• How are teachers trained (encouraged) to prepare
  their students for the exam?
• (questions by Prof. Alderson)

3
COURSES AVAILABLE

• Modern English Examinations Teacher Training (60
  hours)
• Module 1 Introduction to skill-based
  language testing assessment
• Module 2 Assessing Reading
• Module 3 Assessing Listening
• Module 4 Assessing Use of English
• Module 5 Assessing Writing
• Module 6 Interlocutor Training
• Module 7 Assessing Speaking

4

• Assessing Receptive Skills (30 hours)
• Interlocutor Training
• Assessing Speaking Skills
• A2/B1 (30 hours)
• Interlocutor Training
• Assessing Speaking Skills
  B2 (30 hours)

5

•   Assessing Writing Skills
• A2/B1 (30 hours)
•   Assessing Writing Skills
• B2 (30 hours)

6

• The materials used on the courses
• (tasks, rating scales, interlocutor's
• frame, etc.) have all been piloted in real
  Hungarian schools with real Hungarian students.

7

• All of the training courses have gone through
  the accreditation process of the Hungarian
• in-service teacher training framework.
• They run live.

8

• Basic training principles as presented in
  In-service teacher development some basic
  principles in ELTJ 49/3 by Hayes, D (1995)
• All teacher development activities should be
  classroom-centred.
• (Hayes, 1995 256)

9
Basic training principles

• Teachers should be involved in the preparation
  of courses.
• (Hayes, 1995 257)

10
Basic training principles

• Trainers should themselves be teachers.
• (Hayes, 1995 257)

11
Basic training principles

• Training methodology should be largely task-based
  and inductive.
• (Hayes, 1995 257)

12
Basic training principles

• Training / development sessions should value
  participants existing knowledge.
• (Hayes, 1995 258)

13
Basic training principles

• Teacher development sessions should offer
  opportunities for participants to share knowledge
  and ideas.
• (Hayes, 1995 260)

14

• The general training approach of the courses is
  based on
• the cyclical process
• of experiential learning
• (Kolb 1984).

15

• Stage 1 Concrete experience
• Stage 2 Reflective observation
• Stage 3 Abstract conceptualisation
• Stage 4 Active experimentation

16
Course design

• Each course cycle contains the following
  recurring elements
• a demonstration of suitable alternative
  assessment approaches techniques

17

• familiarising participants with all the possible
  task- and text types of the given skill
• presenting piloted, calibrated, benchmarked
  examples from the Into Europe series with the
  results from the pilots

18

• opportunities to actually do the sample tasks
  presented
• opportunities to reflect on the teaching
  implications

19

• reference to the teachers own classroom
  experiences, examining the possible problem
  points, sharing experiences, offering guidance,
  solutions

20
Participant feedback from an awareness raising
course

• Teaching and testing are becoming friends,
  walking in the classroom hand in hand, like
  learners and teachers, with washback sponge.
• I can't wait to use today's ideas at home, with
  my students.

21

• Now I'll be able to judge critically the
  listening exercises in the course book that I
  use.
• Thank you for making me aware of even those
  "small things" (the tricky ones) in designing
  tests. I will definitely be more careful when
  writing test papers for my students.

22
The desired impact of the training programmes

• teachers will use better teaching
• and testing techniques
• positive washback
• teachers will be better-informed about European
  good practice in language testing assessment

23

• teachers awareness will develop in terms of what
  skills can be tested and how they can be tested,
  and also how students can perform at these tests
• as a result of the above, language learners
  language skills and test performance will improve

24
COURSE PARTICIPANTS ARE FAMILIARISED WITH GOOD
PRACTICE THROUGH

• standardised examination procedures demonstrated
  by a Model Examination developed for training
  purposes
• examiner training focusing on both objectively
  subjectively assessed skills
• the use of benchmarked performances in the
  training courses for speaking and writing skills

25
A model of benchmarking procedures was designed
and piloted by the project team under the
guidance of Prof. Charles Alderson (the advisor
of the Project).

• In the Hungarian context the model consisted of
• five main phases
• selecting sample performances from the pilot
  examinations
• selecting judges
• home marking by judges
• live benchmarking exercise
• editing and standardising justifications

26
The main purpose of the benchmarking exercise to
reach agreement on grades using the analytic
rating scales developed by the Project
Team. Relating the performances to the Common
European Framework could be a supplementary
exercise. ? For this purpose the 9-point scales
(Overall Spoken / Written Interaction) of the CEF
were used.
27
The Use of Benchmarked Performances in the
Training of Assessors
The benchmarks and justifications produced by the
judges in the benchmarking sessions are used for
supporting the pre-course tasks and the
face-to-face assessor training course.
? Benchmarked performance samples illustrate
candidate performance at different levels of the
scales.
28
In the face-to-face training phase,the
benchmarks and justifications are revealed to
course participantsin different ways at
different stages of the training.
29

• According to the CEF (Guideline 4),
• future examiners should undertake appropriate
  training.

• The training procedures developed by the Project
• have the following aims
• to provide future examiners with sufficient
  information about the model language examination
  they are going to be trained for (outline, task
  types, mode)
• to familiarise future oral examiners with
  standard interlocutor behaviour
• to familiarise participants with the main
  principles and procedures of assessing speaking
  and writing performances

30

• Further aims
• to introduce the idea and practice of using
  analytic rating scales for assessing speaking and
  writing performances
• to enable participants to develop the necessary
  interlocuting and assessing skills
• to ensure valid and reliable assessment of live
  performances through standardisation
• to equip trainees with transferable skills (there
  is a special need for this in Hungary)

31
The Outline of the Training Model

• Stage 1 pre-course distance learning
• self-study of an Introductory Training Pack
• accomplishing the pre-course tasks (analysing and
  marking writing scripts or sample video
  performances)

32

• The Introductory Training Pack
• contains
• An overview of the Speaking / Writing Model
  Examination
• Guidelines for interlocutor behaviour
• Guidelines for assessor behaviour
• Pre-course tasks
• Self-assessment questions
• Appendices Benchmarks Justifications for the
  Sample Tests, Examples of Candidate Language, CEF
  Scales, Glossary of Useful Terms

33
The Outline of the Training Model

• Stage 2
• Live training courses
• (a series of workshop sessions)

34
Example 1 Speaking Interlocutor Training

• discussing the experiences of the distance phase
• analysing video samples of both standard and
  non-standard interlocutor behaviour
• standardisation of the administration procedure
  through simulated examination situations (role
  plays)
• demonstrating alternative approaches (paired vs
  individual tests)

35
Role-play Cards for Part 1 (The Interview)
Candidate You are a shy, not very talkative
candidate who tends to wait for guiding
questions. You often reply with one or two short
sentences only.
Interlocutor You are the interlocutor who asks
the questions of the first part of the speaking
test. You have to elicit as much speech from the
candidate as possible. Please remember to ask the
questions listed in the Interlocutor Frame.
36
Example 2 Speaking Assessor Training

• discussing the experiences of the distance phase
• introduction to assessing oral performances
  modes and techniques of assessment
• familiarisation with the analytic rating scale
• standardisation of the assessment procedure
• comparing performances at different levels

37
The Outline of the Training Model

• Stage 3 a distance phase
• Practical application of the acquired skills
  mock tests conducted in the participants own
  environment.

38
Conclusions

• It is not impossible to become a trained examiner
  without formal training.
• Training should involve distance and face-to-face
  elements as well to ensure that future examiners
  go through each and every phase of the difficult
  and complex standardisation process.
• One training course is not enough.
• Only further practice and monitoring examiner
  behaviour can ensure that candidates language
  abilitiy is assessed in a standard manner, and
  the assessments are valid and reliable.

39

• WASHBACK
• Are there preparatory materials?
• How are teachers trained (encouraged) to prepare
  their students for the exam?
• (questions by Prof. Alderson)

40

• Teaching and testing are becoming friends,
  walking in the classroom hand in hand, like
  learners and teachers, with washback sponge.