Assessing a learners level of spoken English

1
Assessing a learners level of spoken English

• Sue Davies
• IATEFL 2009
• Cardiff

2
The challenges

• utility
• discrimination
• reliability
• practicality
• validity
• positive washback

3
Enhancing reliability

• Rating scale should be revised and re-validated.
• Tasks and language benchmarked
• Ongoing monitoring of tests and performance
• Examiner performance monitored
• Interlocutor performance monitored
• An interlocutor frame should be produced

4
Rating scale

• Tests recorded
• CEFR at the heart of the process
• What candidate can do
• How well

5
What

• Interact competently if not always accurately in
  everyday situations
• Communicate personal information, opinions,
  ideas, feelings
• Communicate in a variety of social situations
  using an appropriate range of functional language
• Exchange information, feelings and opinions to
  perform a task
• Narrate, describe, explain and express opinions
  in extended speech
• Contribute points to a simple discussion

6
How well ARPF

• Initiates and follows the norms of turn-taking,
  prompts and manages the discourse with a degree
  of independence
• Connects descriptions and narrations in simple
  ways
• Speaks without undue hesitation unless searching
  for information, vocabulary or reformulating

7
Specifications

• Tight descriptions of each task
• Functions
• Grammatical knowledge
• Topics

8
Benchmarking Relating language examinations to
the CEFR - Manual

• Does the test measure what it says it measures?
• Can it be demonstrated that the test is at the
  level of
• proficiency it claims to be?
• Does the test provide consistent measures of
  precisely
• the abilities we are interested in?
• What evidence is provided to back up any claims
  made
• about the test?

9
Ongoing monitoring

• Independent panel
• On all fronts
• CEFR

10
Construct-irrelevant variance

• Gender
• Test conditions
• Education
• Experience
• Age
• Personality

11
ISESOL

• C2 Mastery
• C1 Expert
• B2 Communicator
• B1 Achiever
• A2 Access
• A1 Preliminary

12
Four parts of the test

• Part One To give personal information
• Part Two To communicate appropriately in social
  situations
• Part Three To exchange information and opinions
• Part Four To give a presentation and answer
  questions on it.

13
Distinguishing levels

• length of the test
• linguistic difficulty and complexity
• range of topics and functions
• nature of the tasks

14
Interlocutor Framework
15
Interlocutor Framework
16
Interlocutor Framework

• Test reliability
• Natural use
• Reasons for diverting from the Framework

17
Qualities of a good interlocutor

• Make candidate feel comfortable smile, keep eye
  contact, show interest, manage inappropriate
  responses
• Elicit as much language as possible use gestures
  to help understanding, give thinking time, use
  open-ended questions
• Control the test organise yourself, the room and
  paperwork, prepare thoroughly, keep to time
  limits, deliver the interlocutor script naturally
• Be familiar with the levels know the
  descriptors, adjust to the level of the test,
  support candidates at lower levels

18
Further details

• City Guilds
• 1 Giltspur Street
• London
• EC1A 9DD
• T 44 (0)20 7294 2468
• F 44 (0)20 7294 2400
• www.cityandguilds.com/ieq