Assessing Vocabulary and Writing

1
AssessingVocabulary and Writing
2
Assessing Vocabulary
3
Dimensions of Word Knowledge

• Accessibility
• Morphophonology
• Syntax
• Semantics
• Collocation
• Association
• Conceptualization
• Frequency
• Formality
• Register

4
Each dimension represents a continuum
Complete knowledge
No knowledge
5
Commonly used vocabulary tests

• Which dimensions of word knowledge do they
  usually assess?
• Do they take into consideration the scalar
  (continuum) qualities of word knowledge?

6
Dimensions of Word Knowledge

• Accessibility
• Morphophonology
• Syntax
• Semantics
• Collocation
• Association
• Conceptualization
• Frequency
• Formality
• Register

7
Vocabulary Size versusVocabulary Strength

• Size the number of words a learner knows
• Strength (or Depth) how well the learner knows
  certain words

8
Popular tests of vocabulary size

• Nations Vocabulary Levels Test
• The Eurocentres Vocabulary Size Test
• TOEFL
• Peabody

9
Popular tests of vocabularysize and strength

• The Vocabulary Knowledge Scale
• The Depth of Vocabulary Knowledge Test
• Computer Adaptive Test of Size and Strength

10
Comprehensive Assessment of Vocabulary
Knowledge(Based on samples of learners speech
and writing)

• Holistic measures
• Primary trait measures
• Rubrics
• Analytical measures

11
Comprehensive Analytical Assessment of Vocabulary
Knowledge and Use

• Tokens
• Types
• Lexical Density
• Lexical Sophistication
• Lexical Frequency Profile
• Lexical Diversity
• Lexical Error Indices
• Lexical Choice Analysis
• Lexical Style/Lexical Signatures

12
Assessing Writing
13
Popular approaches to assessing writing abilities

• Holistic assessment
• Primary trait assessment
• Rubric assessment
• Analytical assessment
• Portfolio assessment
• Showcase portfolios
• Collections portfolios
• Assessment portfolios

14
Which approaches have these advantages?

• ADVANTAGES
• A. Indicates to what extent students are
  benefiting from instruction.
• B. Even with minimal rater training can produce
  high reliability.
• C. Gives a very direct measure of writing
  effectiveness, using a simple protocol.
• D. Provides the most detailed feedback to
  writers.
• E. Is the most learner-centered.
• F. Is best for diagnosing grammatical and
  mechanical needs.
• G. Is best for diagnosing discourse-level,
  content, and organizational needs.
• H. Is the most practical for proficiency-rating
  and placement-level purposes.
• I. Most directly links assessment with
  instruction.
• J. Encourages students to reflect on their work,
  analyze progress, and set improvement goals.
• K. Encourages students to produce writing that is
  worth sharing and retaining for review.

15
Which approaches have these disadvantages?

• DISADVANTAGES
• A. Provides the writer with little or no
  feedback.
• B. Can make the teacher feel that she is giving
  up control of the assessment process.
• C. Provides feedback that may steer the writers
  attention to areas of writing that are least
  important to the overall effectiveness of her
  text.
• D. Relies on teacher-selected criteria that may
  be either over-representative or
  under-representative of the components of good
  writing.
• E. Requires multiple trained raters in order to
  achieve adequate reliability.
• F. Can be extremely time-consuming.
• G. Requires raters to evaluate writing in a way
  that real-world readers do not do.
• H. Relies on several rating criteria that should
  presumably be weighted differently, but weighting
  criteria is difficult and subjective.
• I. Is least likely to lead directly to
  improvements in writing.
• J. Provides the least useful assessment
  information to teachers.
• K. Provides the least useful assessment
  information to writers.
• L. Is the most subjective.
• M. Provides the most confusing assessment
  results.
• N. Is the least pedagogically sound.
• O. Is the least criterion-referenced.

16
Principles of Testing Writing

• Set writing tasks that are properly
  representative of the tasks that learners should
  be able to perform.
• The tasks should truly represent learners
  writing ability.
• Use as many tasks as you can.
• Test only writing ability.
• Restrict options.

17
Principles of Testing Writing (cont.)

• The tasks must be scored reliably.
• Elicit sufficiently long samples.
• Establish a clear set of criteria for scoring
• Whether holistic, primary-trait, rubric, or
  analytic
• Use 2-4 raters.
• Train the raters.
• Monitor raters performance.

18
Some new directions in assessing writing

• Self evaluation
• Peer evaluation
• Picture and film stimuli
• Writing for authentic purposes
• Process evaluation and progress profiles

19
Some realistic day-to-day writing tasks

• Writing checks
• Paying bills
• Taking notes and phone messages
• Writing reminder notes to oneself
• Writing a shopping list
• Writing a schedule/itinerary for oneself
• Writing letters/e-mail messages to friends and
  relatives
• Writing letters to people/companies you do
  business with
• -inquiries
• -complaints
• -appreciation
• Filling out forms
• -credit card applications and the like
• -medical background forms for healthcare
  providers
• -personal background forms for potential
  employers
• Writing a classified ad for a newspaper
• Writing a letter to the editor
• Writing a curriculum vitae or resume
• Writing a cover letter for a job application

20
How do you assess the writing abilities of
low-level learners?

• realistic tasks (tasks they are truly likely to
  encounter)
• structured/discrete-point test items