New Directions in Foreign Language Aptitude Testing

1
New Directions in Foreign Language Aptitude
Testing

• John Lett, John Thain, Ward Keesling, Marzenna
  Krol
• Defense Language Institute Foreign Language
  Center
• Monterey, CA 93944
• IMTA-2003, Pensacola, FL

2
Session Overview

• Background
• Recent/Current Initiatives
• Future Plans

3
Selection for Military Language Training (IET)

• ?Recruiting
• ?ASVAB
• ?DLAB
• ?BASIC TRAINING
• ?DLIFLC

4
DLAB Selection and Assignment

• Language Difficulty Category
• I
• II
• III
• IV

• Minimum DLAB
• 85
• 90
• 95
• 100

5
Description of DLAB

• Multiple choice, 90 minutes
• Range 0-164, mean 100, SD 15
• Four parts
• Bio data
• Spoken stress
• Deductive rule application
• Inductive pattern application

6
RESEARCH BASE

• Carrolls four components
• phonetic coding ability
• grammatical sensitivity
• rote learning ability
• inductive language learning ability
• Other FL aptitude tests
• MLAT (Carroll-Sapon)
• PLAB (Pimsleur)

7
DLIFLC Learning Environment

• Intensive, small classes
• Six hours daily
• 26 to 63 weeks
• Proficiency-oriented approach
• Graduation requirement
• L2, R2, S1 on ILR Scale
• http//govtilr.org

8
How well does it work?

• Very well, thank you
• Language Skill Change Project, mid-1980s
• Army Research Institute, late-1980s
• DLIFLC program data, early 1980s to
• present

9
LSCP

• 1903 US Army students
• Korean, Russian, German, Spanish
• Eleven blocks of predictor variables
• Forced order-of-entry MRA of L, R, S,
• course completion
• DLAB added variance, especially for
• Russian and Korean

10
ARI (White Park)

• gt5000 cases, ASVAB DLAB
• ST and DLAB correlated .51
• Raising ST cut would raise yield
• Of ST 104, 1.3 made DLAB 100
• Of ST 130, 46 made DLAB 100

11
FY02 Program DataSuccess rates for new
students as a function oflanguage difficulty
category and DLAB score
Difficulty Category DLAB Minimum Min 0 9 pts Min 10 - 19 Min 20 29 Min 30
Cat I 85 76 81 74 96
Cat III 95 70 73 87 96
Cat IV 100 68 72 82 87
12
(No Transcript)
13
So, why a new one??

• Protection against loss or compromise
• Changes in instruction and testing
• Demand for more, and more highly proficient,
  military language specialists
• Push to test more personnel
• Pressure to computerize
• Desire to improve

14
Recent Initiatives

• DLAB Scramble
• Computerized DLAB
• DLIFLC programming beta testing
• Liaison with MEPCOM
• Via MAPWG, Nov 02
• SY Coleman follow-on
• New Mousetrap Project

15
New Mousetrap Overview

• Objective
• Update knowledge to inform design decisions
• Method
• Identify relevant experts
• Provide relevant knowledge and charge
• Receive and exchange position papers
• Two-day workshop at DLIFLC (Oct 03)

16
Personnel

• Consultants
• William J. Strickland, Vice President, Human
  Resources Research Organization (HumRRO)
• Peter J. Robinson, Professor of Linguistics,
  Department of English, Aoyama Gakuin University,
  Tokyo, Japan
• Daniel J Reed, Language Assessment Specialist,
  Program in TESOL and Applied Linguistics, Indiana
  University
• US Government Madeline E. Ehrman, Director,
  Research, Evaluation and Development, Foreign
  Service Institute (FSI) and Subject Matter
  Expert, Center for the Advanced Study of Language
  (CASL).
• Facilitators
• Dorry Kenyon, Director, Language Testing
  Division David MacGregor, Research Assistant
  and Paula M. Winke, Test Development Coordinator,
  Center for Applied Linguistics
• Perot Systems Government Services, via the OPM
  Training Management Assistance program

17
Charge to Consultants

• How well do the current DLAB/ASVAB represent
  aptitudes, abilities, prior experiences, or other
  components critical to learning a foreign
  language in the context of the DLIFLC?
• What measures of aptitudes, abilities, prior
  experiences, or other components, if any, should
  be added to or removed from DLAB to improve
  selection for
• all languages and skills?
• specific language types or language skills?

18
Fundamental Questions

• What do theoretical developments in educational
  and cognitive psychology with regard to
  classroom-based, second language learning imply
  for the improvement of the DLAB? Are there new
  constructs that relate to acquisition of a second
  language that can be used in a predictive
  battery?
• What do current theories and practices in
  assessing aptitude for foreign language learning
  among adults, or for adult learning in general,
  imply for the improvement of the DLAB? Have there
  been significant developments in conceptualizing
  the assessment of aptitude for foreign language
  learning since the era when the DLAB or the
  Modern Language Aptitude Test (MLAT) were first
  developed?
• Could personnel selection and classification for
  language specialist career fields be improved by
  relying more heavily on measures of general
  aptitude for learning rather than seeking to
  refine measures of aptitude for learning specific
  kinds of things, such as languages in general, or
  language families, or specific languages or
  language skills? What mix of these approaches
  would yield the best predictions of success at
  DLIFLC?

19
Preliminary Findings

• Keep all or most of existing DLAB, with some
  changes or additions from existing tests.
• Add new subtests, e.g.
• perceptual speed
• working memory
• phonological discrimination
• L1 listening ability under poor acoustics
• Consider a two-tiered approach
• Investigate alternative scoring strategies, e.g.
• DLAB composites
• Compensatory models

20
Next Steps

• Complete current projects
• Design short-term studies
• Development long-range research agenda

21
Questions or suggestions?

• John.Lett_at_monterey.army.mil