Assessing adult literacy

1
Assessing adult literacy The aim, use and
benefits of standardized screening tools
2
"You teach a child to read, and he or her will be
able to pass a literacy test."
3
Presentation

• Research questions
• Tools and methods
• Why screen? Why not?
• What to expect from screening?

4
Research questions

• Screening tools
• What is the aim, use and benefits of screening
  tools in the process of identifying low literate
  adults?
• Standardized screening tools
• Practical usability
• Broad and standardized screening instrument
  desirable?

5
Policy on literacy

• Flemish government
• Increase of literacy in the Flemish population
• Policy documents
• Strategic plan on Increasing literacy
• Operational plan on Increasing literacy
• Objective phased and systematic screening of
  literacy among adults

6
Tools and methods

• Three qualitative research methods
• A literature survey of literacy skills and the
  screening of those skills
• A document analysis (qualitative content
  analysis) of 31 existing screening devices
• Semi-structured in-depth interviews with 33 key
  figures from Flanders and The Netherlands
• Low literate individuals and representatives
• Screening professionals
• Policymakers
• Academic experts
• Professionals working in secondary education

7
What is screening?

• Literacy screening implies
• based on behaviour or performance which may or
  may not be induced
• the literacy skills of an individual or group
• are assessed and evaluated
• using a benchmark or norm
• (in a short protocol)

8
Screening how?

• Five types of screening instruments
• Test measuring an induced literacy performance
  using a device developed beforehand (which may or
  may not be standardized)
• Proxy measurement - mapping out factors that show
  high correlation with low literacy
• Self-assessment making an estimate of ones own
  literacy performance level on the basis of
  structured questions
• Interview / discussion oral questioning of the
  extent of literacy on the basis of a
  questionnaire
• Observation consciously observing behaviour
  with a view to describing and estimating literacy
  skills

9
Screening why (not)?

• To strengthen adult literacy, the first step is
  the indentification of low-literacy (on an
  individual level).
• Still not al social domains (civil society
  organisations, health care, work place
  organisations, ) seem equally open to assessment
  by means of a standardized screening tool.
• Reasons
• The interpretation and operationalisation of
  literacy itself
• The adequate functionality norm for literacy (and
  its effects)
• The very limited diagnostic information (and what
  to do with it?)

10
Multi-literacies

• Multiple viewpoints
• Level of literacy performance (the degree of
  literacy of an individual)
• vertical dimension
• The range of contexts and situations in which an
  individual can function using written language
• horizontal dimension
• Different user perspectives (micro, meso, macro)
• Different sorts of literacy, different sorts of
  text (prose literacy, document literacy,
  numeracy, digital literacy, )

11
Multi-literacies
12
Multi-literacies
Literacy as a cover term is so broad it must
almost be defined for each occasion on which it
is used. (Kintgen et al, 1988) () there are
no clean cut logical or empirical criteria that
can help settle disputes about what functional
literacy is or is not. These facts should not be
taken as a sign of inadequacy of the definition.
The definiendum itself is a fuzzy reality and it
should not be presented as if it were not. (De
Glopper, 1992)
13
Multi-literacies

• Consequences for screening
• It seems impossible to screen all aspects of
  literacy
• together
• to the same extent
• in the timeframe of a quick screening protocol
• Content analysis of existing screening devices
  no single instrument focuses on literacy as a
  whole they test separate skills (writing,
  listening, ) or one sort of literacy (often
  prose and document literacy)

14
The cut-off point (1)

• The cut-off / norm?
• When is a person or group functionally literate?
• Where is the boundary between having and not
  having adequate literacy skills (the at risk
  line)
• Should we use one?
• From a theoretical point of view
• Yes how can we evaluate literacy skills when
  there is no norm? It is necessary to identify the
  at risk group opposed to a not at risk group
• But no it can never do justice to the
  complexity of literacy
• One or more?
• One - for the totality of the population
• More than one for each subgroup or population
  category

15
The cut-off point (2)

• The cut-off / norm?
• Who should define or specify the cut-off
  criteria?
• Intense societal or public debate
• International comparison
• or the screened individual?
• Practical consequences
• Too high a large population at risk
• targeted policy is impossible
• over-problematisation / people talked into
  believing they have a problem
• Too low
• a wrongful acceptance of a lack of minimum
  skills
• ! Screened individual has to accept the cut-off
  point

16
Screening information

• What information to expect from screening?
• Realism has to rule
• Only a cursory and generalised portrait of
  performance(illusory effect)
• Offers no evidence or knowledge of the interplay
  between literacy skills and experiences, no
  diagnostic information
• But
• this makes explicit what otherwise would stay
  unnoticed or intuitive
• Important
• For most interviewees, screening is just a first
  step that should not take place without a
  possible
• Diagnosis
• Follow-up
• Training

17
Screening expectations

• Because research data warn for unrealistic
  expectations
• Any (new) screening instrument should
• Critical success factors
• exhibit several essential features
• demonstrate adequate levels of validity,
  reliability
• it should be neutral, fair, efficient
• should be guided by the particularities of the
  target group
• A well-defined target group is necessary
• A group with a uniform needs profile
• A group that can be reached for screening (for
  instance schools, providers of vocational
  training courses, )

18
The use of a screening tool

• Perhaps the most important element not the
  screening tool itself but the way it is used
• screening should be
• Be part of a formative process
• Screening results should motivate the screened
  individual to embark on an educational process
• Start from the needs of the person
• Aim strengthen adult literacy
• Not simply counting heads or labelling people
  at risk

19
Assessing adult literacy The aim, use and
benefits of standardized screening tools
Queestions, comments, feedbag,
feedback
20
Download the research report (in Dutch)

• www.hiva.be
• www.cteno.be

21
(No Transcript)
22
Document analysis
Quickscan Intaketoets alfabetisering NT2 NT2 toets online
Speuren naar cijfers en letters ItemDito Nivor-toetsen lezen en schrijven
Domino 2 Diss NT2 profieltoets
Domino 3 Tool Staatsexamen NT2
Domino 4 NT2 Profieltoets alfabetisering Centraal examen
Tibo - toets instroom beroepsopleiding Naturalisatietoets Spel-direct
Voorbeeldtoetstaken NT2 1.2 Taal- en rekentoets voor reïntegratie Staal. Schriftelijke taalvaardigheid
Voorbeeldtoetstaken NT 2 1.1 Taal en rekenen Instaptoets 1.2-2.1
Digibo 3.0 Voortgangstoetsen Havo en vwo
Intaketoets NT2 Digitale schrijftoetsen VMBO