Apresentao do PowerPoint

1
DEVELOPING EVALUATION CULTURE
Technical features of testing and assessment in
Brazil
Maria Cecilia Guedes Condeixa Science education
and evaluation consultant Brazil
2
LARGE SCALE EVALUATION IN BRAZIL
External evaluation of learning and school
education
How future generations are learning and improving
abilities, competencies and knowledge
Collect significant and reliable data
Enlargement of democracy and democratic
mechanisms
3
Major large-scale assessments
Nationwide / International
INEP Instituto Nacional de Pesquisas e Estudos
Pedagógicos Anísio Teixeira
MEC Ministry of Education
United Nations  
4
SAEB/ANEB NATIONAL EVALUATION OF BASIC SCHOOL
Biannual survey all over the country
Samples of 4th and 8th grades of fundamental
school and 3rd grade of Middle school students
Conduct studies to measure students Reading
and Mathematics performance associated to
schooling conditions
5
Instruments Mathematics and Reading multiple
choice items questionnaires to detect
schooling conditions
Since 1993 Comparable results are assured by
educational statistics theory
Proper amount of items are kept in secret and
saved for subsequent testing (link items)
Part of technical strategies to assure
comparability of results throughout the
years Characterizes an assessment
6
OECD / PISA PROGRAMME FOR INTERNATIONAL
STUDENTS ASSESSMENT
Three-year cycle of studies focusing on reading,
mathematical and science literacy and
background conditions
Survey is based on stratified sample of 15 year
old students
Brazil participates, as partner country, in all
three cycles
7
Each cycle has a major domain 2000 reading
literacy 43 countries 2003 mathematical
literacy 41 countries 2006 science literacy
57 countries
Types of items multiple choice, closed
constructed response (involves judgment of
proposals in a context) open constructed
response (requires personal writing)
8
BRAZILIAN STATES THAT DEVELOP LARGE-SCALE
EVALUATION
Bahia (since 2003)
Pernambuco (since 2002)
Minas Gerais (since 2000)
Ceará (since 2000)
São Paulo (since 1995)
9
Evaluation tends to be annual and to all students
Building references for learning and teaching
Improving local educational policies
All items become available to teachers by request
or on line (no link items)
No link items Possibilities to compare
results are constrained Characterizes testing
procedures
10
BASIC STEPS FOR BUILDING INDICATORS
1st Step - Organizing a framework
Table of specification (minimum framework)
guided by curriculum and school practices
defines themes, contexts, concepts, attitudes
and procedures that may be measured reflects
expectations of specific measurements
11
EXAMPLE 1
The State of São Paulo evaluation table of
Specification has been refined in the course of
its 10 years of existence.
2005 specifications for Reading test
Each line brings a descriptor of evaluation that
guides test item building.
TABLE SARESP 2005
12
EXAMPLE 2
The 2006 Science Framework of Pisa
13
2nd. Step - Building and adjusting the
prime-matter
of tests the bundle of items
  Requires specific expertise Items are
object of call for tenders (generally) Bidders
are non-governmental institutes specialized in
evaluation Items may go to a field trial
Building questionnaires to teachers, principals
and parents Other instruments required to
school conditions evaluation
14
3rd. Step - Application of tests, questionnaires
and other instruments
4th. Step - Development of outcomes
reports indicators