National Assessment Systems:

1
National Assessment Systems

• Lessons Learned from International Experience
• by Luis Benveniste

2
Overview

• International perspectives on student assessment
• Why have Southern Cone countries espoused
  national assessment systems?
• Conclusions

3
Objectives of Assessments 1. To Promote
Educational Quality

• by monitoring student learning at the classroom,
  school, province and national levels
• by linking results back to educators and
  policymakers to inform instructional and
  curricular reform
• by promoting high national standards for student
  performance
• by promoting clear expectations for school level
  performance

4
Objectives of Assessments 2. To Inform Policy
Making

• by evaluating the impact of specific policies or
  interventions
• by using results to design compensatory policies
• by identifying variables associated with student
  learning

5
Objectives of Assessments 3. To Foster Greater
Efficiency and Accountability

• by linking student achievement results to
  incentives for teachers, schools, districts or
  provinces
• by fostering public accountability and
  transparency
• by informing parents and families about school
  performance and promoting choice

6
Objectives of Assessments 4. To Inform Student
Selectivity

• by using tests to place students in a certain
  track or school
• by using results to select who progresses to the
  next level
• by using tests to determine if a student can
  graduate
• Singapore Examinations and Assessment Board

7
International Assessments

• Comparative perspectives on achievement levels by
  subject
• Analyze effects of country policy differences
• Build regional networks to share knowledge and
  strengthen local capacity

8
International Assessments SACMEQ

• Consortium of fifteen Ministries of Education in
  Southern and Eastern Africa
• Two major cross-national studies and a third
  under development
• Training modules

9
International Assessments PASEC

• Member countries of CONFEMENFrancophone Africa
• Cohort studies of student achievement
• Thematic studiese.g. contract teachers or
  teacher pre-service training

10
Impact of International Assessments TIMSS

• United States Lower performance relative to
  Asian countries in 1995, 1999 and 2003 math and
  science
• A curriculum that is a mile long and an inch
  deepLack of opportunities for children to
  construct meaning
• US vs. East Asian educational model
  collaborative teaching, in-depth conceptual
  curriculum, parental involvement
• Results provided momentum to curriculum overhaul
  and standards-based reform

11
Overview

• International perspectives on student assessment
• Why have Southern Cone countries espoused
  national assessment systems?
• Conclusions

12
Southern Cone Countries
13
National Assessments Comparative Overview
14
National Assessments Comparative Overview
15
Political Context Chile

• Political context at inception of testing
• Municipal decentralization and privatization
  (voucher system)
• Promote consumer accountability
• Promote market competition and choice

16
Political Context Chile

• Assessment characteristics SIMCE
• Universal test in Grades 4 and 8
• Public dissemination of school level outcomes
• Parental outreach strategies
• Rankings by school (within socio-economic
  banding)
• Promotion of school level responsibility over
  testing outcomes

17
Political Context Chile

• Linking test scores to education resourcing
• Targeting the Poor ? Assessment results as an
  avenue for channeling compensatory funding
  (materials and technical assistance) to low
  performing schools (P-900 Program)
• Providing financial incentives to improve teacher
  performance ? National Evaluation System of
  School Performance (SNED) awards salary bonus to
  teachers in high performing schools

18
Targeting the PoorChiles P-900 Program

• Equity focus -- Compensatory program for lowest
  scoring (900) schools in the SIMCE test
• Open to public and subsidized schools
• Three-year formal commitment
• Technical assistance educational resources
• Annual Action Plan

19
Targeting the PoorChiles P-900 Program

• SIMCE as a starting point for diagnosis
• Comprehensive approach
• Teacher professional development
• Targeted support to students at-risk
• Pedagogic counseling and guidance
• Distribution of educational materials and
  classroom library
• Integration of classroom and home environment
• Creation of School Management Team

20
Targeting the PoorChiles P-900 Program
21
Incentives for Performance Chiles SNED

• National Evaluation System of School Performance
  (SNED)
• ?
• SIMCE assessment data as a means to
• Link improved teacher performance to teacher
  compensation
• Reward school practices which contribute to
  student learning
• SNED award is equivalent to a one-month salary

22
Incentives for Performance Chiles SNED

• SIMCE accounts for 65 of SNED score
• Effectiveness Raw SIMCE score in Math and
  Spanish
• Value Added Average SIMCE score gain
• Other parameters
• Equality of opportunities (retention rates
  integration projects)
• Working conditions (full staffing substitute
  teachers)
• Initiative (pedagogical activities, school
  development plan)
• Parent-teacher integration

23
Incentives for Performance Chiles SNED

• Financial awards directed to entire teaching team
  to encourage collaboration
• Address discrimination against disadvantaged
  students
• Clear criteria established on expected behaviors
  and desirable school characteristics
• Schools compared to others with similar
  geographic and socioeconomic characteristics
• Fair and transparent system

24
Promoting Consumer Choice and Accountability
Chile

• Public dissemination of school level outcomes
• Parental outreach strategies through media
• Rankings by school (within socio-economic
  banding)
• Promotion of school level responsibility over
  testing outcomes

25
Political Context Uruguay

• Political context at inception of testing
• Centralized educational system
• Assessment as a means to pursue consolidation of
  social equity
• Reference to Argentine and Chilean experience
  assessment practices

26
Political Context Uruguay

• Assessment characteristics UMRE
• Universal test in Grade 6, but controlled sample
  for Grade 3
• Confidentiality of school level data
• Teacher outreach strategies
• Focus on equity and socio-cultural variables
  rather than student achievement scores per se
• Accent on State responsibility over testing
  outcomes

27
UruguayBuilding Consensus

• Teacher Unions concerns
• Categorization/stigmatization of schools
• Usage for teacher incentives/punishments or merit
  pay schemes
• Promotion of private over public education
• Promotion of market-oriented reforms

28
UruguayBuilding Consensus

• Educators concerns
• Concerns over external organization and
  application
• Competition to the inspectorate
• Appropriateness of standardized/multiple choice
  instruments
• Concerns over fairness of using same test for
  all SES contexts

29
UruguayOvercoming resistance

• When UMRE appeared, we had a brick on each
  hand. I was ready to kill them. I had all my
  justifications against them ready. Little by
  little they convinced us. Now, after all that
  has happened and as we get more results, they
  convince us even more. It is OK that the test is
  obligatory. It has been a valuable experience.
• A representative from the Association
• of Private Education Establishments

30
UruguayGovernments strategy

• Focus goals of assessment system
• (Re)tailor national assessment priorities prior
  to launching UMRE
• Espouse a participatory approach
• Extensive prior information campaign
• Involve educators in design process
• Creation of a diverse Advisory Board

31
UruguayGovernments strategy

• Managing the stakes of assessment
• Emphasize State responsibility in education
  quality
• Limited distribution of school scores
• Create a climate that focused on pedagogical
  dimension of testing
• Mobilize inspectorate as support network to
  foster pedagogical reflection

32
UruguayGovernments strategy

• Contextualize student achievement in relation to
  socio-cultural variables
• Equity objectives Link assessment to specific
  compensatory/support activities
• Focus on lessons from effective schools from
  low SES contexts
• Leveling playing field between public and private
  sectors

33
UruguayPresentation of Results
Percentage of Students by Performance in Math
and Sociocultural Context
34
Overview

• International perspectives on student assessment
• Why have Southern Cone countries espoused
  national assessment systems?
• Conclusions

35
Pitfalls of Testing

• Risks of High-Stake Scenarios
• The stigma of results
• Poor results lead to frustration among teachers
  who may feel disempowered to bring about change
• Stigmatizes low performing schools, leading to
  the "flight" of better students, leading to
  further segregation in educational services

36
Pitfalls of Testing

• Risks of Low-Stake Scenarios
• Educators pay little attention to the test
• Assessment comes and goes
• No changes in school or classroom behavior
• "Lost opportunity" to reflect on student
  achievement, classroom pedagogy or school
  administration

37
Some Concluding Remarks

• High-stake assessment systems
• Foster public accountability
• Performance incentive systems
• Prompt close alignment with tests
• Motivate changes in school/classroom
  behavior
• But may also have negative consequences, such
  as teaching to the test
• Prompt political resistance

38
Some Concluding Remarks

• Low-stake assessment systems
• Limited accountability
• No direct consequences vis-à-vis test results
• Looser coupling between ed practices and tests
• Can weaken backwash effect of testing
• Onus is on Government to foster greater test
  data use, leading to changes in teaching and
  learning

39
Good Assessment Systems

• Underscore educational goals desired
• Highlight challenges in student achievement,
  classroom pedagogy and school administration
• Provide a framework to map change, design
  necessary interventions and redirect resources at
  school and classroom levels

40
Teacher comments Uruguay

• On the basis of the exam results, we developed
  a plan for the following year. For instance, the
  discussion over problem resolution was very
  important for us in order to go deeper into this
  issue, to work more on reasoning. I dont know
  if this took us further away from the official
  curricular program, but ... Also, weve been
  working on the language curriculum in teacher
  meetings. In these sessions we analyzed some of
  the test items.

41

• Thank You