Impact Evaluation of the QIDS UP:

1
Africa Program for Education Impact Evaluation
Impact Evaluation of the QIDS UP Learning and
Teaching Support Material
Department of Education, South Africa
AFRICA IMPACT EVALUATION INITIATIVE, AFTRL
2
QIDS UP

• QIDS UP aims to raise school achievement of South
  African pupils towards international standards by
  raising primary schools above a set standard of
  infrastructure and school inputs within the next
  5 years in the poorest 40 of learners.
• QIDS UP will provide
• Teaching and learning material
• Qualified teachers
• Infrastructure
• Reduction of class size

3
Evaluation Questions

• What is the impact of the provision of the
  minimum resource package on teaching and learning
  outcomes in the quintile 1 and 2 schools?
• What is the additional impact of teacher and
  management training on utilization of learning
  material on teaching and learning outcomes?

4
Indicators

• Learning achievement
• literacy and numeracy for grade 3
• Quality of teaching
• learner perception,
• participation,
• use of material (exercises from the textbooks,
  frequency of use of material)
• Availability of material
• learner textbook ratio per learning program
  (literacy, numeracy and life skills)
• Management
• textbook retrieval rate

5
Evaluation Design(Gauteng Q1-2 Western Cape
Q2, option Q1)

• Question 1 Regression Discontinuity design 250
  participating schools representing 100 of
  eligible schools
• Impact measured as the difference in outcome
  indicators for schools below (125) and above
  (125) the 21.9 cutoff.
• Question 2 Experimental design
• Training will be randomly assigned to half of the
  schools in the program (125 out of the 250
  participating schools) to all principals and
  grade 3 teachers in each school.
• The additional impact will be measured as the
  difference in outcomes between trained schools
  and not trained schools. (w. double dif)

6
Sample and Data

• EXISTING
• March 2007 Auditing census of School
  Infrastructure (30,000 schools).
• Sept 2007 National Grade 3 Assessment. Random
  sample of schools and learners in the country (1
  day in each school).
• Contextual questionnaires principals, teachers
  and student and parents (1 additional day per
  school).
• Provincial administrative system/monitoring
  system of QIDS UP financial and in kind
  resources delivered to each school.
• ADDITIONAL
• Gauteng. Expand sample to the 375 bottom schools.
  Test schools in Sept 2007 and Sept 2008
• Western Cape. Expand sample to the 144 bottom
  schools. Test schools in Sept 2007 and Sept 2008
• One additional province to be considered

7
Staffing Plan
Team National project manager Provincial project
managers Research team (Lead researcher WB, JPAL,
local researcher(s), Quality Assurance
(DoE)) Data collectors Steering Committee
National project manager Provincial project
managers Lead researcher Quality Assurance
8
Timeline

Submit Concept Note to DDG and DG for approval. 6 or 13 Jul 2007
Feedback from management 10 or 17 Jul 2007
Provide comments on contextual questionnaires Jun-07
Consultative meeting with QIDSUP and Provinces End Jul 2007
Data collection (baseline) Sep-07
School training Oct-Nov-07
QIDS UP roll out Starting Jan 2008
Data collection (follow up) Sep-08
Impact evaluation analysis Dec-08
9
Budget