Pitchbook A4 template

1
19 FEBRUARY 2009
SAMS WORKSHOP - THE 2008 NSC RESULTS The Case
of Mathematics
John D. Volmink - Chairperson of Umalusi
2
The Class of 2008

• New National Senior Certificate (NSC) is a
  completely new qualification based on the
  National Curriculum Statement (NCS)
• The NSC replaces the Senior Certificate which was
  written at the end of 2007 and which will finally
  be phased out in 2011
• Some 533 561 sat for the NSC (590 000 enrolled)
• Approximately 62,5 of all those who have written
  have met the requirements of a pass

3
Comparing the NSC with that of previous years

• The NSC differs from the previous Senior
  Certificate both in structure as well as in
  grading
• A critical feature of the Senior Certificate was
  that the aggregate score was important in
  determining a pass or fail
• To obtain a pass in the NSC candidate needs 40
  in at least three subjects and 30 in three
  others
• Not completely appropriate to compare overall
  results between the two qualifications

4
The quality of the results

• Concerns about quality has become a thorny issue
  and the subject of much media debate (albeit
  somewhat uninformed at most times).
• SA public uses the matric results as the main
  indicators of quality of the education system and
  so there is an understandable concern about what
  these new results of a new qualification is
  saying about the current state of education
• It is important however that we interrogate some
  of our assumptions that inform our beliefs about
  standards and quality

5
Some criteria for comparison

• The number of candidates passing
• The quality of the results
• The trustworthiness of standardisation process of
  the results
• The reliability of the predicative quality of the
  exam for performance at HE level
• The standard of the curriculum

6
Assumptions do affect our perceptions

• One assumption that fuels skepticism is that
• more means less or
• more means worse

7
The number of candidates passing

• It is the clear that there has been a steady
  increase over the last decade in the number of
  learners who enrolled for and wrote the Senior
  Certificate
• Massification is a natural outcome of an
  education system that is non-racist, non-sexist
  and democratic
• The underlying philosophy of the NCS had been
  designed to ensure that most people achieve the
  minimum requirement for a pass

8
Stellenbosch University Statement (January 2008)

• In the light of phasing out of matriculation
  exemption and of the distinction between subjects
  on the higher and standard grade, a significant
  number of learners will probably qualify for
  basic admission to university on the basis of
  their subject combination..The implication is
  that many more learners may be able to apply for
  admission to Stellenbosch University. .
• With this background, it becomes clear why it is
  necessary for the University to have its own
  measure to interpret the meaning and value of the
  new NSC and also to generate additional
  information in order to make finer distinctions.

9
The number of candidates passing Trends in
senior certificate enrolment and passes
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
10
The number of candidates passing Trends in
senior certificate enrolment and passes
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
11
Senior Certificate examination pass rates
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
12
Senior Certificate examination pass rates
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
13
Senior Certificate examination results by era
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
14
One interpretation of the trend in pass rates

• One of the first steps taken by Kader Asmal
  on his appointment as Minister in 1999 was to
  launch a campaign to improve SC pass rates...
  However deeper analysis by Umalusi and others
  since 2003 strongly indicates that the bulk of
  these effects were achieved by manipulating the
  results by means of four measures eliminating
  high risk candidates, encouraging candidates to
  register at the easier standard grade level,
  lowering the standard of the examination
  questions, and using political arguments rather
  than statistical techniques to raise raw scores
  during the moderation process (Dr Nick
  Taylor, JET sercices, member of Umalusi
  Statistics Committee)

15
Senior Certificate examination results and entry
into HE
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
16
Senior Certificate examination results and HE
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
17
Senior Certificate examination results and HE
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
18
Senior Certificate examination results and HE
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
19
Senior Certificate examination results
Senior Certificate examination results, all
schools, 1994-2008
Sources DoE, 2004, 2005a, 2005b, 2005c, 2007,
2008, 2009
20
The trustworthiness standardisation of the
results

• Responsibility for matriculation results has
  changed hands
• JMB (1918 to 1992)
• SAFCERT (1992 2001)
• Umalusi (2002 to date)
• The one constant has been the standardisation
  process used for the examinations.
• This is crucial to
• Obtain equivalence of the standard of the results
  
• Across years, subjects and examination
  authorities
• It is also necessary in order to deliver a
  relatively constant product to the HE sector and
  to the workplace

21
Adjustment of Marks for the Class of 2008

• Number of subjects 29
• Number of examination subjects 54
• Raw marks accepted 30
• Moderate upward adjustments 16
• Moderate downward adjustments 7
• Slightly larger upward adjustments were made in
  the case of Physical Science
• Cannot solve the problems of teaching and
  learning using graph paper

22
What informed the 2008 standardisation of the NSC
results?

• Raw marks, particularly in practical subjects
• Historical averages used 5 year national
  and/or provincial marks, as well as combining HG
  and SG averages where applicable.
• Hybrid norms were created for new subjects from
  learner performances in similar subjects across
  provinces
• Findings of the Umalusi year-long research
  project into comparative evaluation and old and
  new curricula and examinations for selected
  subjects.

23
Implications of the 2008 NSC results for HE

• NSC examination was possibly harder for those at
  the bottom and softer for those at the top
• Need additional information to discriminate at
  the top end.
• However most candidates with B pass in NSC
  would have equivalency wrt cognitive demand and
  content knowledge of a SC candidate who obtained
  exemption pass in 2007.
• The NSC candidate also brings additional values,
  skills and attitudes.

24
Umalusis research on comparisons of NSC with
NATED 550 Quality of the curriculum

• A year-long research project was conducted to
  determine
• comparative evaluation of old and new curricula
• relative levels of cognitive demand of the 2008
  NSC examinations
• Research completed by Umalusi into the standard
  of the NCS curriculum confirms that in most cases
  the NCS presents a greater cognitive challenge
• The NCS also represents modern, updated and more
  demanding versions of previous subjects

25
The Case of Mathematics

• Mathematics is continuously singled out in
  discussions on
• Curriculum reforms
• Examination results
• Historically 30 to 40 of secondary schools in
  the country simply did not offer any mathematics
  beyond grade nine
• In 2008 all 590,000 learners took some form of
  mathematics

26
Mathematics and the Class of 2008

• The Class of 2008 had 63,038 learners who passed
  mathematics at the 50 level or higher
• This stands in contrast to the 25,000 who passed
  Higher Grade mathematics in 2007
• A further 207,230 learners passed mathematical
  literacy, most of whom would not have done any
  mathematics in the previous system. A total of
  16,557 passed ML at the level of 80 or higher

27
Math pass rate
Number passing Math at different thresholds
Sources DoE, Abdridged Report on the 2008 NSC
Examniation Results, December 2008 Doe,
Technical Report NSC, December 2008
28
Trends in HG Math pass rate
Number passing HG Math

• Pass at 50 or higher

Sources Vithal, 2008, Table constructed from
data in CDE Research Report No 13 Kahn 2007, DoE
Senior Certificate Technical Report 2006 DoE
Senior Certificate Report 2007. Figures rounded
off
29
Trends in HG Math pass rate
Number passing HG Math

• Pass at 50 or higher

Sources Vithal, 2008, Table constructed from
data in CDE Research Report No 13 Kahn 2007, DoE
Senior Certificate Technical Report 2006 DoE
Senior Certificate Report 2007. Figures rounded
off
30
Percentage distribution of Mathematics results
over three years
31
Number of enrolments over three years from 2005
to 2007
32
Percentage distribution of learners raw marks in
2005
33
Percentage distribution of learners raw marks in
2006
34
Percentage distribution of learners raw marks in
2007
35
Percentage distribution of learners raw marks in
2008
36
Cognitive demand for HG and SG Mathematics
of marks at each level of cognitive demand on
HG and SG Math papers
Sources Draft Composite Exam Analysis Report
Maths, Umalusi Research, 2008
37
Comparison of cognitive between 2008 exemplar and
final Mathematics papers
Comparison of cognitive between 2008 exemplar and
final Mathematics papers with suggested Subject
Assessment Guidelines (SAG)
Sources Draft Composite Exam Analysis Report
Maths, Umalusi Research, 2008
38
Some concerns about mathematics and the Class of
2008

• It is of concern that it appears that the
  cognitive challenge for mathematics for the class
  of 2008 was closer to the old SG level as judged
  by the Umalusi research panel
• Furthermore there was a mismatch between the
  suggested cognitive demand as provided by the
  SAGs and the cognitive challenge in the exemplars
  and the final mathematics examination
• Only about 46 of learners met the minimum
  requirements for a pass in mathematics

39
The NCS philosophy

• The National Curriculum Statement represents
  different set of standards for all
• It reflects the new values embedded in the
  Constitution
• Also aims to develop learners who can respond to
  the growth and development of knowledge and
  technology and the demands of the 21st century

40
What is the NCS?

• Nationally set curriculum
• Internationally benchmarked
• Modern and relevant 21st Century skills
• Reflect the progressive values of the
  constitution
• Focus on Africa and South Africa

41
The design of the NCS
ASs
leaves of the tree
LOs
branches of the tree
COs and DOs
trunk of the tree
10 FUNDAMENTAL VALUES roots of the tree
CONSTITUTION soil
in which the Fundamental Values are grounded
42
Nine principles of the NCS

• Social transformation
• Outcomes-based education
• High knowledge and skills
• Integration and applied competence
• Progression
• Articulation and portability
• Human rights, inclusivity, environmental and
  social justice
• Valuing indigenous knowledge systems
• Credibility, quality and efficiency

PRINCIPLES nutrients that feed the tree
43
Five critical and seven developmental outcomes

• Solve problems
• Work with others
• Manage self
• Communicate early
• Use science and technology
• Understand world as a set of related systems
• Strategies to learn
• Citizenship
• Cultural and aesthetic sensitivity
• Education and career opportunities
• Entrepreneurial opportunities