The Impact of NCEA on Student Learning

1
The Impact of NCEA on Student Learning
Motivation

• Luanna Meyer
• Professor of Education (Research)
• Victoria University of Wellington
• 4 July 2006

2
This research was supported in part by Contract
397-2148 awarded to VUW from the Ministry of
Education

• Research Team at Victoria University
• Luanna Meyer, College of Education
• John McClure, School of Psychology
• Frank Walkey, School of Psychology
• Lynanne McKenzie, College of Education
• Kirsty Weir, School of Psychology

3
The Impact of NCEA on Student Learning
Motivation

• Multi-method Quantitative Qualitative
• Student survey, year 10 and years 11-13 item
  ratings, individual item responses, open-ended
  questions
• Student achievement data (years 11, 12 and 13)
• Student focus groups (year 10 and years 12-13)
• Student individual interviews (year 11)
• Parent focus groups
• Teacher focus groups

4
Major Focus Student Voice

• Purposive sample from 20 secondary schools
• 15 North Island and 5 South Island
• 8 Auckland region 11 urban Auckland,
  Christchurch, or Wellington 9 in smaller cities,
  towns, or rural areasboth islands, Northland and
  the East Coast
• Size mix, largest to smallest
• Diverse student population, 2 Maori immersion
• 17 coed state, 1 integrated state, 1 boys, 1
  girls
• School decile levels 5 decile 1-2 schools
  10 decile 3-8 schools 5 decile
  9-10 schools

5
Major Research Questions

• What is the connection between student choice and
  student learning?
• 2. What is the impact of NCEA design features
  on student enjoyment and motivation?
• 3. What is the relationship between student
  motivation and student achievement?

6
Theory Research Evidence on Motivation
Orientations to Learning
Attribution theory (Weiner, 1985) Outcomes
attributed to effort, ability, task difficulty
and luck, each of which varies re locus and
stability

• If a teacher sees a student struggling with a new
  task, what should he/she say? Why?
• (a) Keep trying, its easy!
• (b) Thats very hardyoull have to work at it,
  and Ill check later to see if you need more
  help.

7
Theory Research Evidence on Motivation
Orientations to Learning

• Views of Intelligence
• Static Ability or an individual characteristic
  that cannot be changed

• Incremental A skill or set of skills and
  knowledge that can be expanded
• Examples What is ones view of artistic talent?
  Mathematical ability? Knowledge of technology?

8
The impact of motivation orientations on learning
and achievement

• Task values that motivate achievement

• Younger children motivated primarily by interest
  and enjoyment

• Older children also increasingly motivated by
  judgments of utility and importance

9
Survey of Students in Year 10 and in Years 11-13

• Demographic questions Gender, year in school,
  student status (international or domestic) and
  whether the student worked part-time
• Level of NCEA the student expected to complete
  before leaving school Levels 1, 2 and/or 3
• Future plans Three things the student is most
  likely to do after leaving school, chosen from 12
  possibilities including further study, gap year,
  employment, going overseas, doing sports, just
  hanging out, starting a family etc

10
The Student Survey

• Self-rating sections of the survey
• Influence on choice of subjects
• How students think about their learning
• What students like/dont like about NCEA, and
  other assessments
• Open-ended 3 things you like and 3 things you do
  not like about NCEA and the Record of Learning

11
Data from the Record of Learning (Years 11-13)

• Total credits achieved (AS and US)
• Total Achievement Standards attained, then by AME
• Total Unit Standards attained
• Achievement Standard grade average
• The grade average on the Record of Learning
  (1-100 range)
• All NCEA qualifications to date (Levels 1, 2, 3)
• For year 13s, UE achieved yes or no
• Ethnicity and fee rebate status
• Note Several other variables were also entered,
  including SNAs and any additional national
  qualifications attained

12
Survey Participants

• Year 10 2,083 respondents
• 1,976 domestic and 79 international students
• 29 reported being employed part-time

• Year 11-13 3,569 respondents
• 3,290 (92) domestic and 191 (5) international,
  with 88 not responding for this item
• 47 reported being employed part-time

13
The level of NCEA students indicated they plan to
finish before leaving school
14
What levels of NCEA do Year 11-13 students of
different ethnicities expect to complete?
15
Percentages reporting part-time work by school
decile levels
16
Percentage of students reporting doing part-time
work by ethnicity
17
Average number of credits by ethnicity across all
achievement variables
Total total credits gained Total Unit
total unit standards gained Total Ach total
achievement standards gained Achieved
Achievement standards Achieved Merit
Achievement standards with Merit Excellence
Achievement standards with Excellence GPA
Grade point average
18
Total credits achieved by different ethnic groups
19
Total Unit Standards completed by ethnicity
20
Total numbers of Achievement Standard credits by
ethnicity
21
Quartile GPA performance by percentage of Year
11-13 students in different ethnic groups
22
What percentage of year 13 students in different
ethnic groups achieved University Entrance?
23
Survey Results Influences on Subject Choices

• Utility or Importance Subject needed to achieve
  a future goal (e.g., for a career or for
  University Entrance)
• External Subject taken for peripheral reasons
  (e.g., taken by my friends, its easy, it fitted
  my timetable)
• Interest Subject matches personal interest and
  enjoyment (e.g., I enjoy the subject)

24
Sample Items for Influences on Subject Choice

• Utility/Importance
• The subject gets me the number of credits I need
• It is related to what I might study at tertiary
  in future
• I need it for University Entrance
• It is related to a future job or career goal
• Interest
• Im interested in the subject
• I enjoy the subject
• Im very good at the subject

25
Influences on Subject Choice

• External Factors
• My friends are taking it
• I like the teacher who teaches the subject
• Because the subject is assessed by assignments
  and not final exams
• The subject is easy
• It fits my timetable

26
Relationship between Influences on Subject
Choices and Credits and Grades Attained
plt.001
plt.01, plt.001
27
Relationships of Choice Influences to Achievement

• Both the Utility/Importance and Interest motives
  relate positively to students GPA and attainment
  of Merit and Excellence

• The External factor is negatively related to GPA
  and all levels of passes (AME), and is positively
  related to the number of Unit Standards passed.
• Unit Standards appealing most to students not
  selecting subjects based on interest or utility.

28
Influences on Student Subject Choices

• One might have predicted that students
  accumulating primarily Unit Standards were
  motivated primarily by Utility/Importance, but
  instead they appear to be making choices based on
  External Factors.
• What might this mean?
• How can we encourage students enrolling in Unit
  Standards to make better decisions?

29
Gender Differences

• Girls were more likely than boys to attribute
  subject choices to the Utility/Importance and
  Interest motives
• There were however no differences by gender for
  self-ratings of External influences
• By Years 12-13, boys shift from selecting
  subjects based on External influences to basing
  decisions on Utility/Importance
• This could be because boys shift to working
  harder between years 11 and 12-13, or because
  those who are high on External motivation have
  left school by year 12-13.

30
How Students Think about Their Learning

• Doing My Best Valuing work to attain merit or
  excellence and aiming to get a good education,
  not just accumulate credits
• Doing Just Enough Doing only what is required to
  attain needed credits or to get by, influenced
  more by what friends think and whether study
  interferes with other things

31
Sample Items for How Students Think about their
Learning

• Doing My Best
• I will strive for Merit or Excellence even when I
  dont need this to achieve my goals
• I aim at getting a good education, not just
  completing tasks to get credits in NCEA
• I want credits from school that lead to a good
  job or career
• Doing Just Enough
• Once I have my 80 credits, Ill be satisfied
• What my friends think influences whether I work
  in school
• For me, getting achieved will be good enough

32
Relationship between Achievement and How Students
Think about their Learning

• Doing My Best correlates positively with GPA
  scores, getting Merit and Excellence, and the
  number of total credits attained.
• Doing My Best correlates positively to
  achievement standard credits and negatively to
  unit standard credits
• Doing Just Enough is related to lower achievement
  and more unit standard credits

33
What Students Like/Dont Like about NCEA and
Other Assessments

• Work Avoidance Views that favour easy options
  and doing only the minimum
• Getting Feedback Wanting more feedback on ones
  work and information throughout a course not just
  at the end
• The Excellence Factor Working to maximise ones
  achievement and wanting to demonstrate high
  achievement

34
Sample Items for What Students Like/Dont Like
about NCEA and the Record of Learning

• Work Avoidance
• Being able to relax after I get my 80 credits
• Not having to do parts of a course that I dont
  like when I dont need those credits
• Getting Feedback
• Taking subjects where the teacher assesses my
  work during the course rather than only through a
  final exam
• Getting feedback on my work
• Excellence
• Being able to seek more than the minimum credits
  whenever I wish
• Having a final end-of-course external exam with a
  grade scaled so I
• can compare myself with others

35
Correlations between What Students Think about
their Learning, What Students like/Dont Like
about NCEA and Assessment, and the Number of
Credits Attained
plt.01 plt.001
36
Correlations between What Students Think about
their Learning, What they Like and Dont Like
about NCEA and Assessment, and their Grades
plt.05, plt.01 plt.001
37
Gender Differences?

• Girls scored higher than boys for the Doing My
  Best and Getting Feedback factors
• However
• There were no gender differences on Doing Just
  Enough, Work Avoidance, and Excellence
  orientations

38
Relationship between Achievement and What
Students Like/Dont Like about NCEA and
Assessment

• Work Avoidance correlates negatively to students
  grade averages and total credits gained, as well
  as to Merit and Excellence
• Excellence factor related positively to all
  measures of student achievement
• Getting Feedback correlates to the number of
  Achieved grades. A diverse group of students
  valued Getting Feedbackacross high and low
  achievingbut students from low decile schools
  were most interested in this factor

39
Relationships Across Factors and Predicting
Achievement

• Students motivated by Doing My Best were most
  likely to select subjects based on
  Utility/Importance and less likely to report
  External influences
• Students motivated by Doing Just Enough were most
  likely to select subjects based on External
  influences unrelated to the subject.

40
Is Doing Just Enough Good Enough?

• If students who are Doing Just Enough are
  acquiring fewer creditswhich they arewill a
  goal of just enough end up being enough?
• This motivation orientation assumes that students
  know and can work towards attaining exactly what
  they needno more, no less.

41
Is Doing Just Enough Good Enough?

• Unless this motivation orientation is challenged,
  are we allowing students to settle for less and
  sell themselves short?

Do we really believe that Doing Just Enough
rather than Doing My Best is all that some
students are capable of affirming and
achieving?
42
Influences on School Performance

• Students performance in school will be
  influenced by their beliefs about their own
  abilities (static or incremental) and personal
  characteristics (motivation orientation,
  perseverance, etc.)
• These characteristics can be related to and
  influenced by gender, culture, family
  circumstances and economic resources.
• But both can also be influenced by what the
  school and teachers do, the curriculum, and the
  teaching and learning processthese are the
  things we can change!

43
Designing Effective Interventions

• We found strong relationships between school
  performance and motivation orientations reported
  by the studentsconsistent with the international
  research literature.

Student dispositions and attributions are
themselves amenable to change and can be
influenced by what schools and teachers do.
How might we scaffold student learning to improve
student achievement and goal attainment?
44
Key Findings from Qualitative Analyses of Focus
Groups and Interviews

• Parents indicated strong support for NCEA in
  principle while suggesting some areas for
  improvement
• Students, teachers, and parents are
  overwhelmingly positive about the impact of
  internal assessment on student study/learning
  behaviours and achievement
• Students continue to perceive some uneven
  opportunities across schools for access to more
  subject choices and study across levels of NCEA
• Students and teachers perceive inconsistencies in
  the difficulty of some standards across some
  subjects
• There were frequent comments on getting a Not
  Achieved despite answering Merit and/or
  Excellence questionsmost had never experienced
  this, but had heard about this anomaly and felt
  it was illogical and unfair

45
Qualifications Design Issues

• Areas of Strength
• Students and adults are extremely positive about
  the mix of internal and external assessments,
  valuing internal to guide learning and workload
  and valuing external to support perceptions of
  NCEAs credibility and towards consistency across
  schools and subjects

• Students and adults were overwhelmingly positive
  about internal assessment overall, while
  affirming the importance of ensuring that
  teaching and learning should not focus too
  narrowly on standards rather than subjects

46
Qualifications Design Issues

• Issues to consider
• Some student comments suggest that the 80 credit
  requirement can become a maximum and encourage a
  minimalist approach to learning

• Students who commented on attaining merit and
  excellence perceived that more opportunities were
  needed to motivate and recognise higher levels of
  achievement

• Students, parents and teachers made many
  suggestions for improving NCEA more detailed
  grade bands, awarding Merit/Excellence points for
  NCEA, continuing to work on inconsistencies
  across subjects where these appear, and resolving
  anomalies (e.g., Not Achieved despite passing
  Merit or Excellence questions)

47
Why Student Voice?

• Students motivational orientations can be
  changed
• We need to consider the impact of interventions
  on not only immediate school performance but on
  students underlying dispositions to learningif
  our goal is to support long term, meaningful
  student achievement outcomes controlled by the
  learner and not by others!

Longitudinal research can assist in directing us
towards making decisions towards better
outcomessupported by actual achievement
evidencerather than those driven by the past or
based on adult beliefs about what our children
need.