Are there

1
Are there Hidden Variables in Students Initial
Knowledge State Which Correlate with Learning
Gains?

• David E. Meltzer
• Department of Physics and Astronomy
• Iowa State University
• AAPT Winter Meeting
• January 12, 1999
• Anaheim, California

2
What Affects Learning Gains?

• Normalized Learning Gain Hakes g is a
  widely used measure of conceptual learning.
  
  
  g(posttest-pretest)/(100-pretest)
• g is not correlated with pretest scores.
• g is virtually invariant among traditional
  instructors.
• g is correlated with instructional method (higher
  g found for Interactive Engagement methods).
• ____________________________________________
• Many studies assert that correlations exist
  between students performance in physics and
  various preinstruction parameters (e.g.,
  mathematical skill, reasoning ability, etc.)
• Is it possible that such a correlation might
  also be reflected in learning gains as measured
  by g on the FCI, CSE, or similar conceptual
  diagnostic instruments?

3
What if there are Hidden Variables Correlated
with g?

• If g is correlated with any precourse measure
  (such as mathematical ability), this would have
  to be taken into account when comparing
  learning-gain data.
• It is usually assumed that pretest score
  instructional method together determine
  posttest score. However, if precourse measures
  are correlated with g, then
• other hidden variables besides pretest score
  would be required to fully characterize a
  students preinstruction mental state-function.
• could no longer assume that, e.g., equal FCI
  pretest scores necessarily imply equal posttest
  scores for courses taught with identical
  instructional methods.

4
Previous Studies on Factors Influencing Students
Achievement in Physics

• More than a dozen studies report that
  mathematical knowledge is significantly
  correlated with students grades.
• Several studies suggest that logical reasoning
  ability is an independent factor as well.
• However
• These studies almost all use traditional,
  quantitative end-of-chapter problems as their
  performance criterion.
• All of these studies focus on students scores on
  course exams, which are not necessarily the same
  (nor even necessarily correlated) with how much
  the student actually learned in the course.

5
Guiding Themes of This Work

• Research has shown that success on traditional
  problems is not necessarily indicative of
  students conceptual knowledge.
• Students who perform well on exams may have
  learned little, and students who have lower exam
  scores may have learned much (if they started
  with little or no previous knowledge).
• Here we report investigations of factors related
  to learning gains, as measured by pre/posttests
  of conceptual understanding.

6
How Can We Search for Possible Hidden Variables
in Initial Knowledge State?

• Study relationship between learning gains and
• ACT Math score (two samples)
• Math Skills Pretest (algebra trig) (one sample)
• Pattern of wrong answers on conceptual
  diagnostic pretest (three samples)

7
Could a Math Skills Pretest be a Predictor of
Performance?

• H.T. Hudson and others have found significant
  correlation between performance on math skills
  pretest and student performance on traditional,
  quantitative exams.
• Here we examine possible correlation with
  learning gain on a qualitative, conceptual
  diagnostic test (CSE).
• Previous study by Hake et al. (1994) Students
  with high learning gains on FCI scored 19 higher
  (than low gainers) on math skills test taken when
  entering university.
• Fall 98 data sample 59 students enrolled in
  second semester of non-calculus general physics
  course 63 female. Math pretest taken within
  previous 18 months (before taking first semester
  course).

8
Diagnostic Instruments

• Conceptual Survey of Electricity (23-item
  abridged version), by Hieggelke, Maloney, OKuma,
  and Van Heuvelen. It contains qualitative
  questions and answers, virtually no quantitative
  calculations. Given both as pretest and
  posttest.
• Diagnostic Math Skills Test (38 items) by H.T.
  Hudson. Algebraic manipulations, simultaneous
  equations, word problems, trigonometry, graphical
  calculations, unit conversions, exponential
  notation. Not a mathematical reasoning test.
  Given as pretest only.

9
Learning Gains vs. Math Pretest ScoresFall 98
10
Math Pretest Score for High and Low GainersFall
98
11
Does this imply that improving algebraic skills
will lead to increased conceptual learning gains?

• Probably not.
• More likely, performance on a math skills
  test is related to other relevant parameters.
• (Reasoning ability? Learning rate?
  Motivation?)

12
Learning Gains vs. Wrong Answer PretestFall
98
13
Wrong Answer Pretest for High and Low
GainersFall 98
14
Wrong Answer Pretest for High and Low
GainersFall 97
15
Wrong Answer Pretest for High and Low
GainersSpring 98
16
Analysis of Wrong Answer Pattern on Conceptual
Pretest

• Wrong Answers on 11 (out of 23) items on CSE
  pretest analyzed
• Certain specific answer options are identified as
  favored (though incorrect), perhaps
  representing transitional states of knowledge.
• Percentage of favored options selected is
  assigned as WA score. (Correctly answered
  questions are ignored.)

17
Summary

• There is significant evidence that precourse
  measures may be correlated with students
  individual learning gains (even normalized
  gains).
• Purely quantitative skills may be (indirectly)
  related to conceptual learning ability.
• Patterns of wrong answer choices may provide
  evidence of students initial knowledge state
  (and of their probable learning gains in a
  course).