Research on Undergraduate Learning in STEM Disciplines

1
Research on Undergraduate Learning in STEM
Disciplines
Karl A. Smith Civil Engineering University of
Minnesota ksmith_at_umn.edu www.ce.umn.edu/smith
National Research Council National Science
Resources Center Math/Science Partnerships
Workshop December 5-7, 2004
2
Backdrop

• National Research Council Reports
• How People Learn Brain, Mind, Experience, and
  School (1999).
• How People Learn Bridging Research and Practice
  (2000).
• Knowing What Students Know The Science and
  Design of Educational Assessment (2001).
• The Knowledge Economy and Postsecondary Education
  (2002). Chapter 6 Creating High-Quality
  Learning Environments Guidelines from Research
  on How People Learn

3
(No Transcript)
4
(No Transcript)
5
Session Highlights

• Provide overview of some findings from reports
  related to teaching learning.
• Do a activities with you to illustrate some of
  the points covered in the reports.
• Discuss implications for designing learning
  environments that are learner centered, knowledge
  centered, assessment centered, and community
  centered.

6
Designing Learning Environments Based on HPL
7
Learner-Centered Learning Environments
8
Learner-Centered Learning Environments
9
Learner-Centered Learning Environments
10
Knowledge-Centered Learning Environments
11
Assessment-Centered Learning Environments
12
Assessment-Centered Learning Environments
13
Community-Centered Learning Environments
14
Summary Points

• There is an emerging science of learning
• It has major implications for all aspects of
  schooling -- curriculum, instruction, assessment,
  plus preservice and inservice teacher education
• It provides a basis for knowing when, how and why
  to use various instructional strategies
• It can guide the intelligent design and use of
  new curricular materials as well as information
  technologies

15
Lila M. Smith
16
Pedago-pathologies B Lee Shulman Amnesia Fantasi
a Inertia Shulman, Lee S. 1999. Taking
learning seriously. Change, 31 (4), 11-17.
17
What do we do about these pathologies? Lee
Shulman Activity Reflection Collaboration
Passion Combined with generative content and the
creation of powerful learning communities
Shulman, Lee S. 1999. Taking learning
seriously. Change, 31 (4), 11-17.
18
Lila M. Smith
19
Tracking Change - Seymour "The greatest single
challenge to SMET pedagogical reform remains the
problem of whether and how large classes can be
infused with more active and interactive learning
methods." Seymour, Elaine. 2001. Tracking the
processes of change in US undergraduate education
in science, mathematics, engineering, and
technology. Science Education, 86, 79-105.
20
Formulate-Share-Listen-Create (Think-Pair-Share)

• Individually read the quote To teach is to
  engage students in learning. . .
• Underline/Highlight words and/or phrase that
  stand out for you
• Turn to the person next to you, introduce
  yourself
• Share words and/or phrases that stood out and
  discuss

21
To teach is to engage students in learning thus
teaching consists of getting students involved in
the active construction of knowledge. . .The aim
of teaching is not only to transmit information,
but also to transform students from passive
recipients of other people's knowledge into
active constructors of their own and others'
knowledge. . .Teaching is fundamentally about
creating the pedagogical, social, and ethical
conditions under which students agree to take
charge of their own learning, individually and
collectively Education for judgment The
artistry of discussion leadership. Edited by C.
Roland Christensen, David A. Garvin, and Ann
Sweet. Cambridge, MA Harvard Business School,
1991.
22
Strategies for Energizing Large Classes From
Small Groups to Learning Communities Jean
MacGregor, James Cooper, Karl Smith, Pamela
Robinson New Directions for Teaching and
Learning, No. 81, 2000. Jossey- Bass
23
Book Ends on a Class Session
24
Informal CL (Book Ends on a Class Session) with
Concept Tests Physics Peer Instruction Eric
Mazur - Harvard B http//galileo.harvard.edu Pee
r Instruction www.prenhall.com Richard Hake
http//www.physics.indiana.edu/hake/ Chemistry
Chemistry ConcepTests - UW Madison B
www.chem.wisc.edu/concept Video Making
Lectures Interactive with ConcepTests ModularChem
Consortium B http//mc2.cchem.berkeley.edu/ STEM
TEC Video How Change Happens Breaking the
ATeach as You Were Taught_at_ Cycle B Films for the
Humanities Sciences B www.films.com Thinking
Together video Derek Bok Center B
www.fas.harvard.edu/bok_cen/
25
Richard Hake (Interactive engagement vs
traditional methods) http//www.physics.indiana.ed
u/hake/
Traditional (lecture)
Interactive (active/cooperative)
ltggt Concept Inventory Gain/Total
26
(No Transcript)
27
The Hake Plot of FCI
35.00
SDI
30.00
ALS
WP
25.00
20.00
PI(HU)
15.00
ASU(nc)
WP
10.00
ASU(c)
HU
5.00
0.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
Pretest (Percent)
28
Physics (Mechanics) ConceptsThe Force Concept
Inventory (FCI)

• A 30 item multiple choice test to probe student's
  understanding of basic concepts in mechanics.
• The choice of topics is based on careful thought
  about what the fundamental issues and concepts
  are in Newtonian dynamics.
• Uses common speech rather than cueing specific
  physics principles.
• The distractors (wrong answers) are based on
  students' common inferences.

29
FCI Question 17
An elevator is being lifted up an elevator shaft
at a constant speed by a steel cable, as shown in
the figure. All frictional effects are
negligible. In this situation, forces on the
elevator are such that
Pre 64 18 2 11 5
Post 36 60 0 2 1
(A) the upward force by the cable is greater than
the downward force of gravity. (B) the upward
force by the cable is equal to the downward
force of gravity. (C) the upward force by the
cable is smaller thanthe down ward force of
gravity. (D) the upward force by the cable is
greater than the sum of the downward force of
gravity and a downward force due to the
air. (E) None of the above. (The elevator goes
up because the cable is shortened, not because an
upward force is exerted on the elevator by the
cable).
30
Problem Based Cooperative Learning Format TASK
Solve the problem(s) or Complete the
project. INDIVIDUAL Estimate answer. Note
strategy. COOPERATIVE One set of answers from
the group, strive for agreement, make sure
everyone is able to explain the strategies used
to solve each problem. EXPECTED CRITERIA FOR
SUCCESS Everyone must be able to explain the
strategies used to solve each problem. EVALUATION
Best answer within available resources or
constraints. INDIVIDUAL ACCOUNTABILITY One
member from your group may be randomly chosen to
explain (a) the answer and (b) how to solve each
problem. EXPECTED BEHAVIORS Active
participating, checking, encouraging, and
elaborating by all members. INTERGROUP
COOPERATION Whenever it is helpful, check
procedures, answers, and strategies with another
group.
31
Technical Estimation Exercise TASK
INDIVIDUAL Quick Estimate (10 seconds). Note
strategy. COOPERATIVE Improved Estimate (5
minutes). One set of answers from the group,
strive for agreement, make sure everyone is able
to explain the strategies used to arrive at the
improved estimate. EXPECTED CRITERIA FOR
SUCCESS Everyone must be able to explain the
strategies used to arrive at your improved
estimate. EVALUATION Best answer within
available resources or constraints. INDIVIDUAL
ACCOUNTABILITY One member from your group may
be randomly chosen to explain (a) your estimate
and (b) how you arrived at it. EXPECTED
BEHAVIORS Active participating, checking,
encouraging, and elaborating by all
members. INTERGROUP COOPERATION Whenever it is
helpful, check procedures, answers, and
strategies with another group.
32
Model 1 (lower bound) let L be the length of
the room, let W be its width, let H be its
height, and let D be the diameter of a ping
pong ball. Then the volume of the room is
Vroom L W H, and the volume of
a ball (treating it as a cube) is
Vball D3, so number of balls (Vroom) /
(Vball) (L W H) / (D3).
33
Model 2 (upper bound) let L be the length of
the room, let W be its width, let H be its
height, and let D be the diameter of a ping
pong ball. Then the volume of the room is
Vroom L W H, and the volume of
a ball (treating it as a sphere) is
Vball 4/3 Br3, so number of balls
(Vroom) / (Vball) (L W H) / (4/3 Br3).
34
Model 1 (Vroom / D3ball) B Lower Bound Model 2
(Vroom / (4/3 Br3ball)) B Upper Bound Upper
Bound/Lower Bound 6/B . 2 How does this ratio
compare with 1.The estimation of the diameter of
the ball? 2.The estimation of the dimensions of
the room?
35
Model World
Real World
Model
Vr/Vb
Calc
36
(No Transcript)
37
Problem-Based Learning
START
Apply it
Problem posed
Normative Professional Curriculum 1. Teach the
relevant basic science, 2. Teach the relevant
applied science, and 3. Allow for a practicum
to connect the science to actual practice.
Learn it
Identify what we need to know
Subject-Based Learning
START
Given problem to illustrate how to use it
Told what we need to know
Learn it
38

• Problem-Based Learning (PBL)
• Problem-based learning is the learning that
  results from the process of working toward the
  understanding or resolution of a problem. The
  problem is encountered first in the learning
  process B Barrows and Tamlyn, 1980
• Core Features of PBL
• Learning is student-centered
• Learning occurs in small student groups
• Teachers are facilitators or guides
• Problems are the organizing focus and stimulus
  for learning
• Problems are the vehicle for the development of
  clinical problem-solving skills
• New information is acquired through self-directed
  learning

39
Group Processing B Plus/Delta Format B
Delta Things Group Could Improve
Plus Things That Group Did Well
40
Cooperative Learning is instruction that involves
people working in teams to accomplish a common
goal, under conditions that involve both positive
interdependence (all members must cooperate to
complete the task) and individual and group
accountability (each member is accountable for
the complete final outcome). Key
Concepts Positive Interdependence Individual and
Group Accountability Face-to-Face Promotive
Interaction Teamwork Skills Group Processing
41
Cooperative Learning Research Support Johnson,
D.W., Johnson, R.T., Smith, K.A. 1998.
Cooperative learning returns to college What
evidence is there that it works? Change, 30 (4),
26-35. Over 300 Experimental Studies First
study conducted in 1924 High Generalizability
Multiple Outcomes
Outcomes 1. Achievement and retention 2.
Critical thinking and higher-level reasoning 3.
Differentiated views of others 4. Accurate
understanding of others' perspectives 5. Liking
for classmates and teacher 6. Liking for subject
areas 7. Teamwork skills
42
Small-Group Learning Meta-analysis
Springer, L., Stanne, M. E., Donovan, S. 1999.
Effects of small-group learning on
undergraduates in science, mathematics,
engineering, and technology A meta-analysis.
Review of Educational Research, 69(1), 21-52.
Small-group (predominantly cooperative) learning
in postsecondary science, mathematics,
engineering, and technology (SMET). 383 reports
from 1980 or later, 39 of which met the rigorous
inclusion criteria for meta-analysis. The main
effect of small-group learning on achievement,
persistence, and attitudes among undergraduates
in SMET was significant and positive. Mean
effect sizes for achievement, persistence, and
attitudes were 0.51, 0.46, and 0.55,
respectively.
43
Creating High-Quality Learning Environments
Guidelines from Research on How People Learn
Understanding by Design Wiggins McTighe
Backward Design Stage 1.Identify Desired
Results Stage 2.Determine Acceptable
Evidence Stage 3.Plan Learning Experiences and
Instruction
Wiggins, G. McTighe, J. 1998. Understanding
by design. ASCD.
44
Backward Design

• Stage 1. Identify Desired Results
• Filter 1. To what extent does the idea,
  topic, or
• process represent a big idea or
  having
• enduring value beyond the
  classroom?
• Filter 2. To what extent does the idea,
  topic, or
• process reside at the heart of
  the discipline?
• Filter 3. To what extent does the idea,
  topic, or
• process require uncoverage?
• Filter 4. To what extent does the idea,
  topic, or
• process offer potential for
  engaging
• students?

45
Backward Design

• Stage 2. Determine Acceptable Evidence
• Types of Assessment
• Quiz and Test Items
• Simple, content-focused test items
• Academic Prompts
• Open-ended questions or problems that
• require the student to think critically
• Performance Tasks or Projects
• Complex challenges that mirror the
  issues or
• problems faced by graduates, they are
  authentic

46
Backward Design

• Stage 3. Plan Learning Experiences Instruction
• What enabling knowledge (facts, concepts, and
  principles) and skills (procedures) will students
  need to perform effectively and achieve desired
  results?
• What activities will equip students with the
  needed knowledge and skills?
• What will need to be taught and coached, and how
  should it be taught, in light of performance
  goals?
• What materials and resources are best suited to
  accomplish these goals?
• Is the overall design coherent and effective?

47
It could well be that faculty members of the
twenty-first century college or university will
find it necessary to set aside their roles as
teachers and instead become designers of learning
experiences, processes, and environments James
Duderstadt, 1999
48
We never educate directly, but indirectly by
means of the environment. Whether we permit
chance environments to do the work, or whether we
design environments for the purpose makes a great
difference. John Dewey, 1906
49
CAEE Vision for Engineering Education
 Center for the Advancement of Engineering
Education Cindy Atman, Director
50
CAEE Team

• University of Washington
• Colorado School of Mines
• Howard University
• Stanford University
• University of Minnesota
• CAEE Affiliate Organizations
• City College of New York (CCNY), Edmonds
  Community College, Highline Community College
  (HCC), National Action Council for Minorities in
  Engineering (NACME), North Carolina AT (NCAT),
  San Jose State University (SJSU), University of
  Texas, El Paso (UTEP), Women in Engineering
  Programs Advocates Network (WEPAN) and Xavier
  University

51
CAEE - Elements for Success

• Scholarship on Learning Engineering
  Learn about the engineering student experience
• Scholarship on Engineering Teaching Help
  faculty improve student learning
• Scholarship on Engineering Education Institutes
  Cultivate
  future leaders in engineering education

52
CAEE Approach
Theory
Research that makes a difference . . . in theory
and practice
Research
Practice
53
Center for the Integration of Research, Teaching,
and Learning (CIRTL) NSF Center for Learning
and Teaching University of Wisconsin -
Madison Michigan State University Pennsylvania
State University
54
develop a national STEM faculty ...
UNDERGRADS Community College Liberal
Arts HBCU Masters University Comprehensive
Univ. Research University
FACULTY Community College Liberal
Arts HBCU Masters University Comprehensive
Univ. Research University
100 RUs gt 80 Ph.Ds
55
Teaching-as-Research
The nation must develop STEM faculties who
themselves continuously inquire into their
students learning.

• Engagement in teaching as engagement in STEM
  research
• Hypothesize, experiment, observe, analyze,
  improve

• Aligns with skills and inclinations of
  graduates-
• through-faculty, and fosters engagement
  in
• teaching reform

• Leads to self-sustained improvement of STEM
  education

56
A Work-in-Progress NAE Center for the
Advancement of Scholarship on Engineering
Education

• Norman L. Fortenberry, Sc.D.
• Director, CASEE
• http//www.nae.edu/CASEE
• nfortenb_at_nae.edu
• [phone]

November 8, 2003
57
CASEE Mission

• Enable engineering education to meet, in a
  significantly better way, the needs of employers,
  educators, students, and society at large.

CASEE Objectives

• Working collaboratively with key stakeholders,
  CASEE
• Encourages rigorous research on all elements of
  the engineering education system, and
• Seeks broad dissemination, adoption, and use of
  research findings.

58
Research Thrust Areas

• 1. Define the bodies-of-knowledge required for
  engineering practice and use of engineering study
  for other careers.
• 2. Develop strategies that value diversity in the
  formulation and solution of engineering problems.
• 3. Develop cost-effective and time-efficient
  strategies and technologies for
• Improving student learning, and
• Enhancing the instructional effectiveness of
  current and future faculty.
• 4. Develop assessments of student learning and
  instructional effectiveness.

59
Conducting Rigorous Research in Engineering
Education Creating a Community of Practice

• NSF-CCLI-ND
• American Society for Engineering Education
• Karl Smith Ruth Streveler
• University of Minnesota
• Colorado School of Mines

60
Rigorous Research Workshop

• Initial Event for year-long project
• Presenters and evaluators representing
• American Society for Engineering Education (ASEE)
• American Educational Research Association (AERA)
• Professional and Organizational Development
  Network in Higher Education (POD)
• Faculty funded by two NSF projects
• Conducting Rigorous Research in Engineering
  Education (NSF DUE-0341127)
• Strengthening HBCU Engineering Education Research
  Capacity (NSF HRDF-041194)
• Council of HBCU Engineering Deans
• Center for the Advancement of Scholarship in
  Engineering Education (CASEE)
• National Academy of Engineering (NAE)