Assessment of Learning in Informal Environments

1
Assessment of Learning in Informal Environments
2
Assessment What do people know?

• Do they know enough?
• What have they learned?
• Can we help them learn more?

3
Assessment v. Evaluation

• Evaluation to study quality of the learning
  environment
• Is it meeting our goals?
• How can we make it better?
• Should we keep funding?
• Is it something we can scale?

4
Assessment as Part of Evaluation

• Program is good because theres learning
• On average across participants
• Current climate summative judgment about
  whether programs work

5
Two big questions

• Q1 What do we assess? (What is being learned?)
• Q2 How do we assess it? (What would be good
  evidence?)

6
Q1 Whats being learned?

• A broad of skills, knowledge, attitudes,
  dispositions, feelings, and behaviors
• Often can be unanticipated
• Can become evident at different points in time
• Can occur at different scales

7
Q2 Whats our evidence?

• Need to know what the outcome is that we are
  targeting.
• Need to match outcome with learning opportunity.
• Needs to account for the norms of the setting.

8
Major challenges?

• Learning opportunities (and outcomes)
• Depend upon participants purposes, experiences,
  and choices
• Contingent, emerging in interaction
• Designed to support a life-long, life-wide,
  life-deep cultural infrastructure for learning
• Norms leisure, play, and sociality

9
Still, lots of work gets done

• Positive result related to all six strands
• Full range of methods
• Interviews, naturalistic observations,
  conversation analyses, video, think-alouds, pre-
  and posttesting, journaling, sorting tasks
• Open-ended, non-threatening content,
  conversational, comfortable

10
Strand 1 Developing Interest in Science

• Science experiences increase interest (BSET,
  2004 DeHaven Wiest, 2003 Gibson Chase,
  2002 Tisdal, 2004)
• Increased interest promotes learning (Barron,
  2006 Engle Conant, 2002 Flynn et al., 206
  Sachatello-Sawyer, 2006)
• Interest promotes long-term outcomes (Tai et al.,
  2006)

11
Strand 2 Understanding Science Knowledge

• Early knowledge of natural world comes from
  everyday life (Ross et al. 2003 Saxe et al.,
  2007)
• Informal experiences can potentially serve as
  productive resources for enduring understanding
  (Johnson, 2005 Gelman, 2003 Korn, 2006)
• Knowledge is sought and systematized based on
  interest and need (Bell et al., 2006 Sandoval,
  2005)

12
Strand 3 Engaging in Scientific Reasoning

• Everyday reasoning entails thinking about
  causality and complex relations among observed
  variables. (Gopnik et al., 2004 Tschirgi, 1980)
• Reasoning and inquiry are supported by
  interactivity in designed spaces. (Falk, et al.,
  2004 Korn, 2004 Randoll, 2005)
• Observation, interpretation, mean-making, and
  questioning affected by design. (Allen, 2002
  Callanan Jipson, 2001 Humphrey Gutwill,
  2005)

13
Strand 4 Reflecting on Science

• Children and adults struggle to understanding
  science as a dynamic process. (Osborne et al.,
  2003 Sandoval, 2005)
• Designed settings and youth programs have helped
  participants examine the nature of science.
  (Pedretti, 2004 Randol, 2005)
• Reflections on own learning are readily cued and
  may be particularly accessible in everyday
  settings. (Jones, 2005 Sandoval, 2005)

14
Strand 5 Engaging in Scientific Practices

• Feature of program-based and designed settings,
  with emphasis on discursive practice and hands-on
  activities. (Korn, 2004 Jones, 2003 Salz et
  al., 2004)
• Parent-child interactions support extended
  conversations about evidence and other features
  of exhibits. (Callanan et al., 2002 Crowley
  Galco, 2001)

15
Strand 6 Identifying with the Scientific
Enterprise

• When deeply embedded in community issues, science
  programs can support sustained interest and
  identification. (Zacharia Calabrese Barton,
  2003)
• Designed and everyday spaces, learners
  personalize and integrate learning experiences
  with values and identity. (Fadigan Hammrich,
  2004 Gupta Siegal, 2008)

16
Conclusion 13

• Currently there are not good outcome measures for
  assessing the science learning goals of informal
  settings. Conventional academic achievement
  measures (e.g., standardized tests of science
  achievement) are too narrow and not well aligned
  to the goals of informal providers.

17
Recommendation 7

• Researchers and evaluators should use assessment
  methods that do not violate participants
  expectations about learning in informal settings.
  Methods should address the science strands,
  provide valid evidence across topics and venues,
  and be designed in ways that allow educators and
  learners alike to reflect on the learning taking
  place in these environments.

18
Core guidelines

• Stay open to broad range of possible outcomes
  from the experience
• Take into account the norms and nature of the
  informal experience for participants
• Make sure what we try to measure is true to
  learners experience and our intent