Introducing Principled Assessment Design For Inquiry PADI

1
Introducing Principled Assessment Design For
Inquiry (PADI)

• Daisy Wise, University of Maryland
• Futoshi Yumoto, University of Maryland, Abt
  Associates Inc.

Supported by NSF, via IERI Partners SRI
International University of Maryland Universit
y of California at Berkeley (BEAR
Center) Lawrence Hall of Science
(FOSS) University of Michigan (BioKIDS)
Information http//padi.sri.com
Acknowledgements This material is based on the
work supported by the National Science Foundation
under grant REC-0129331 (PADI Implementation
Grant). Disclaimer Any opinions, findings, and
conclusions or recommendations expressed in this
material are those of the authors and do not
necessarily reflect the views of the National
Science Foundation.
2
The Problem

• Creating good inquiry tasks is hard.
• How to do it validly and efficiently?
• Whats the connection with standards?
• Whats the connection with the learning
  psychology?
• How to implement within given content, for given
  kinds ages of students?
• How do you score it?
• How to sort out confounded evidence?
• How to leverage technology?

3
The PADI Solution

• Conceptual frameworks representations
• Evidence-centered design approach
• Object model for designing assessment arguments
  and assessment tasks.
• Allows for the collaboration of individuals with
  expertise from different domains.
• Uses a common language and representations
• Software tools to aid processes of
• Design
• Implementation
• Delivery

4
The PADI Solution (2)

• PADI focus is on ...
• Design framework (Design patterns, templates,
  delivery system)
• Assessing inquiry in science
• PADI focus is not on ...
• Tasks, per se (although there are examples,
  through FOSS, BioKIDS, other exemplars)
• Authoring systems
• Delivery systems (exception scoring engine
  module)

5
Evidence-Centered Design

• The assessment problem To draw inferences about
  student proficiencies based on limited
  observations.
• Evidence-Centered Design Defines elements of the
  assessment argument and relationships between
  them.
• What do we want to know about students?
• What constitutes evidence of knowing?
• How can that evidence be elicited from students?
• What are the appropriate statistical techniques
  for making valid student inferences about what
  students know from what students do?

6
The PADI Solution

• Evidence-Centered Design Layers
• Mislevy, R., Riconscent, M. (2005).
  Evidence-centered assessment design Layers
  structures and terminology. (PADI Technical
  Report 9). Menlo Park, CA SRI International.
• Downing S.Mgt, Haladyna, T. M. (2006). Handbook
  of test development. Mahwaj, NJ. Erlbaum.

7
The PADI Solution
PADI Design Patterns Narrative structures for
the design of an assessment of pervasive kinds of
knowledge / skill / capabilities, such as
model-based reasoning, inquiry cycles, building
scientific explanations.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

8
The PADI Solution
PADI Templates Task Specs Technical
structures for specifying elements such as what
skills are being measured, the tools used for the
assessment, and the statistical analysis
procedures, and their interaction.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

9
Main Components of PADI
10
Assessment Objects in the PADI Design System
11
GLOBE Program

• Global Learning and Observations to Benefit the
  Environment (GLOBE) is a worldwide hands-on,
  elementary and secondary school-based education
  and science program (www.globe.gov).
• Investigation assessments developed for GLOBE in
  a previous NSF grant address multiple, integrated
  inquiry skills within Earth Science topics
  (globeassessment.sri.com).

12
Task Model Variables in GLOBE

13
Student Model in GLOBE
14
Evidence Model in GLOBE

15
PADI Tools Available

PADI Task Specifications
PADI Design System
Design
PADI Design Wizards
Instantiated Tasks
Assessment Authoring System
Implementation
PADI Calibration Engine
Design Team
Student Database
PADI Scoring Engine
Assessment Delivery System

Students
PADI Gradebook
15
16
Conclusion

• The PADI system is
• A collaborative environment for creating coherent
  assessments.
• Interactive and web-based
• Flexible to allow for many different assessment
  types
• A way for increasing conceptual understanding of
  the construct being measured.