Valid, Reliable - PowerPoint PPT Presentation

1 / 25
About This Presentation
Title:

Valid, Reliable

Description:

Need for valid, reliable, efficient instruments to determine instructional ... 2 = Title 1 1 = public, 1 = parochial. 2 = non-Title 1 both = public & mixed SES ... – PowerPoint PPT presentation

Number of Views:70
Avg rating:3.0/5.0
Slides: 26
Provided by: kathlee85
Category:

less

Transcript and Presenter's Notes

Title: Valid, Reliable


1
Valid, Reliable Efficient
  • A Psychometric Evaluation of
  • Flash Word Recognition and
  • NSSI Passage Reading Measures

2
Kathleen J. BrownMatthew K. FieldsUniversity of
UtahR. Darrell Morris Appalachian State
University
3
Impetus for Current Study
  • Need for valid, reliable, efficient instruments
    to determine instructional reading level
  • Flaws with current instruments
  • DRA no rate time consuming
  • DIBELS screen only
  • IRIs psychometric evaluation often weak or
    missing

4
Impetus for Current Study
  • Growing use of Flash selected graded passages
    a.k.a. NSSI (Virginia/ASU effect)
  • Initial psychometric evaluations positive
  • (Frye, 2004 Frye Trathen, 2004 Frye Trathen,
    Olson, Schlagal, 2002 Palmer, Trathen, Olson,
    Schlagal, 2002)

5
Theoretical Framework
(Anastasi, 1988 APA, 1985)
6
Methods
  • 4 schools
  • 2 Title 1 1 public, 1 parochial
  • 2 non-Title 1 both public mixed SES
  • 192 students in G2-G5 in March, 2006
  • Rank ordered DIBELS or QRI, then sampled 12
    students per grade 4 high, 4 average, 4 poor to
    achieve a representative distribution for testing

7
Methods
  • 135 minutes of assessment in 3 sessions
  • Presentation order counterbalanced
  • Flash item selection counterbalanced
  • 9 on data team 4 hours protocol training
  • Manual flash interrater differences n.s.

8
Alternate Form Reliability
  • measure of temporal stability for scores
  • measure of consistency of response for scores

9
Alternate Form Reliability
  • To what extent are NSSI A passage scores
    equivalent to NSSI B passage scores?
  • To what extent are computer Flash scores
    equivalent to manual Flash scores?

10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Results Alternate Form Reliability
p lt .01
14
Content Validity
  • provides evidence that items on test represent a
    specific domain
  • provides evidence that the format and response
    properties of the test represent the domain

15
Content Validity
  • To what extent do the NSSI passages
    reflect/measure expected grade level benchmarks?
  • Maybe look at separate means for accuracy, rate,
    comp report those to show
  • To what extent does the Flash measure reading
    instructional level?

16
NSSI Reading Level Criteria
17
Performance Levelfor NSSI by Grade
18
Performance Levelfor Flash by Format Criterion
19
Concurrent Validity
  • To what extent are Flash scores and NSSI scores
    consistent with scores achieved on a flagship
    standardized reading measure (i.e., the GORT)?

20
(No Transcript)
21
Results Concurrent Validity
p lt .01
22
Average Performance Levelfor NSSI GORT by
Grade Level
23
Conclusions For G2-G5
  • NSSI A and NSSI B seem to have high validity for
    identifying students instructional reading
    levels
  • NSSI A and NSSI B can be considered equivalent
    forms

24
Conclusions For G2-G5
  • Manual Flash and Computer Flash seem to have high
    validity for identifying students instructional
    reading levels when the criterion is set at 85
  • Manual Flash and Computer Flash seem to be
    equivalent forms

25
Conclusions For G2-G5
  • The GORT does not seem to have high validity for
    identifying students instructional levelsat any
    grade level.
  • The GORT over-predicts instructional levelby
    approx. 2 years.
  • Note most GORT comp questions are passage
    independent (Keenan Betjemann, 2006)
Write a Comment
User Comments (0)
About PowerShow.com