The Primary Grade Reading

1
The Primary Grade Reading Writing Teacher
Knowledge Project

• Developing the Literacy Instruction Knowledge
  Scales (LIKS) Assessment System
• International Reading Association
• Atlanta, Georgia
• May 7, 2008

2
The Primary Grade Reading Writing Teacher
Knowledge Project

• A four-year 1 million dollar research grant
  funded by the Institute of Education Sciences
  (IES) of the U.S. Department of Education.

3
The Primary Grade Reading Writing Teacher
Knowledge Project

• Project researchers
• Ray Reutzel, USU Janice Dole, U of U Richard
  Sudweeks, BYU Parker Fawson, USU John Smith,
  USU Sylvia Reed, USU Jamison Fargo, USU
  Rebecca Donaldson, USU Kerry Herman, U of U
  Cindy Jones, USU and Chris Wiggins, U of U.
• USU BYU U of U

4
The Primary Grade Reading Writing Teacher
Knowledge Project

• Major Purposes of the Project
• Develop a comprehensive assessment of grade 1-3
  teacher knowledge about reading and writing
  instruction.
• Conduct a validity study of this comprehensive
  teacher knowledge assessment to predict grade 1-3
  student reading and writing achievement.

5
The Primary Grade Reading Writing Teacher
Knowledge Project

• General Teacher Knowledge Theoretical Frame
• Began with Shulmans Theoretical Framework of
  Teacher Knowledge
• Most well-known constructs are
• Content knowledge
• Pedagogical content knowledge

6
The Primary Grade Reading Writing Teacher
Knowledge Project

• Reading /Writing Teacher Knowledge Theoretical
  Frame
• Evidence-Based Research
• Research-based strategies for instructing reading
  and writing effectively.
• Jobs Analysis Approach
• What does it take to do this job?
• Inert vs. Enacted Teacher Knowledge
• Knowledge held but not put to use in classroom
  instruction vs. knowledge that is put to use in
  classroom instruction.

7
The Primary Grade Reading Writing Teacher
Knowledge Project

• Indirect Knowledge Measures
• Teacher perceptions of their knowledge about
  effective reading/writing instruction
• Teacher beliefs about effective instruction
  practices
• Teacher demographics, i.e. number of courses
  taken, degrees, licenses, etc.

8
The Primary Grade Reading Writing Teacher
Knowledge Project

• Direct Knowledge Measures
• Written Knowledge Tests or Written Knowledge
  Surveys of Teaching Reading or Writing
• Content Knowledge
• Pedagogical Knowledge
• Content Pedagogical Knowledge
• Standards/Curriculum
• Literature, Programs, Materials
• Assessment

9
The Primary Grade Reading Writing Teacher
Knowledge Project

• Direct Observation of Enacted Teacher Knowledge
• Job analysis What does it take to do this job?
• What effective instructional practices would we
  expect to see every day in primary-grade
  classrooms based on research evidence that would
  impact student achievement?

10
The Primary Grade Reading Writing Teacher
Knowledge Project

• Direct Observation of Enacted Teacher Knowledge
• How can the quantity and quality of the
  instruction provided be measured?
• Case or scenario based written questions
• Scaled classroom observations
• Teacher artifacts, e.g., lesson plan content
  analysis
• Teacher rating of self or other teachers video
  recorded lessons

11
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence-Based Knowledge Taxonomy Development
• Literature Review
• Reviewed over 3,500 research articles and
  research handbook chapters published in the past
  20 years.
• Developed a Taxonomy of Teacher Knowledge for
  Reading and Writing

12
Teacher Knowledge Taxonomy for Reading

• Learning, Teaching Theory, and Models
• Learning and Teaching
• Classroom Environment
• General Reading Instructional Practices
• Reading Instructional Materials, Curricula, and
  Programs
• Texts
• Language and Oral Language
• Concepts About Print
• Phonological and Phonemic

• Decoding
• Spelling
• Fluency
• Vocabulary
• Comprehension
• Diversity
• Motivation and Engagement
• Technology and Reading
• Reading Assessment
• Family-School Connections
• Teacher Self-Awareness

13
Teacher Knowledge Taxonomy for Writing

• Learning, Teaching Theory, and Models
• Learning and Teaching Philosophies
• Classroom Environment
• General Writing Instructional Practices
• Modes and Characteristics of Effective Writing
  Instruction
• Writing Curriculum, Standards, and Programs
• Rhetoric and Texts
• Stages of Writing Development

• Concepts About Writing
• Writing Process
• Spelling
• Handwriting
• Grammar
• Composition
• Diversity and Writing
• Writing Motivation and Engagement
• Technology and Writing
• Writing Assessment
• Family-School Connections
• Teacher Self-Awareness

14
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence-Based Knowledge Taxonomy Development
• Classroom Observations Interviews

15
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence-Based Knowledge Taxonomy Development
• Focus Group Meetings
• Enumerative listing task
• Included-term structure questions

16
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence-Based Knowledge Taxonomy Development
• External Expert Review of Knowledge Taxonomy
• Data Audit Trail Verification

17
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence Based Knowledge Instrument
  Development--the Literacy Instruction Knowledge
  Scales (LIKS)
• Two Subscales of the LIKS
• Written Knowledge Survey --paper and pencil
  test, multiple-choice (inert and enacted
  knowledge)
• Classroom Teaching Observation --observation
  scale (enacted knowledge)

18
The Primary Grade Reading Writing Teacher
Knowledge Project

• Domain Specification--we determined three teacher
  knowledge domains for reading and writing
• Decoding and Fluency Instruction
• Comprehension and Vocabulary Instruction
• Writing Instruction

19
The Primary Grade Reading Writing Teacher
Knowledge Project

• Iterative cycle of development
• After the Classroom Teaching Observation LIKS
  subscale was written, researchers returned to
  written knowledge survey to match with classroom
  observation rubrics (map inert practice onto
  enacted practice) and vice versa.
• e.g. if teachers know that background knowledge
  affects comprehension, then do they activate and
  build background knowledge before their students
  read a lesson?

20
Literacy Instruction Knowledge Scale (LIKS)

• Written Survey Subscale of Grades 1-3 Teacher
  Knowledge of Evidence-based Reading and Writing
  Instruction

21
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Knowledge Survey
• Item Sampling Table

22
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Writing Specifications
• Issues to consider when creating questions
• Have four options when possible. Quality of
  distracters is more important than quantity. Not
  necessary for every item to have the same number
  of answer options.
• Avoid first-person and second-person when writing
  test items.
• Consider using an alternative response format.
  This is a multipart question where the respondent
  marks yes or no for each item independently.
• We should try to have more content-dependent item
  sets so that the work test-takers have done to
  read an item can be applied to more than one
  question. Get mileage out of scenarios.

23
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Writing Specifications Style Guide
• Commas in a series put a comma before final
  and i.e. cat, big, and dug.
• Always spell out and.
• Leave out of the following in the question
  sentence at the end of the stem.
• When we have an incomplete sentence as the stem,
  we want to be sure that each option completes the
  sentence. Each option should have a period.
• When a word needs to be emphasized, underline it.
  
• Comparatives and superlatives should be
  bold-faced (e.g. best, most, least)
• On incomplete stems, end with ellipses.

24
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Writing Specifications Style Guide
• Use the term student or students not child
  or children.
• Words in item stems like NOT and EXCEPT should be
  all caps.
• When writing a context dependent item set
  (testlet), there should be directions to the
  test-taker (Questions 3-9 refer to the following
  writing sample)
• Items should have a maximum of 5 options.
• Samples of text should be boxed.

25
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Writing Specifications Style Guide
• Option designators should be all caps followed by
  a period, as shown below.
• Test writers are dumber than
• A bag of hammers
• A box of rocks
• Soap
• Most
• Use forward slashes around linguistic elements
  such as /sl/.

26
The Primary Grade Reading Writing Teacher
Knowledge Project

• Inert Knowledge Question
• 1. Which word ends with a consonant blend?
  
• A. fast
• B. hitch
• C. with
• D. wish

27
The Primary Grade Reading Writing Teacher
Knowledge Project

• Enacted Knowledge Question
• 1. Mrs. Payne and her first-grade students are
  reading a story entitled The Bear from their
  basal reader. In order to teach her students to
  monitor their comprehension Mrs. Payne should
  instruct her students to
• A. stop reading and clarify when they do not
  understand.
• B. ask her a question when they do not
  understand.
• C. retell the story after every few pages.
• D. ask themselves a combination of literal and
  inferential questions as they read.

28
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Knowledge Subscale Determining
  Correct Answers for Items
• Converging on the correct answer using the
  evidence-base
• Cognitive Laboratories with Reading Coaches and
  Grade 1-3 Reading Teachers
• Revisions, Revisions, Revisions
• Tryouts of items with 1300 grade 1-3 teachers in
  randomly sampled elementary schools in 9 Utah
  school districts

29
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Teacher Knowledge Subscale
  Reliability Study 1 Results
• Cronbachs Alpha and a one parameter Rasch IRT
  was .816 for the initial150 written items in the
  item pool for the 1st try out study.

30
The Primary Grade Reading Writing Teacher
Knowledge Project

• Inert Knowledge Items Results of Study 1
  Tryout

31
The Primary Grade Reading Writing Teacher
Knowledge Project

• Enacted Teacher Knowledge Items Results of
  Study 1 Tryout

32
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Distracter Analyses Good Item
• 20. Which is a distinguishing characteristic of
  phonemic awareness instruction?
• A. uses printed letters
• B. uses two cueing systems
• C. does not use printed letters
• D. links meaning to sound

33
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Distracter Table Analyses Good Item
• Inter Item Correlation - .43
• Difficulty Index 36
• Discrimination Index 58

34
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Distracter Analyses Bad Item
• 22. According to research, the least effective
  way to teach vocabulary words to students
• is through the use of
• A. definitions.
• B. context clues.
• C. examples.
• D. synonyms.

35
The Primary Grade Reading Writing Teacher
Knowledge Project

• Item Distracter Table Analyses Bad Item

36
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Teacher Knowledge Subscale
• Enacted Knowledge Decoding
• 1. Mrs. Funke is teaching her students to
  identify multi-syllable words. Which is an
  appropriate first step for her to do?
• A. model analyzing words for familiar prefixes
  and suffixes
• B. show students how to blend individual
  letter-sounds, left-to-right
• C. model how to look for little words in big
  words
• D. demonstrate sequentially blending onsets and
  rimes
• Item Inter-correlation - .35 Difficulty Index
  42 Discrimination Index - .50

37
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Teacher Knowledge Subscale
• Enacted Knowledge Comprehension
• 1. Mr. Foxs students say they understand their
  science textbooks even though they are unable to
  correctly answer questions about the content.
  What comprehension strategy would best help his
  students to realize they may not understand the
  content as they read?
• A. monitoring and fix-up strategies
  
• B. making mental pictures of the text
• C. activating their background knowledge
• D. answering questions at the end of the chapter
• Item Inter-correlation - .36 Difficulty Index
  53 Discrimination Index 52

38
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Teacher Knowledge Subscale
• Enacted Knowledge - Writing
• Students in a third grade class are writing
  narratives that use sequence. Printed below is
  an excerpt from JaNells draft.
• My family went to Disneland. First we rode
  space moutun and then we went to Cindrellas
  cassle and shook hands with pluto and mickey. you
  can have your pikchr taken with them. They were
  nice. Mom let me have my pikchur taken, with
  mickey. We ate lunch after that. We got to buy
  some suvaneers. Before, we went to the hotel. We
  also went on the Indeana jones ride.

39
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Written Teacher Knowledge Subscale
• Enacted Knowledge - Writing
• 1. What would experts suggest JaNell be taught
  about revision?
• A. proper nouns should be capitalized
• B. how to reorder the text
• C. how to fix a sentence fragment
• D. how to use apostrophes and commas
• Item Inter-correlation - .31 Difficulty Index
  33 Discrimination Index 48

40
The Primary Grade Reading Writing Teacher
Knowledge Project

• Written Teacher Subscale Knowledge Try Out
  Results from Reliability Studies 2 3
• Number of items were reduced from 150 to 103.
• Cronbachs Alphas were
• Reliability Study 2 Decoding - .73
  Comprehension 73 and Writing - .59
• Reliability Study 3 Decoding .70
  Comprehension - .74 and Writing - .58

41
Literacy Instruction Knowledge Scale (LIKS)

• Classroom Observation Subscale of Grades 1-3
  Teacher Knowledge of Evidence-based Reading and
  Writing Instruction

42
The Primary Grade Reading Writing Teacher
Knowledge Project

• Evidence Based Classroom Observation LIKS
  Subscale
• Evidence-Based Practices to be Observed in
  Reading and Writing Instruction in classrooms
• Scaling of the Instrument--1-7 determines both
  quantity and quality of instruction
• Writing Scaling Descriptors--jointly developed
• Pilot Try Outs -- gr. 1-3 classrooms and DVD
  recordings of teacher language arts instructional
  blocks.

43
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
44
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
45
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
46
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
47
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
48
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
49
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
50
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
51
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
52
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
53
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
54
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
55
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
56
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
57
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
58
The Primary Grade Reading Writing Teacher
Knowledge Project
LIKS Classroom Observation Subscale
59
The Primary Grade Reading Writing Teacher
Knowledge Project
Example of Using LIKS Classroom Observation
Subscale
60
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Classroom Observation Subscale Try Out
• Study 1 Results
• Phi coefficients represent the reliability
  coefficients of two raters on two occasions
  rating teachers decoding, comprehension, and
  writing instruction using the LIKS Classroom
  Observation Subscale were as follows
• Decoding -.86
• Comprehension -.75
• Writing- .82

61
The Primary Grade Reading Writing Teacher
Knowledge Project

• Summary, Conclusions and Future Directions
• Beginning year 4 of the LIKS project fall 2009.
• Written Knowledge Survey 4th reliability study
  completed by May 31, 2008.
• Each domain subtest, decoding, comprehension, and
  writing, should be approach or exceed a .80
  reliability the target.

62
The Primary Grade Reading Writing Teacher
Knowledge Project

• LIKS Classroom Observation Scale--has been
  revised multiple times in connection with changes
  in the LIKS Written Teacher Knowledge Subscale.
• LIKS Classroom Observation Scale
• 2 Dependability and Generalizability studies have
  been conducted. The results of one are available
  now. Within three weeks of this presentation the
  data from the second study will be available.

63
The Primary Grade Reading Writing Teacher
Knowledge Project

• Final Study Conducted Next Year
• Predictive validity study of LIKS (both
  subscales) with grade 1-3 student achievement in
  reading and writing.
• Teacher Inert Knowledge gt Teacher Enacted
  Knowledge gt Student Achievement

64
The Primary Grade Reading Writing Teacher
Knowledge Project

• Unique Aspects of This Project
• Explicit theoretical framework for the study.
• Exhaustive knowledge domain specification
• Exhaustive review of research to develop a
  taxonomy of reading and writing topics.
• Cross-checking of topics with experienced
  teachers and reading experts.

65
The Primary Grade Reading Writing Teacher
Knowledge Project

• Unique Aspects of This Project
• Notion of inert vs. enacted teacher knowledge
  measurement.
• Measurement of teacher classroom instructional
  practices in addition to assessing teacher
  knowledge using a written survey or test made-up
  of multiple-choice items.
• A study of teacher knowledge used to predict
  student reading and writing achievement scores.

66
The Primary Grade Reading Writing Teacher
Knowledge Project

• Emma Eccles Jones Center for Early Childhood
  Education
• Utah State University
• Logan, UT
• www.coe.usu.edu/ecc
• Click on Presentations and Publications Button