Unpacking the Field Test Year

1
Unpacking the Field Test Year

• Lorin W. Anderson
• Purveyor of Common Sense

2
(No Transcript)
3
The Structure of the Presentation

• Why do we field test new/revised courses?
• Why is RBT important?
• What are the major differences between RBT-based
  courses and previous courses?
• What is expected of teachers who are teaching an
  RBT-course during the field test year?
• How do I examine a Curriculum Guide?

4
WHY DO WE FIELD TEST?
I have learned the novice can often see things
that the expert overlooks. All that is necessary
is not to be afraid of making mistakes.    
Abraham Maslow
(1908 - 1970)
5

• The purpose of field testing is to verify that
  the system works in the actual user environment
  (or a representative set of locations). The
  emphasis is on verifying business processes,
  interfaces, connectivity, co-residency with other
  applications, and performance on the actual user
  hardware.
• Field testing is an optional phase, but is
  strongly recommended whenever significant changes
  have been made or when a new system is deployed.

www.bestpractices.cahwnet.gov
6

• Data gathered from field studies are fed back
  into the planning and design process and
  adjustments made accordingly. The result is a
  more complete design, more refined cost
  estimates, and a more accurate understanding of
  the viability of a proposed project.

www.twdb.state.tx.us
7

• What are the three major problems associated with
  field testing?
• Poor data are obtained.
• Good data are misinterpreted or ignored.
• The designers of the system/program become
  defensive.

8
WHY IS RBT IMPORTANT?

• Before we can answer this question, we need to
  answer a related question Why are objectives
  important?

9

• Would you tell me please, which way I ought to go
  from here?" asked Alice."That depends a great
  deal on where you want to go to," said the Cat.
  "I don't much care, where," said Alice. "Then
  it doesn't matter which way you go," said the
  Cat.

Lewis Carroll, Alice in Wonderland
10
SO, WHY IS RBT IMPORTANT?

• RBT provides a common framework (lens) for
  understanding objectives.
• RBT limits the verbs included in statements of
  objectives so that a common understanding of
  intended cognitive processes across courses and
  subjects is more likely.
• RBT provides a guide for enhancing and examining
  curriculum alignment.

11
Common Framework

• Unique frameworks Idiosyncratic understanding

Common frameworks Shared understanding
12
Reducing the Number of Verbs A Lesson from
Mathematics

• Add
• Analyze
• Apply
• Approximate
• Build
• Classify
• Collect
• Communicate
• Compare
• Compute

• Connect
• Consolidate
• Construct
• Contrast
• Convert
• Create
• Describe
• Determine
• Develop
• Display

13
A Few More

• Divide
• Draw
• Estimate
• Examine
• Explain
• Extend
• Evaluate
• Find
• Formulate
• Graph

• Identify
• Interpret
• Investigate
• Justify
• Label
• List
• Locate
• Make
• Model
• Monitor

14
And they keep on comin

• Multiply
• Order
• Organize
• Plot
• Produce
• Recognize
• Reflect
• Select
• Show
• Simplify

• Solve
• State
• Subtract
• Translate
• Understand
• Use
• Write

15
THE 25 Verbs

• Remember (Recognize, Recall)
• Understand (Interpret, Exemplify, Classify,
  Summarize, Infer, Compare, Explain)
• Apply (Execute, Implement)
• Analyze (Differentiate, Organize, Attribute)
• Evaluate (Check, Critique)
• Create (Generate, Plan, Produce)

16
Improved Alignment
Objectives
Assessment
Instructional Activities/ Materials
17
Why is alignment important?

• It increases students opportunity to learn (more
  time is spent on more important things)
• It increases the validity of assessments. With
  valid assessments, students can demonstrate they
  have, in fact, learned.
• It enables teachers to be (and be seen as) more
  successful. More effective teachers develop a
  sense of self-efficacy (a can do attitude)
  which has been found to be a central factor in
  continuing teacher effectiveness.

18
WHAT DIFFERENCES SHOULD I SEE?

• Course blueprints are shorter, generally from 1
  to 3 pages in length, with an average length of 2
  pages.
• All verbs are taken from the 6 cognitive process
  categories and 19 specific processes.
• The emphasis in most courses has shifted from
  memorization to understanding.

19
We say a student understands if he or she can

• Interpret (paraphrase or act out)
• Exemplify (find or give examples)
• Classify (place in categories)
• Summarize (often in 25 words or less)
• Infer (make predictions)
• Compare (similarities AND differences)
• Explain (why things happen)

Teachers should learn the meaning of each of
these!
20
WHAT DIFFERENCES? (CONTINUED)

• Each course contains a Unit Overview that
  provides teachers with useful information about
  the unit BEFORE they begin teaching it.
• Each course contains unpacked content rather
  than an outline of key terms and facts.

21
Content Outlines vs. Unpacked Content

• Words only
• Organized alphabetically
• Written in technical language

• Words and meanings
• Organized structurally
• Write in language of the students

Note. You will rarely if ever see the phrase
unpacked content in the Curriculum Guides.
Phrases such as Content Organizer are used
instead.
22
Examples of Language Differences

• Language of Students
• Demand is the desire to own something and the
  ability to pay for it.

• Technical Language
• Demand is the want or desire to possess a good or
  service with the necessary goods, services, or
  financial instruments needed to make a legal
  transaction for those goods or services.

23
More Differences The Need to Engage Students

• Most courses include essential questions
  (although they may call them different things.
  FACS, for example, calls them Learning
  Questions).
• All courses include a recommended sequence of
  activities along with a statement of the
  relevance of each activity to the objective.

24
TEACHING DURING THE FIELD TEST YEAR
25
Field Test Trade-Offs

• The only way to make sure the released and secure
  items are of equal difficulty, on average, is to
  field test ALL items and then divide the items
  into secure and released groups based on the
  results.
• As a consequence, there will be no released (or
  classroom) items available during the field
  test year. After the first year, however, you
  will have increased confidence in the
  comparability of the released and secure items in
  terms of their validity, reliability, and
  difficulty.

26
Requirements for Teachers

• Submit anticipated grades for all students
• Space is provided on each students answer sheet
• The anticipated grade for a student must be based
  exclusively on student performance relative to
  the course competencies and objectives. Behavior
  and attendance SHOULD NOT be considered in
  determining the anticipated grade.

27
Requirements for Teachers (contd)

• Complete on-line survey
• The purpose of the survey is to provide
  constructive feedback to the consultants and
  course design teams (i.e., what is really good
  what changes need to be made)
• Access to the survey will be through a hyperlink
  enclosed in an e-mail sent to each teacher.
• The survey is to be completed within 10 days
  prior to the end of the course (semester).

28
To complete the on-line survey teachers will need
to

• Keep notes as they teach the course (either using
  journals or writing on post-its that can be
  placed at appropriate places in the Curriculum
  Guides)
• Understand how to examine the Curriculum Guide

29
HOW DO I EXAMINE THE CURRICULUM GUIDE?

• 1. Pay attention to the objective, focusing on
  the verb and the RBT designation.
• 2. Pay attention to the connections among the
  objective, the unpacked content, the flow of
  activities, and the test items. When the
  connection is tight we say that the curriculum
  is ALIGNED.

30
Lets look at few examples!
31
6208 Business and Information Technology Education
3.01 Recall personal characteristics and skills
needed for a successful entrepreneur.
32
Unpacking content What are the characteristics
and skills?

• Persistent. Entrepreneurs work until a job is
  done. They are determined to overcome problems
  and to pursue their goals in order to make their
  business successful.
• Creative. Entrepreneurs are constantly looking
  for new ways to solve problems.
• Inquisitive. Entrepreneurs are curious about
  things that may affect their business. They are
  not afraid to ask questions or to conduct
  research in order to solve problems.

33
Relevant Activities
34
(No Transcript)
35
(No Transcript)
36
Test Items

• Which personal characteristic of successful
  entrepreneurs reflects that they work until the
  job is done?
• Creative
• Inquisitive
• Persistent
• Self-Confident
• Which personal characteristic of successful
  entrepreneurs reflects that they constantly look
  for new ways to solve problems?
• Creative
• Persistent
• Responsible
• Self-Confident

37
7075 Food Technology
6.01 Classify non-microbial food hazards.
38
Unpacking content What are the non-microbial
food hazards?

• Biological Hazards
• Seafood Toxins
• Plant Toxins
• Fungal Toxins
• Chemical Hazards
• Toxic Metals
• Cleaning Agents
• Pesticides and Insecticides
• Physical (e.g., fingernails, jewelry, hair,
  chipped glass)

39
Relevant Activities
40
(No Transcript)
41
(No Transcript)
42
Test Items

• Which is an example of a physical hazard?
• Dust particles
• Herbicide residue
• Mold spores
• Parasites
• Which is an example of a biological hazard?
• Dust particles
• Heavy metals
• Metal filings
• Poisonous mushrooms

43
7711 Masonry I
7.02 Apply procedures for laying a dry bond
44
Unpacking content What is the procedure?

• Check drawing specifications for bond type
• Mark the spacing on the foundation for each brick
  and mortar joint
• Lay units on foundation beginning at the starting
  point
• Check specifications for mortar joint size and
  use a 3/8 or ½ inch board for a mortar joint
  spacing jig
• Adjust spacing and mark end of unit
• Use framing square to mark 90 degree corner at
  the ends of the wall
• Check corner layout on drawings

45
Relevant Activities
46
(No Transcript)
47
(No Transcript)
48
Test Items

• While laying a dry bond the mason sees that an
  opening will require a change in spacing. What
  should he/she do?
• Adjust the spacing
• Check corner layout
• Lay units on the foundation
• Mark the end of every unit
• While laying out a dry bond the mason realizes
  that he/she is laying the wrong bond pattern.
  What should he/she do?
• Check corner layout
• Lay units on the foundation
• Mark the end of every unit
• Start the procedure over

49
Reviewing the Key Questions

• Why do we field test new/revised courses?
• Why is RBT important?
• What are the major differences between RBT-based
  courses and previous courses?
• What is expected of teachers who are teaching an
  RBT-course during the field test year?
• How do I examine a Curriculum Guide?