Chapter 10: Fair or Foul . . . The play

1
Chapter 10 Fair or Foul . . .The plays the
thing!

• I'll have groundsmore relative than this, the
  play's the thing wherein I'll catch the
  conscience of the King.
• Hamlet, Prince of Denmark
• Or
• Catching your own conscience reflecting on your
  testing approaches.

2
Topics on Administering Analyzing TestsWhen
using them for important summative assessments

• Test administration
• Item analysis
• Cheating

3
Fairness Real and Perceived

• Important that students perceive value in the
  assessments
• Motivated to do especially well on tests related
  to
• recorded grades, report cards
• passing or failing decisions
• Three attributes of perceived fairness
• Teach what you test.
• Broadly meant not just what you said in class.
• Give adequate notice.
• Pop quizzes are okay as long as you let students
  know they are a possibility.
• Be fair in grading.
• Consistent and reasonable.
• Fairness does not mean easy. Many students
  appreciate high standards and a challenging test.

4
Suggestions, prior to test day, for . .
.Preparing Test Materials

• Give clear directions what, if any,
  supplemental materials are permitted (scratch
  paper, dictionaries, calculators).
• Use a simple format use white space and
  logical page breaks.
• Allow lead time for copying dont race into
  office and expect your test to be done right
  away.
• Spot check pages complete, in correct order,
  not inverted.
• Take one test and answer it non-fatal mistakes
  you can correct with oral directions fatal means
  recopy (dont postpone).
• Worry about security keep secure may wish to
  number copies.
• Comparable forms for multi-section courses
  students in later classes will ask students in
  earlier sections about the test.

5
Suggestions, prior to test day, for . . .
Preparing Yourself and the Students

• Assuming you have provided excellent content
  instruction, recommended actions include
• Give adequate notice to the students. This
  includes the type(s) of items that will be found
  on the test.
• Schedule the test so that you have time
  built-in when the test is completed for scoring
  of the test.
• Consider likely circumstances.
• Are there any special equipment needs?
• Is there anything scheduled that could be
  disruptive (fire drill, pep rally)?
• Should you pre-guide students as to what to do
  if there is unanticipated commotion in the room
  or building? (What to do with test documents.)
• Make advance preparation for accommodations.
  Accommodation is the technical term for
  alterations in testing conditions related to
  student disability. Specifically
• An accommodation changes a condition (e.g.,
  same test more time)
• A modification changes the test itself (e.g.,
  different test)

6
Ohio Assessment Accommodations Criteria
established in Ohio Administrative Code Rule
3301-13-03

• To quote from the Accommodations statements found
  on the Ohio Department of Education (ODE)
  website Statewide Assessment Accommodations
  include a list of common assessment
  accommodations meant to help IEP teams make
  appropriate decisions about allowable
  accommodations for statewide assessment. It is
  not an exhaustive list. Designations such as
  "allowable" or "not allowable" for the test
  subjects/grades and accommodations have been made
  in light of the four allowable criteria for
  statewide assessment accommodations explained at
  the beginning of the list.
• To quote from other parts of the ODE statement
  specific to testing
• Any accommodation that gives a student with
  disabilities an unreasonable advantage is not
  allowable, because it does not allow for valid
  assumptions to be made from the results.
• Accommodations are changes made in how a student
  has access to the curriculum or demonstrates
  learning. . . . In testing situations,
  accommodations are changes in format, response,
  environment, timing or scheduling that does not
  alter in a significant way what the test measures
  or the comparability of the scores.

7
Suggestions, on the day of the test, for . . .
Immediate Preparation of Students

• Think about classroom seating arrangement
  would rearranging be offensive how would you
  rearrange (e.g. every other seat).
• Determine how materials (e.g., test, answer
  sheet) will be passed out (yourself passed front
  to back) - check that everyone has all materials.
• Establish control students often talk when
  materials are passed about, decide when to
  declare conversation time over.
• Direct students to place names on answer sheet
  and on test itself if this is important to you
  for security reasons. This may suggest to
  students that you look at any marks they make on
  the test itself.
• Review test directions some students skip the
  written directions while others agonize too long
  over them (should be no surprises).
• Provide directions as to what the students do
  when they finish their test (e.g., turn in, wait
  quietly, study).
• Give quick words of encouragement humor may
  work or may backfire.

8
Suggestions, once the test starts, for . . .
Managing the Testing Environment

• Monitor the testing situation dont use this
  time as personal time to work on other classes
  circulate around the room, help students stay on
  task see if it appears someone is not correctly
  following the directions (make sure you tell
  students you will be doing this). When you do
  sit, sit behind students out of their vision.
• Answer questions with fairness in mind dont
  give hints to individual students, clarification
  only. If it appears what you might be telling
  them is important, consider telling the entire
  class.
• Apprise students of remaining time if timing is
  important or if there is no clock visible.

9
Suggestions, once the test finishes, for . . .
Scoring and Returning Test Results

• Many teachers dislike this part it is time
  consuming. The utility of having provided
  yourself with a previously prepared scoring key
  and time to score is key for success in this
  area. I suggest you score exams yourself.
• Return results promptly most often the sooner
  the better. Getting results back quickly is good
  for the mental health of students, and yourself!
• Dont announce individual scores (although you
  can publicly acknowledge a students outstanding
  performance) dont post grades (legal issue will
  be discussed later). You may make general
  narrative statements (either positive or
  negative) . . . I prefer positive (for example,
  the class did well).
• Reiterate the scoring system you used. Your
  scoring approach should not be a surprise to the
  students but, by restating it, you demonstrate
  you tested them exactly as you said you would.
• Decide how you want to deal with student
  questions. Will you do over all items, some
  items? What kinds of questions will you accept,
  how long will the questions go on? Will you
  change grades based on the discussions?

10
Suggestions, once the test review finishes, for .
. . Retaining Test Results

• Recover materials as desired do this in such a
  way that students do not see the scores of
  others. Do you want all parts back? Can
  students keep old tests for studying? If
  collected, what will you do with them?
• Record scores do this after you have gone over
  test with students. Use grade book (hardcopy or
  electronic).
• Make notes - on changes you would like to make in
  the assessment system when you teach this in the
  future. Do this now while it is fresh in your
  mind.

11
Classroom Item Analysis . . .Examining the
quality of individual items p and D
statistics

• When the test is completed, we have two basic
  questions
• Was the difficulty of the item appropriate or was
  it too hard or too easy? I can compute the item
  difficulty index by dividing the number of
  correct answers by total answers
• p percent correct
• The higher the value the easier the item
• If the item was difficult, was this difficulty
  okay? Does it discriminate between those who know
  and those who dont? To answer this we compute
  the item discrimination index. In this case we
  look at each item in terms of did the students
  who do the best on this test tend to get it right
  more so than the students who did the worst. The
  book divides the class in half (H L) I tend to
  like thirds (H, M, L). Compare the percent
  correct between H L groups
• D H - L
• The higher the value, the more the item
  discriminates . . . notice it go negative or
  positive. Whats a or - mean?

12
Item Analysis ExampleNote the values in the
table are percentagesof correct responses,
High/Low/Total Group
13
Classroom Item Analysis, So What . . .

• Item analysis is important because a poor or bad
  item may add to the unreliability of the test.
  Why? Because if your purpose in the test was to
  distinguish between those students who know the
  material and those who do not, a bad item fails
  to work toward this end (e.g., items answered
  correctly by all, or items that no one answers
  correctly.)
• A rough "rule-of-thumb" is that if the item
  difficulty is more than .75, it is an easy item
  if the difficulty is below .25, it is a difficult
  item.
• It is my feeling that items with difficulties
  less than 20 or more than 80 (especially if
  they fail they discriminate) deserve your
  attention. Such items should either be revised or
  replaced. Perhaps you have a non-functioning
  distracter. Perhaps your question misleads the
  best students. An exception might be at the
  beginning of a test where easier items (90 or
  higher) may be desirable for initial confidence
  building.

14
Taking item analysis to the next level . . .
Item Analysis Programs

• If you get into item analysis on a large scale
  (many students/items) consider
• Independent vender computer software Test
  Packages on test construction which often
  include item analysis features. These may not be
  good long term since the companies that produced
  them come and go (no tech support).
• The Test Development Software Package provided
  by the publisher of the textbook you are using.
• Test item analysis using the Microsoft Excel
  spreadsheet program.
• Your school district may have a centralized test
  scoring service (usually limited to selected
  response types of exams). This likely means that
  you would need to require that your students use
  a bubble sheet because it would be scanned.
• These computer programs may use more complex
  calculations for indices (aka, indexes) related
  to item difficulty or discrimination but the
  concept will be the same. For example, item
  discrimination might be presented in terms of a
  correlation coefficient (with a Greek letter or
  an unusual sounding name, of course). Remember
  how to interpret a correlation coefficient? The
  scale is -1 to 1 so you would interpret it
  much like the D statistic.

15
Cheating

• Graphic from A Taxonomy of Text and Exam Fraud
  by Dennis Maynes, 2008

16
Say it aint so, Joe . . .Cheating and Academic
Integrity

• Most teachers would rather not discuss this
  perhaps a form of denial.
• Prevalence
• All levels of education
• Student View No big deal everybody does it.
• Practices
• Plagiarizing, especially Internet-based
• Cut and paste without attribution
• Other projects completed outside class
• Assistance from parents, siblings, family
  friends, fellow students range from incidental to
  complete authorship
• Cheating on exams, tests (crib notes, look at
  anothers test, pass notes)
• Lets look at some cheating examples a major
  source was
• Gary K. Clabaugh Edward G. Rozycki, Preventing
  Cheating and Plagiarism, 2nd Edition (2003)
  Oreland, PA NewFoundations Press

17
Taking note of crib notes . . .So called
because of how one cradles the small sheets.

• OLD SCHOOL
• Get into the classroom prior to test and write
  crib notes on the desktop or the back rest of
  seat in front. Variation, tape note folder on
  underside of desktop.
• Make sure the crib notes can be easily destroyed.
  Example, chew gum during exam if a teacher
  becomes suspicious eat the piece of paper. Do NOT
  write on yourself, it is impossible to hide if a
  teacher questions you.
• Print off crib notes in size 6 font about 4 wide
  by ½ long. Acquire transparent pen (BIC
  recommended). Roll the paper around the tube of
  ink and slide it in.
• Remove label from a clear beverage, such as
  Snapple. Put notes on the back of the label, then
  paste it back in place using transparent glue.
  During the exam take slow "thoughtful" swigs out
  of the bottle.
• Wear laced up shoes on the day of the exam (with
  laces remove). Attach your crib notes behind
  each tongue of the shoe.
• NEW SCHOOL
• Crib notes on an electronic organizer a mini
  tape recorder can be fitted with a tiny earphone
  that can be hidden by combing hair over the ear
  programmable calculators use of a cell phone in
  a lavatory.

18
Other Cheating Gambits . . .

• Signal answers among group. Keep an eye out for
  the rearrangement of items on desktops, coded
  body movements, and hand signals.
• A student who has postponed a test or who takes
  it in a later section, enlists a confederate to
  smuggle an extra copy of the exam out of the exam
  room.
• Fraternity approach. Smuggle a copy of the test
  from the room at the end of the test, note the
  answer sequence when the test is reviewed then
  keep both the test and the answers on file.
• Student sits for the exam, pretends to be taking
  the test, but completes nothing. Then they turn
  in the blank exam and makes sure the instructor
  notices it has been returned. When the tests are
  graded and the student gets a zero because they
  didn't take the test, he or she complains
  bitterly that their test has been lost.
• When tests are returned students either alter
  their answers from wrong to right or fill in
  answers they deliberately left blank. Then they
  complained that their test was marked
  incorrectly.
• Students fake injury or illness to postpone tests
  for which they are unprepared.
• Students show up late, feign surprise and claim
  that they misunderstood when the test was to be
  given.

19
Cheating Suggestions for Prevention

• Create positive classroom atmosphere - reasonable
  standards, sensible workloads, keep content
  interesting, keep upbeat attitude yourself.
• Talk to students about cheating define cheating
  behavior, identify consequences immediate/long
  term. Is cheating rewarded in society?
• Reduce anxiety and uncertainty - be clear and
  informative about upcoming tests, what will be
  tested, how.
• Be vigilant consider changing seating, walk
  among the students during testing.
• When cheating occurs, confront it teachers
  disregard of cheating is one reason students say
  they feel so free to do it.
• Determine if your school has a policy support it
  - if the policy is nonfunctional, then help
  develop new policy.

20
Practical Advice

• Use summary checklists regarding each aspect of
  test administration to remind you of
  considerations at each phase you value.
• Learn to do simple item analysis, use
  occasionally.
• Take cheating seriously.

21
Terms/Concepts to Review andStudy on Your Own

• accommodations
• D
• item analysis
• item bank
• item difficulty index
• item discrimination index
• modifications
• p or p-value