Basic Analysis Systems and Codes

1
Topic 2

• Basic Analysis Systems and Codes

2
A Bit About Analysis Codes

• In Topic 1 we talked a bit about how observation
  of teaching and coaching has changed over the
  years.
• What we are going to talk about now is what we
  call systematic observation.
• This method of observation still competes with
  eyeballing (coming in and just watching),
  rating scales, checklists and writing down what
  the observer saw.

3
So What Are These Techniques?

• Eyeballing
• An outsider (usually an administrator) enters the
  teaching or coaching setting and looks at the
  ongoing activities without making any formal
  written record of what is seen.
• Based on memory, feedback MAY be provided
• This technique is used not only for official
  evaluation purposes, but is also used to monitor
  teachers and coaches in their jobs.
• Whatever is picked up in passing may be used in a
  final evaluation for job retention.

4

• Rating Scales and Checklists
• For rating scales the observer gives an opinion
  on various aspects of the learning environment
  (or game environment).
• For checklists the observer merely marks whether
  certain things were said or done, based on their
  opinion.

5

• Anecdotal Recording
• This is similar to eye-balling, except that the
  observer writes down some of the things seen and
  heard.
• This is also called narrative recording and
  informal analysis.
• One last form is called scripting, which is when
  the observer tries to write down everything that
  was said and done by the teacher or coach during
  the observation time.

6
Limitations to Traditional Methods of Gathering
Data

• There is a lack of objectivity, reliability and
  specificit y
• Results reflect the observers opinion about
  events that were seen.
• Reflects biases, personal feelings and history of
  experiences.
• Because of this, evaluations/observations tend
  NOT to be consistent either with the person doing
  the observation, or even between two people
  observing the same setting.
• There is a lack of specific data upon which a
  person can make changes to improve their
  professional practice.

7
So What Is Systematic Observation?

• Systematic observation concentrates only on
  observable events and behaviors that can be
  recorded and measured.
• It focuses on descriptive information that is
  relatively objective. Specific coding rules
  require the observer to first describe the events
  that occurred as accurately as possible, and then
  make suggestions for change (if necessary).
• However, the data collected is purely contextual-
  they need to be looked at in terms of the
  situation in which they occurred (in other words,
  you cant use a one size fits all rule!).

8
Steps to Doing Systematic Observation

• There is a process to be followed when doing
  systematic observation- more than just going into
  a gym or to a playing field and watching whats
  going on!
• The steps that follow help to ensure that the
  information collected will be as reliable,
  accurate and valuable as possible- both to the
  person observing and the one being observed.

9
Step 1- Deciding What To Observe

• Why are you observing? Is it for improvement, to
  document a weak area, or to supervise anothers
  performance?
• What do you want to observe- or more importantly,
  what does the person you are watching want you to
  observe?
• Examples of variables to observe include discrete
  teaching/coaching behaviors including cues,
  feedback on skill performance, feedback on
  conduct, questions, praise, criticism analytic
  units or sequences of behaviors such as a
  sequence of questions, providing a correction,
  monitoring the athlete and then giving praise.

10
Step 2- Developing Definitions for the Behaviors
To Be Observed

• Well-developed definitions of behaviors to be
  observed makes the observers job so much easier!
• It is much easier to determine if a behavior
  occurredl
• Fewer disagreements between observers will occurl
• Target behavior is first described in general
  terms, then the final definition is developed.
• Good definitions avoid reference to things
  internal instead, they focus on what is
  external, or that which can be seen.

11
Step 3- Selecting the Most Appropriate
Observation System

• Based on what is to be observed, the observer
  then decides the most appropriate tactic to use
  to gather the information desired, then decides
  if there is a tool already developed that can be
  used.
• Sometimes you are after duration of behaviors
  (how long the behavior lasts), other times you
  are after information about how many times a
  specific behavior occurs.
• You must decide which is most appropriate to use
  for the information you want to collect.

12
Step 4- Establishing Observer Reliability

• Once you decide what system you will use, you
  then need to develop your reliability. Most often
  this is applied to 2 people viewing the same
  event and determining to what degree the two saw
  the same sequence of actions (liken this to
  gymnastics judges, who must be in a specific
  range of scores after notating a routine). It is
  also important that you develop reliability- that
  you see the same thing each time you watch the
  same event occurring (such as if you watch it on
  video).
• This reliability is developed through training
  and practice.

13
Step 5- Making the Actual Observation

• Be prepared to do the observation! Be prepared
  for the behavior of the teacher/coach to change
  because you are watching- and the same is true
  for the students/athletes.
• Be there early so you dont disrupt the routine.
• Try to be inconspicuous- in behavior, where you
  place yourself, your clothing and, dont
  interact with students or athletes unless it
  simply cannot be avoided.
• If you use video, keep it as unobtrusive as
  possible (and make sure it works before you start
  using it!).

14
Step 6- Summarizing and Interpreting the Data

• Once you have completed the observation, tally
  your data relative to what you were observing,
  and then summarize- based on what it was you were
  there to observe.
• Provide the date to the person observed- and then
  let that person interpret the data or place value
  on the data (your job is to present your
  findings).

15
A Few Other Steps

• Your decision about what to observe is based on a
  pre-observation conference you have with the
  person you will be watching. There are several
  documents with tips about this conference that
  have been placed in this Topic area for reading.
• Once you and the other person have decided what
  THEY want observed, then you follow the steps
  outlined previously.
• At the conclusion of your first observation, you
  do a post-conference, which is step 6. Then
  typically, you schedule another observation visit
  to go observe if changes have been made based on
  the data you presented earlier.

16
Now On To the Tools

• There are 4 basic types of recording tactics, and
  all tools that have been developed are
  derivations of one or more of these tactics.
• The four tactics are event recording, duration
  recording, interval recording and momentary time
  sampling.

17
Event Recording

• Used to record behaviors that occur over and over
  again.
• Used to present the frequency that a discrete
  event occurs (how many times did the person say
  ok?).
• Tallying is done with a gate count- each time the
  behavior occurs, a mark is placed in a box or
  specific place on the coding form.
• Must be able to see or ascertain a definite start
  and stop to the event.

18
Sample Event Recording Form
Click here for form
19
Duration Recording

• Used to record the amount of time that
  students/athletes or teachers/coaches spend doing
  something such as waiting for a turn, giving
  instruction, taking attendance, etc.
• Used when a behavior occurs at such high rates
  that it is more meaningful to record how long
  instead of how many.
• A stopwatch is used to record the amount of time
  an event occurred.

20
Sample Duration Recording Form
Click here for form
21
Interval Recording

• Measures the occurrence or non-occurrence of a
  behavior within specific time intervals.
• This combines both event and duration recording.
• Usually have a tape to cue the observer to tell
  when to observe and when to record.

22
Sample Interval Recording Form
Click here for form
23
Momentary Time Sampling

• Very similar to interval recording in that time
  is broken into intervals, but the observation is
  made only at the end of the interval.
• What is coded is that behavior seen at the end of
  the time interval, or that seen for a majority of
  the time in the interval.

24
Sample Momentary Time Sampling Code Form
Click here for form
25
Once Again..

• Dont reinvent the wheel!
• Look at what it is you will be observing, and
  determine if it is a tally (event recording) or
  how long each item lasts (duration recording), or
  what occurs in a specific period of time
  (interval recording) or what is happening at a
  specific point in time (momentary time sampling).
• See if there is already a tool- either one of the
  basic forms, or one of the forms we will be
  looking at later in this course.

26
Now Get To Work

• Now its time for you to put to use some of this
  information. Here are your instructions
• Take the following code (called the Anderson
  Code) and apply it to each of the basic recording
  tactics (event, duration, interval, momentary
  time sampling). The instructions and the forms
  can be found at this link.

27
Heres the Code

• Here are the code letters and definitions
• I- instructing (providing information about
  subject matter, teacher/coach directly involved
  in an instructional episode)
• M- monitor (watching students/athletes move for
  evaluative feedback)
• F- officiate, regulate (blowing whistle,
  directing activity, etc.)
• C- management (class, team- moving
  students/athletes, taking roll, lining up, giving
  announcements)
• B- behavior management (addressing behavior not
  performance- in class)
• X- other (teacher leaves class or coach leaves
  court/field, etc., anything that does not fit
  into a category listed above)

28

• Youll note that there are only 6 letters or
  items to code.
• Dont use more than this- it gets too difficult!
• Go slowly, and if you make a mistake, dont
  worry!
• This assignment is due MONDAY, FEBRUARY 25, 2008.