Title: Experimental Psychology 225 Lecture 1, Fall 2004
1Experimental Psychology 225 Lecture 1, Fall 2004
Course Instructor John Curtin Teaching
Assistants Katie Graf Estes Joanne
Hogle Stephanie Schell Chris Ripley
2Todays Outline
- Introductions
- Go over syllabus policies
- Collect data for Experiment 1
- Out of here a few minutes early (hopefully! -)
3Introductions
Instructor John Curtin, John, Professor
Curtin, Dr. Curtin Office Room 326
Psychology Office Phone 262-0387 Office Hours
Tuesdays 1030-1230 EMail
jjcurtin_at_wisc.edu
4Teaching Assistants/Lab Instructors
Teaching Assistant Katie Graf Estes Office
Room 528 Psychology Office Phone
262-9942/265-0592 Office Hours MO
1230-130 WE 200-300 EMAIL
kmgraf_at_wisc.edu Teaching Assistant Joanne
Hogle Office Room B30 Psychology Office Phone
262-4028 Office Hours TU/TH 230-
330pm EMAIL jmhogle_at_wisc.edu Teaching
Assistant Stephanie Schell Office Room 623
Psychology Office Phone 262-6647 Office Hours
FR, 11am-1pm EMAIL saschell_at_wisc.edu
5Your turn.
6The Syllabus
- Course website
- http//dionysus.psych.wisc.edu/coursewebsites/psy2
25/psy225.htm - Lecture slides (ppt and pdf formats) available
evening before (or sooner). Announce on class
listserv - Provides updates to the syllabus, handouts, etc.
- Can view directly or print out hard copy
- Course email listserv
- fall225-curtin_at_lists.students.wisc.edu
- Will distribute information frequently
- Use as discussion list for class
- Copy sent to TAs and me
7Class Organization
- Two 1 ¼ hour lectures/week led by me
- One Discussion (1 hour) and one Lab (2 hours)
per week w/TA - No distinction between discussion and lab. Think
of them as three hours of applied instruction
each week - Not specifically designed to answer questions
about lecture - Think of lecture and lab as independent but
related courses covering same material from
different perspectives - Lecture will cover research methodology/statistics
at conceptual/abstract level - Lab will teach from a hands on practical level
- Many topics will be covered in both lecture and
lab. However each will also contain unique
material - Lab will also emphasize writing instruction
- Timing of coverage will vary across lecture and
lab
8Course Objectives
- Introduce the research process in Psychology
- Understand and critically evaluate published
research (informed consumer) - Learn to design, conduct, and analyze research
projects - Learn to communicate research results (written
and oral) - Learn scientific writing
- Critical consumer of information in general
(e.g., politics, policy)
9Required Recommended Texts
- Ray, W. J. (2003). Methods Toward a science
of behavior and experience (7th ed.). Pacific
Grove, CA Brooks/Cole Publishing Company. - American Psychological Association. (2001).
Publication manual of the American Psychological
Association (5th ed.). Washington, DC American
Psychological Association. - Recommended Text Kirkpatrick, L. A. Brooke,
C. F. (2003). A Simple Guide to SPSS for Windows
(revised ed.) Belmont CA Wadsworth/Thomson
Learning.
Available at University Bookstore and Underground
Textbook Exchange. Used APA manuals available
on Amazon
10Grading Policies
Examinations
- Three exams during lecture for 350 points (100,
100, 150) - Two sections multiple choice and short
answer/essay, - Third exam will be somewhat (50 pts) cumulative
- Make up exams will be given for those with
legitimate, verifiable excuses (i.e., illness
with doctors note). Make-up exams will be
different and more difficult! - Will go over exams in class following exam
- Scheduled exam dates are Sept. 30th, Nov. 11th,
and Dec. 17th _at_ 225PM) Dates are subject to
change
11Grading Policies
Class exercises/homework
- Occasionally, I will ask you to complete
something before class in order to facilitate
class discussions or to reinforce learning of
specific points. - Graded as complete or incomplete. No late class
exercises will be accepted.
- Class participation/Expression of ideas
- I am reserving 20 pts for class participation
- I will be open for suggestions on the format
(more on this later) - My current plan is
- 2 pts per session
- Cant get more than 2 pts per session
- 10 sessions gets all points
12Grading Policies
- Laboratory papers
- Six primary writing assignments based on
experiments you conduct (3 graded, 3 P/F) - Assignment 1 Method results P/F 5pts
- Assignment 2 Method results Graded 50pts
- Assignment 3 Introduction P/F 5pts
- Assignment 4 Introduction, results Graded 75
pts - Assignment 5 Discussion P/F 5 pts
- Assignment 6 (IP) Full scientific report
Graded 125 pts - Graded assignments will be assessed a penalty of
10 deduction per day late - P/F assignments not accepted late
- The 225 CURVE
- How it works
13Instructor/TA Evaluation
- Evaluations of course progress, and teaching
style of instructor/TAs - Allows for online changes in course that benefit
you (and me) - Anonymous Use Student Number and I will not
decode until end of class - Will provide 1 point for each one completed.
Easy way to get points added into your final grade
14Attendance
- Strongly encouraged but not required
- Much material in lecture and labs will not be
covered in the book (e.g., content, critical
thinking) - Need to attend to get participation points
15Problems w/ Course
- Please see me. I will do everything possible to
correct the problem - Comment in evaluations
16Schedule of Readings and Topics
- Topics are listed by week and date for lecture
- Text readings are listed by week. Have readings
completed by the end of the week in which they
are assigned - Additional readings (primary source journal
articles will be assigned as course progresses - Schedule can change but will be announced in
class and updated on the web - Additional readings in APA manual outlined in lab
syllabus
17Note-taking recommendations
- Print out notes ahead of time or afterwards
- If afterwards, slide numbers are provided Dont
copy slide content into notes - Blank spaces in slides for answers to questions
(class points for answers). Designed to
encourage and allow for critical thinking - My perspective on learning Think actively and
critically in class - This course is very different from other
psychology classes. Not a content/memorization
course - Reminder This is a 5 credit course with a very
large workload (equivalent to two regular courses)
18Homework
I want you to find (and make a copy of) an
article in a magazine or newspaper. This article
should be covering what you think are scientific
findings. You will write three BRIEF paragraphs
on this article. In the first paragraph,
summarize the main points of the article. Next,
write 1 paragraph on what you think science is.
Finally, write 1 paragraph about what you think
makes your particular article scientific (so you
relate it back to your second paragraph). Bring
both the article and this writing assignment
(TYPED) to class Tuesday 9/07/04.
19Homework
- Two homework assignments for next week
- Read and answer questions about Stanovich
article Due 9/14/04 - Read and answer questions about Platt article
Due 9/16/04 - Turn in on due date
- Must be typed. Must be completed by due date. No
late homework - Put your name TA name on homework
- Due dates and digital copy of questions on web
20Experiment 1
- Randomly divide class into three groups (count
off 1-3) - Group one stays here with me
- Group two goes to room XXX with XXXXX
- Group three goes to room XXX with XXXX
21Announcements
- Turn in Media Assignment at end of class today
- Read and answer questions about Stanovich
article Due 9/14/04 - Read and answer questions about Platt article
Due 9/16/04 - Any lecture slide printing problems?
- Re-introduce participation points procedure
22Methods of Obtaining Knowledge
- Tenacity Accepting an idea as valid knowledge
b/c that idea has been accepted for a long period
of time - Common Sense Obtaining knowledge without any
intellectual effort/thinking or involvement of
sensory processes. Includes both Intuition and
Cultural knowledge - Authority Acceptance of an idea as valid
knowledge b/c some respected source claims it is
valid - Rationalism Knowledge is developed through
reasoning processes alone. (Reason) - Empiricism Gaining knowledge through direct
observation
23Write a brief description of what you see
24Rules for Acceptable Observations
- An observation must be available to more than one
person-- intersubjective verification. - The description of the observation from multiple
observers must be quite similar-- reliable. - The conditions under which observation made and
the details about how the measurement was
accomplished must be clearly specified--
operational definition.
25Announcements
- Read and answer questions about Stanovich
article Due 9/14/04 - Read and answer questions about Platt article
Due 9/16/04 - Library research sign-up
- ID number on participation points (5058, 9888,
2299, 9383, 5851, 7044)?
26What is Science?
Lets hear some examples that you found of science
in the media.
Which of the previous methods are not part of the
scientific method of acquiring knowledge? Tenacity
, common sense, authority
Why are they not scientific? i.e., what are
their shortcomings relative to the scientific
method?
- Don't always conform to data (evidence)
- Can be uncritical and accepting
- Subjective (can be overly influenced by cultural,
personal factors)
27What is Science?
- Science is a process of acquiring knowledge
- Based on experience and observation (i.e.,
empiricism) - Incorporates rationalism to build theories
- Characterized by a more critical and systematic
approach to obtaining knowledge - Goal is understanding how and why things happen
- Characteristics of Science
- Based on empirical observation guided by theory
- Uses logic/reasoning to support conclusions
- Observations are systematic, verifiable,
repeatable, and well-defined - Science is cyclical and self-correcting
28Phases of Scientific Research
- 1. Idea generating phase Identify a topic of
interest to study. - 2. Problem-definition phase Refine vague and
general idea(s) generated into precise question
to be studied. Good psychological science should
be guided strongly by theory. - 3. Procedures-design phase Decide on specific
procedures to be used in gathering and
statistical analysis of the data. - 4. Observation phase Using the procedures
devised, collect your observations from the
participants in your study. - 5. Interpretation phase Compare your results
with results you predicted. Do they support your
theory? Leads to further theory development,
modification or discard theory. - 6. Communication phase Prepare a written or
oral report of your study for publication.
Application can follow from here.
29Phases of Scientific Research
Idea generating phase
Problem-definition phase
Procedures-design phase
Observation phase
Interpretation phase
Communication/Application
30Announcements
- Turn in Stanovich homework
- Read and answer questions about Platt article
Due 9/16/04 - Library research sign-up if missed class
31Levels of Constraint
Naturalistic Observation Behavior observed in
the natural environment with no attempt to change
or limit the environment or behavior of the
subjects. Case Study Researcher intervenes
somewhat by asking questions and directing line
of inquiry. Flexible with attention following
interesting responses that seem relevant at that
time. Correlational Research Setting can be
natural or laboratory but variables of interest
and measurement methods are precisely defined and
decided prior to the start of the
study. Differential/Quasi-experimental
Involves comparison of two or more groups.
Groups are selected to be closely matched on all
factors other than the one pre-existing
difference being studied. Experimental
Research Different groups are compared but the
researcher controls assignment to the groups and
directly manipulates the variable/factor being
investigated.
Lower Constraint Higher
Constraint
32Scientific Theory
Theory defined A formalized set of concepts that
organizes observations and inferences and
predicts and explains phenomena. Theory
specifies relationships between constructs
- Characteristics of a good theory
- Usually based on a large body of empirical
findings - Testable (Precise, Specific Falsifiable)
- Parsimonious (simple)
- Organizes the collection of subsequent
observations
33Scientific Theory
Hypothesis A specific prediction based on theory.
A hypothesis is a statement or prediction about
a relationship that should be observed between
two or more variables if the theory is correct.
Variable Any directly observable characteristic
that can take on different values. These include
demographic/subject variables, independent
variables, dependent variables and many other
categories.
Construct Generalized, abstract concept that is
constructed to explain relationships among
observed variables in a particular situation.
Once formulated, constructs are used, as if they
exist, to theoretically predict (hypothesize)
effects on other constructs and their associated
variables in situations that had not been
previously observed.
34Reasoning/Logic in Scientific Theory
- Logic is at the heart of science (Rationalism
component) - Concepts of falsifiability and strong inference
are derived from basic forms of logical reasoning - Used to organize observations into theory
- Used to make predictions about future events from
theory
- Inductive and Deductive Reasoning
- Inductive reasoning When a researcher begins
with observation of specific variables and then
infers constructs and develops theory. - Deductive reasoning When the theory about
psychological constructs serves as a basis for
making predictions about new observations for
specific variables.
35Reasoning/Logic in Scientific Theory
Deduction Theory ? Data
36Propositional Logic
- If p then q
- If you are playing the badgers (or Florida State
Seminoles) in football, then you will lose. - First part of the logical statement is called the
antecedent (p) - Second part is called the consequent (q)
- There are four types of propositions that can be
made in this framework - We can affirm or deny the antecedent (2)
- We can affirm or deny the consequent (2)
- Two are correct and two are incorrect
37Announcements
- Turn in Platt homework at end of class
- Pick up Stanovich homework now
- IDs 5226, see me after class
- IDs 5058, 2299 (9347 Wicke), 7044, 2289, 3232
(Schrammel), 5038 - 2287 2297(Guralski)
- Hands for not registered
- Library research sign-up if missed class
38Affirming the Antecedent (Modus Ponens)
- Confirmatory logic
- 1. If p then q
- p (p is true)
- Therefore, q (q is true)
Is this correct or incorrect logic?
CORRECT
- Example
- 1. Individuals with depression lack the normal
(but inaccurate) protective cognitions about self
worth and efficacy Depressive realism. They
are more accurate in their self-appraisals and
this causes them to be depressed. - Jay has accurate self-appraisal of his
worth/efficacy - .and Jay is depressed
39Affirming the Antecedent (Modus Ponens)
- We use this in everyday life to predict what will
happen based on our understanding of the
consequences of our actions. - This is also how theories inform us about our
world. - You are using this form of logic when forming
hypotheses to test from a theory. Basically, you
are saying what you expect to observe if your
theory is true. -
- If Construct A changes, then Construct B will
change
40Denying the Antecedent
- 1. If p then q
- Not p (p is false)
- Therefore, not q (q is false)
This logical reasoning is INCORRECT but not
really used in science
41Affirming the Antecedent
- When forming hypotheses, we are affirming the
antecedent - In this future experiment, if I manipulate the
Independent variable, then the Dependent variable
will change - (1) If p then q (2) p (3) q
- This is valid/correct reasoning deductive
reasoning
42Affirming/Denying the Consequent
- When testing hypotheses, we are either affirming
or denying the consequent - If Theory is true then these results will be
observed in this experiment our hypothesis will
be supported/true - (1) if p then q (2) q or not q (3) p or not p
- We conclude based on empirical evidence
(observations/data) and statistical analyses that
our hypotheses/predictions in an experiment were
either true (q) or false (not q) - From this, we want to conclude that our theory is
true (p) or false (not p)
43Affirming the Consequent
- 1. If p then q
- q (q is true)
- Therefore, p (p if true)
- Is this correct or incorrect and when in does it
occur?
- Although this logic is not correct, this
situation is frequently encountered in research.
- This is the case when our predictions/hypotheses
are supported
Explain this and its implications?
44Recognizing Logic in Research
You develop an intervention to improve school
performance for disadvantaged children in the
inner-city. The intervention involves 1 year of
classes, involving both parent child. It is
intensive and not many families volunteer to
participate. Two years after completion you
compare school performance of children who
completed the intervention with a sample of
children not offered this intervention. The
intervention kids outperform the other kids.
- What proposition is suggested by this?
- What conclusion might you reach?
- What form of logic must you use to do this?
- What is the problem with this?
- What else needs to be done?
45Affirming the Consequent
- What proposition is suggested by this?
- What conclusion can you reach?
- What form of logic must you use to do this?
- What is the problem with this?
- What else needs to be done?
Your proposition might be If the intervention
is effective, intervention kids will outperform
others Intervention kids will have higher mean
performance that non-intervention kids on some
test. The natural conclusion to reach is that
the intervention was effective. However, this
is an example of affirming the consequent
. .and it is invalid to conclude based on this
type of logic alone that the antecedent
(intervention is effective) is true. Many
other potential explanations for the observation
(intervention kids outperform others) may
exist. Must rule out other explanations in this
study and conceptual replications.
46Affirming the Consequent
- In research design we attempt to rule out all
other causes for the observed outcome but we can
never conclude with certainty that we have proved
our theory with this form of logic. - This is where replication (repeating an
experiment either exactly rarely done not
worthwhile, or conceptually) becomes important.
Although never proved, when predictions from a
theory are repeatedly confirmed, we become more
confident that they are true. Why?
47Affirming the Consequent
- Scientists attempt to overcome the flaws of this
method of logic by exercising as much control
over conditions as possible. - By eliminating as many possible alternatives for
explaining q (superior performance of the
intervention kids), you can increase confidence
in the conclusions about your theories. - This is done through many different methods
including - The use of control groups
- Controlling important variables
- Experimental methods (e.g., random assignment)
are especially effective in establishing control
and eliminating alternatives
48Denying the Consequent (Modus Tollens)
- Disconfirmatory logic
- 1. If p then q
- Not q (q is false)
- Therefore, not p (p if false)
Is this correct or incorrect logic?
CORRECT
The stress response dampening theory states that
alcohol acts directly on fear centers in the
brain and interferes with activity in this
system. Therefore, intoxicated individuals will
not experience as robust (strong) a stress
response as sober ones.
- If alcohol intoxication leads to stress reduction
If SRD theory is true, then intoxicated
individuals will be less stressed than
non-intoxicated individuals - Intoxicated participants are not less stressed
than non-intoxicated participants - Therefore, intoxication does not lead to stress
reduction SRD theory is false
49Denying the Consequent (Modus Tollens)
- This (logic of falsification) is at the heart of
scientific testing of theory (Popper) - By successfully falsifying a theory (i.e., by
falsifying a specific hypothesis), we advance
knowledge confidently. - Unlike the situation where confirm our
predictions, when we fail to find the observed
effect, we can be logically confident that the
theory is incorrect - However, it is not always that simple. WHY?
- Not enough power to find effect due to
- Small sample
- Weak manipulation e.g., effects of nicotine in
withdrawn smokers - Measures werent sensitive enough
- More on this during reliability and also strong
inference
50Recognizing Logic in Research
The FDA considers it unnecessary to label food
that has been genetically engineered, despite
public demand, since the no risks to our health
from eating these foods has been documented.
Industry spokespersons argue that the problem is
one of consumer ignorance and to persuade people
of their viewpoint they have only to educate us
about the genetic engineering (GE) process. The
following argument is used to promote spending on
consumer education.
- If people dont understand GE they will be
reluctant to consume GE food products. - Numerous studies demonstrate that people are
indeed reluctant to eat GE food products. - Therefore, they must not understand GE.
- Education can potentially change this.
Is this a logical argument? Explain? What else
should this person do to strengthen their
argument for the need for education?
51Recognizing Logic in Research
Schachter studied the relationship between
anxiety and the need to affiliate (to be with
others). His theory suggested that anxiety would
cause the need to be with others (basically,
misery loves company!) From this theory, you
can derive the following proposition
- If anxiety causes the need to affiliate, then
anxious people will not choose to wait alone (as
often as non-anxious people) - He conducts a study and finds that anxious people
DO choose to wait alone (as often as non-anxious
people).
What type of logic does this involve and what if,
if anything, can you conclude from this?
52Announcements
- Pick up Platt homework now
- Library research sessions tonight and tomorrow
- STAI homework assignment (email by 830am
Thursday)
53Theory Development and Logic
- Theory advances in the following fashion
- Predict a falsifiable hypothesis based on theory
- Design and conduct an adequate test of the
hypothesis - (a) If prediction is false, modify or discard
theory. This is denying the consequent and it is
logically valid. - (b) If prediction is true conduct
additional tests of theory (replications) which
are aimed at eliminating other explanations.
These replications must be conducted to rule out
other explanations b/c affirming the consequent
does not inevitably indicate that the antecedent
(i.e., theory) is true.
54Wason Example
Determine what my rule is for generating a
correct series. First Series 2 - 4 - 6
Rule/Theory Numbers increase by 2 Tests?
55Wason Example
The rule was All numbers that increase.
- Two strategies for determining rule
- Enumerative Generate examples that confirm your
current theory - Eliminative Generate examples that can
disconfirm your current theory - Individuals using eliminative strategy determine
rule 3x faster than those using enumerative
strategy
- Can you connect these ideas to other discussions
we have had about theory and more generally
science? - Theory develops through falsification (Popper).
- This is the self-correcting part of science,
again
56Falisfication Stanovich Article
- Why did Stanovich object to Benjamin Rushs
defense of bloodletting as a cure for yellow
fever? - How do proponents of ESP explain the inability of
skeptics to demonstrate effects? - According to Stanovich, why is specificity of
predictions important? - According to Stanovich, what is the difference
between a theory and a hypothesis? - Why isnt repeated confirmation of a theory
sufficient evidence of its validity?
57Falsification Stanovich Article
- How could the strategy of falsification be used
in everyday life? - Describe Stanovichs simple model of scientific
progress. Is the public too lazy to think
scientifically? - What should new theories be capable of doing?
58Announcements
- Answers to homeworks on the web
- Exam date
- Exam review
59Platt Strong Inference Article
- List the steps a scientist should take in
applying the method of strong inference to a
problem. - How does strong inference go beyond Poppers
falsification strategy? (How is it similar and
how is it different?) - What are the advantages of strong inference?
- What are the difficulties?
- What question does Platt say you should ask
yourself when planning a new study?
60Operational Definitions
Operational definition Detailed set of
procedures used to measure or manipulate the
levels of a construct. Operational definitions
transform abstract/conceptual constructs into
objective, reliably measurable variables.
61An Indian Fable
It is said that once upon a time a king gathered
a few men who were born blind. They were asked to
describe an elephant, but each one was presented
with only a certain part of it. To one was
presented the head of the elephant, to another
the trunk, to another its ears, to another the
leg, the body, the tail, tuft of the tail, etc.
The one who was presented with the head said
"The elephant is like a pot!" The one who was
presented the trunk answered, "The elephant is
like a hose". The one who touched only the ears
thought that the elephant was a fan, the others
said that it was a pillar, a wall, a rope, a
brush, etc. Then they quarreled among
themselves, each thinking that he was the only
one right and the others were wrong. The obvious
truth is that the elephant is a unity of many
parts, a unity that they could not grasp in their
ignorance.
62Operational Definitions
There are four broad categories of operational
definitions. They are as follows
- Actual behavior
- Physiological/biological
- Self-report
- Manipulation
- It is best to use all four categories, (often in
different studies), to converge on the true
concept you wish to examine. - Using one of the four categories may make the
concept you are trying to examine look different
compared to if you had used a different category.
63Conceptually Defining Anxiety
- As an exercise, pause for a few minutes and write
down, in the space below, your definition of
anxiety, as if you were writing a book of
psychological terms. - Discuss your various conceptual definitions of
anxiety.
A state of apprehension, uncertainty, and
uneasiness resulting from the anticipation of a
realistic or imagined threatening event or
situation from which you can not escape, often
impairing physical and psychological functioning
64Operationally Defining Anxiety
List at least THREE ways by which you could
measure anxiety at a physiological level. Be
quite specific, remembering that your definition
must be clear enough so that another researcher
could replicate it.
- Increased heart rate (arousal)
- Increased skin conductance (arousal)
- Potentiated startle (Yes)
- Increased right frontal brain activity (Yes)
- Increased cortisol (HPA axis activation) (Yes)
65Operationally Defining Anxiety
List at least THREE ways by which you can measure
anxiety at a nonverbal, observational level.
Specifically, what behaviors and/or conditions of
an individual would indicate to an observer that
s/he was experiencing anxiety?
- Speed of approach toward anxiety-eliciting
stimulus - Time spent in an anxiety eliciting environment
- Disruption of performance of an independent task
66Announcements
- New Exam Date Oct 7th
- Review session on Monday or Tuesday _at_ 6pm?
- First class evaluation Due Thursday
- Complete the online human subjects training
module at - http//info.gradsch.wisc.edu/research/compliance/h
umansubjects/tutorial/ - Due 10/07/04
- KT Travis
67Operationally Defining Anxiety
List several ways in which you can measure
anxiety through verbal reports. Specifically,
what kinds of questions would you ask to
determine if an individual was experiencing
anxiety? These can be true/false, multiple
choice, or open-ended questions.
68Operationally Defining Anxiety
Now list several ways in which an researcher
could experimentally produce anxiety with certain
procedures and/or stimuli. Be sure that they are
ethical. Once again be specific enough so that
another researcher could replicate your procedure.
- Give a speech about most embarrassing body part
- Threat of electric shock or other noxious
stimulus - IAPS
69Defining Other Constructs for Homework
- Provide a conceptual definition and
operationalize it across the 4 categories
- CREATIVITY
- DEPRESSION
- INTELLIGENCE
- LOVE
- MOTIVATION
- OBESITY
- SELF-ESTEEM
706 Main Ideas From Exercise
- Sometimes there are different possible conceptual
definitions. Differences in these conceptual
definitions are often times theoretically driven. - The conceptual definition often will influence
how you operationally define something. - Some terms have more relation or less relation
between their conceptual and operational
definition (obesity vs. love). - There are 4 broad categories of operational
definitions (actual behavior, physiological/biolog
ical, self-report, manipulation) - Any one definition can be wrong (incomplete). The
best operational definitions sample across all
areas. Often it's best NOT to stay with one
operational definition only--probably want at
least 2 or 3 categories. - Operational definitions (as opposed to conceptual
definitions) MUST BE precise, specific, and
measurable.
71Measurement Issues
The Measurement of Intelligence An Example
(trivia tape)
- The single trivia question approach. Problems?
- Very inconsistent. (Same person might be judged
intelligent or unintelligent on different
occasions depending on the specific question
asked) - Can not be related to intelligence b/c
inconsistent (and probably other reasons)
- The tape measure approach. Problems?
- Consistent but may not be related to intelligence
- The IQ test
- Consistent (in what ways?)
- Related to intelligence (what evidence?)
72Quantifying Reliability and Validity
To index various types of reliability and
validity we will look at relationships between
observed variables. What (descriptive/inferenti
al) statistic allows us to quantify the magnitude
(and direction) of a relationship between two
variables?
- The correlation coefficient
73Correlation Coefficients
What are the important properties of correlation
coefficients (How do we use them to describe
relationships?
- Correlations range from 1.0 to 1.0
- Strength of relationship is indexed by the
absolute value of the coefficient (bigger is
better) - Direction of relationship is indexed by the sign
of the coefficient
Measures of reliability and validity are
typically expressed as correlations that relate
to the property (specific type of reliability or
validity) we are trying to demonstrate for our
variable.
74Reliability Shooting Bulls
Reliable
Reliable
Unreliable
- Reliability is an index of the consistency of
measurement - Test-retest reliability
- Inter-rater reliability
- Internal consistency
75Test-Retest Reliability
Definition Consistency of measurement across
multiple administrations/measurements separated
by some length of time.
- What is the appropriate length of time to assess?
- Too short of a period of time (hours) may
overestimate reliability by allowing people to
be consistent b/c they remember their previous
responses - Too long of a period of time (e.g., years) may
underestimate reliability because construct may
actually have changed (e.g., height) - Should all tests/measures possess this form of
reliability? - Not necessarily. It depends on whether the
construct is expected to remain stable over time
(e.g., intelligence/aptitude) or to vary (e.g.,
mood)
76Inter-rater Reliability
Definition Consistency of measurement across
multiple observers/raters who are performing the
measurement. Rate how attractive the person is
on a 10 point scale (1 10) with higher scores
indicating more attractiveness.
- Ratings of Psychopathy with PCL
- DSM diagnosis
77Inter-rater Reliability
- How could we improve our ratings of
attractiveness? - Provide a clear definition of what we are rating
(our opinion, group consensus, what makes someone
attractive) - Provide anchors to the scale
- Provide training with feedback
78Inter-rater Reliability The Psychopathy Checklist
- Promiscuous sexual behavior
- Behavioral problems early in life
- Lack of realistic long-term plans
- Impulsiveness
- Irresponsible behavior
- Lack of remorse
- Many marital relationships
- Juvenile delinquency
- Callousness
- Criminal versatility
- Glibness/superficial charm
- Grandiose sense of self-worth
- Tendency to boredom/need for
stimulation - Pathological lying
- Conning/manipulative behavior
- Failure to accept the consequences of actions
- Shallow affect
- Lack of empathy
- Parasitic lifestyle
- Poor behavioral controls
Inter-rater reliability ranges from 0.87 to 0.97
for trained raters Hare, R. D., Harpur, T. J.,
Hakistian, A. R., Forth, A. E., Hart, S. D.,
Newman, J. P. (1990). The revised psychopathy
checklist reliability and factor structure.
Psychological Assessment, 2, 338-341.
79Internal Consistency Reliability
- Definition Consistency across items (typically)
that combine to form the questionnaire/measure - STAI example ( extra item)
- 0.83 0.91 for PCL-R Cooke Michie, 1997
Is the goal to have the highest internal
consistency possible? Not always. Sometimes a
construct is heterogeneous (i.e., consists of
multiple dimensions). To have construct
(content) validity, you will need to measure all
the independent dimensions. Items from these
dimensions may not relate to each other (i.e.,
NEM from MPQ)
80Validity Shooting Bulls Revisited
Valid
Not Valid
Not Valid
Validity (of measurement) is an index whether
your test (or other measure including
behaviors, etc.) truly measures what you think it
measures (i.e., the construct of interest).
- Face validity
- Construct validity
- Content validity
- Criterion validity (predictive concurrent
validity convergent divergent validity)
81Face Validity
Definition Does the test/measure appear to
measure what it purports to measure (e.g., STAI
vs. MMPI)
Why is face validity desirable?
- It is a simple, convenient way to provide initial
support for the validity of a measure. - It can be related to cooperation on the part of
the participant (again, MMPI).
Is face validity always good?
- High face valid measures are open to bias on the
part of the participant. - Face validity is not sufficient to demonstrate
validity. Often something that appears face
valid may not really measure what you think it
does.
82Construct Validity
Definition The degree to which the test/measure
provides an adequate measure of the construct.
- It is an umbrella term that includes concepts of
content validity and criterion validity - It must occur within a theoretical framework
(i.e., you need to be able to theoretically/concep
tually describe the construct to proceed in
assessing the construct validity of its measures.)
83Content Validity
Definition The degree to which the test/measure
adequately covers the construct to be measured
(I.e., does it assess all aspects/dimensions of
the construct)
- It is particularly relevant for achievement tests
(e.g., exams in this class shortstop
performance) - It is often performed in a somewhat subjective
manner (i.e., developing a measure of depression
with expert consensus) - Empirical methods also exist (Factor analysis)
84Criterion Validity
Definition The degree to which scores on the
test are related to other measures of the
construct (e.g., criterion validation of an IQ
test)
- Must assess the relationship between your new
measure and other established measures (ideal if
a gold standard exists) - Need to demonstrate both convergent (relates to
other measures that it should relate to) and
divergent (does not relate to measures that it
should not relate to) validity - This may be done with both concurrent measures
(other measures obtained at the same time) and in
a predictive (look at the ability of the measure
to predict future scores on another measure)
fashion
85Inferential Statistics
Descriptive Statistics Are used to describe,
summarize and simplify data. Provides a single
(typically) numeric value to summarize some
aspect of the overall data set.
Inferential Statistics Are used to infer the
status of a question (about descriptive
statistics) in a full population of individuals
based on a sample from that population. Answers
from inferential statistical are probabilistic.
In other words, all answers have the potential to
be wrong and you will provide an index of that
probability along with your results.
86Describing Data Scales of Measurement
- Nominal Are naming scales. They only have
the property of identity - Ordinal Measure a variable in order of
magnitude. Therefore they have properties of
identity and magnitude - Interval Have properties of ordinal scale and
there are equal intervals between values - Ratio Have properties of interval scale and
there is an absolute zero point
87Describing Data
- Graphical Representation of Data
- Bar Graph
- Histogram
- Frequency Polygon
- Frequency Counts
- Summary Statistics
- Measures of Central Tendency
- Mean
- Median
- Mode
- Measures of Variability/Dispersion
- Range
- Variance
- Standard deviation
- Measures of Shape
88(No Transcript)
89(No Transcript)
90(No Transcript)
91(No Transcript)
92(No Transcript)
93(No Transcript)
94(No Transcript)
95(No Transcript)
96Bar Graph
97Sample Histogram
98Frequency Polygon
99Bar Graph How Much Do Students Drink?
100Frequency Polygon How Much Do Students Drink?
101What is wrong with this picture?
102Frequency Counts
- Drinks/week Frequency
- 0 - 4 203
- 5 - 9 49
- 10 -14 25
- 15 - 19 27
- 20 - 24 16
- 25 - 29 19
- 30 - 34 4
- 35 - 39 6
- 40 - 44 1
- 45 - 49 8
- 50 - 54 0
- 55 - 59 0
- 60 - 64 1
- 65 - 69 1
- 70 - 74 3
- 75 - 79 0
- 80 - 84 2
- 85 - 89 1
103Summary Statistics
- Measures of Central Tendency
- Mode
- Median
- Mean
- Measures of Variability
- Range
- Variance
- Standard deviation
- Measures of Shape
- Skewness
- Kurtosis
104Central Tendency Mode
Definition Most frequent score
- ADVANTAGES
- 1. Most probable score in data set
- 2. Only measure of CT for nominal data
- DISADVANTAGES
- 1. Only based on one score in data set
- 2. Can be more than one of them
105Central Tendency Median
Definition Score at the 50 percentile (middle
value)
- ADVANTAGES
- 1. Not very sensitive to outliers
- 2. Only measure of CT for ordinal data
- 3. Can be used to split data set into equal
parts. (Median split) - DISADVANTAGES
- 1. Uses less information from data set than
mean
106Central Tendency Mean
Definition Arithmetic average
- ADVANTAGES
- 1. Uses most information from data set
- 2. Is used in many parametric inferential
statistics -
- DISADVANTAGES
- 1. Very sensitive to outliers
107Variability Range
Definition Distance between end points of
distribution (spread from end to end)
- ADVANTAGES
- 1. Lets you know range of values
- DISADVANTAGES
- 1. Very sensitive to outliers
- 2. Only based on two scores in data set
108Variability Variance
Definition Average squared distance of scores
from mean (spread from mean in squared units)
- ADVANTAGES
- 1. Based on all scores in data set
- 2. Used often in parametric inferential
statistics - DISADVANTAGES
- 1. Units of measurement often dont make
sense
109Variability Standard Deviation
Definition Square root of variance (spread from
mean in original units)
- ADVANTAGES
- 1. Based on all scores in data set
- 2. Has same units as variable being
described
110Measures of Shape
- Skewness
- How symmetric is distribution?
- Kurtosis
- How peaked is distribution?
- Ratio scores in center vs. tails of distribution
111Skewness 0 Symmetric
112Skewness gt0 Right tail too long
113Skewness lt 0 Left tail too long
114Which Distribution has larger SD?
115Which Distribution has the Higher Mean?
116What is the Modal Grade?
117What is the Median Grade?