Title: RepeatedMeasures Design GLM 4
1Chapter_11
- Repeated-Measures Design (GLM 4)?
2What do you do in a Repeated-Measures ANOVA?
- Testing the same subjects in different
conditions - Extension of the related, within-subjects design
t-test? - Expl. You measure with a questionnaire the
enjoyment of a party after - no alcohol, 1 pint, 2 pints, 3 pints, 4 pints
- Advantage As with the related t-test, the
unsystematic variance is reduced, since we are
testing the same subjects - New assumption sphericity
3The condition of sphericity (denoted as ?) for
within subj designs also called circularity
- Between subj design
- When analysing data from different participants,
you have to assume homogeneity of variances,
i.e., same variances in all groups - Homogeneity of variances is tested with Levene's
test
- Within subj design
- Since your subjects are always the same, there
will necessarily be a relation between the
measurements in the various conditions in a
within subj design. While the relations cannot be
independent they should be the same between all
pairs.
4Compound symmetry and sphericity (??
- Sphericity is a special case of a more general
condition of compound symmetry. - Compound symmetry means that
- (a) Both the variances across conditions are
equal (like homogeneity of variances) and - (b) The covariances between all conditions are
equal - Sphericity is a less restricted form of (b). It
requires that the differences between pairs of
treatment levels (the conditions) have similar
variances. - ? For each pair of treatment levels, the
differences between each pair of scores should
have the same variance.
Variation within experimental conditions is
similar
No two condi- tions are more dependent than any
other two
For more details, see http//www.abdn.ac.uk/psy31
7/personal/files/teaching/spheric.htm
5Sphericity (?)needs at least 3 conditions
Variance of difference ? 'no alcohol 1 pint'
Enjoyment after No alcohol
Enjoyment after 1 pint
Variance of difference ? 'no alc 4 pints'
Variance of difference ? '1 pint 4 pints'
Enjoyment after 4 pints
- The variances of the ??between
- No alc 1 pint
- 1 pint 4 pints
- No alc 4 pints
- should be similar
?? delta difference
6Compound symmetry and sphericity an example
co-variances
- In this example from Kirk (1995), where A1-4
represent the 4 different conditions, the data
have been chosen such that the covariances are
unequal, hence compound symmetry is violated.
(Homogeneity of variances is violated, too.)
However, all covariance pairs are equal (see
equations below) hence sphericity is met.
(Baguley 2004, see website)?
variances
http//homepages.gold.ac.uk/aphome/spheric.html Ki
rk, R. E. (1995). Experimental design procedures
for the behavioral sciences. (3rd ed.). Pacific
Grove Brooks/Cole.
7Measuring sphericity
- Calculate the differences between the scores for
each pair of treatment levels. - Calculate the variances of the differences
- Expl. For the below data, sphericity holds if
- VarianceA-B ? varianceA-C ? varianceB-C
- In the data, there is local sphericity for the
pair A-C (10.3) and B-C (10.3), however the
variance of A-B (17) is somewhat higher. - How can we test whether sphericity still holds?
8Calculating the variance of the differences
between the groups
- 1. calculate the differences between each pair of
values, A-B, A-C, B-C - 2. determine for each group the mean. The mean
for (A-B) 0 (A-C)5.4 (A-C)5.4 - 3. Subtract the mean from each difference
- 4. Square it
- 5. Sum it up
- 6. Divide it by (n-1), here, 4.
1
3
4
2
5,6
9Measuring sphericity summary
Formula for Variance of the differences between
groups ((X-Y)-(?X-Y))2/n-1
Local sphericity (10,3 and 10,3)?
No sphericity (17 vs. 10,3 vs. 10,3)?
10Testing for sphericity Mauchly's test
- With the repeated measures ANOVA, Mauchly's test
of sphericity is automatically conveyed. For the
data above, the test is n.s., i.e., sphericity
can be assumed (although the variance of the
difference between (A-B) vs the other two pairs
(A-C) and (B-C) is higher).
If sphericity is violated, various corrections
(Green- house-Geisser, Huynh- Feldt) can be
used to estimate a better F-value
Lower bound is 1/(k-1) 1/(3-1) .5
11Theory of one-way repeated-measures ANOVA
Compare in a between subject design, the
within- subject variance is the residual
variance SSR!
- In a repeated ANOVA, the experimental effect
shows up in the within-subjects variance, between
the conditions. - The overall within-subjects variance is composed
of the exp variance and error variance - The exp variance derives from the different
treatment subjects have incurred in the various
within-subjects conditions. - The error variance derives from random factors
making subjects behave differently in the various
conditions, apart from the exp effect
12Partitioning variance for repeated ANOVA
SST total Sum of squares
SSW within participants SS
SS between groups, here 'subjectsessays' (will
come into play when calculating effect size only)?
SSR Error Variance
SSM Effect of Experiment (Model Variance)?
Model and residual variance both arise from
within subjects
13Expl 4 lecturers marking 8 essays
- Repeated measures since every lecturer grades all
8 essays (essays ar like subjects in this case)? - Indep Var 4 lecturers (lecturers are like
treatment levels in this case)? - Dep. Var grades in
14SST for grading essays
- First, we calculate the Total Sum of Squares SST
- SST s2grand (N-1)?
- The grand variance is the variance of all scores
- The grand mean is the mean of all scores
- df N-1
- SST 55.028 (32-1)?
- 1705.868
15The within-participant SSW
- In order to obtain the total value of individual
differences, we have to calculate the SS within
each subject and then add them up -
- SSW s2person 1 (n1-1) s2person 2 (n2-1) ....
- ... s2person n (nn-1)?
- Since we are using essays here and not subjects
- SSW s2essay 1 (n1-1) s2essay 2 (n2-1) ....
- ... s2s essay n (nn-1)?
person 1's SS
N of scores/ of lecturers 4 n-13
16Calculating SSW
- SSW 6.92 (4-1) 11.33 (4-1) 20.92 ( 4-1)
18.25 (4-1) 43.58 (4-1) 84.25 (4-1)
132.92 (4-1) 216 (4-1) - 20.76 34 62.75 54.75 130.75
252.75 398.75 648 - 1602.5
- The overall variance due to the
- individuals, here 'essays', is 1602.5
17Calculating SSM
- One part of the 'essay variance' SSM is the
variance explained by the model (experimental
variance, i.e., by the 4 different lecturers). - SSM ? nk (?xk - ?xgrand)2
- 8(68.875-63.94)2 8(64.25-63.94)2
8(65.25-63.94)2 8(57.38-63.94)2 - 554.125
- 554.125 variance units can be explained by our
experimental manipulation, the 4 lecturers.
nk of subjects, here 'essays' 8
df of SSM k-1 4-1 3
18Calculating SSR
- SSR SSW SSM
- 1602.5 554.125
- 1048.375
- The residual SS is the error variance within the
subjects. -
19What is SSR in repeated ANOVA?
SSR Error Variance
Error as Interaction The error term in
within-subject ANOVA designs is equal to the
Subjects x Condition interaction. As such, it is
a measure of the degree to which the effect of
conditions is different for different subjects.
Remember that when variables interact, the effect
of one variable differs depending on the level of
another variable. The error term is a measure of
the degree to which the effect of the variable
"Condition" is different for different levels of
the variable "Subjects." (Each subject is a
different level of the variable "Subjects.") Low
interaction (low error) means that the effect of
conditions is consistent across subjects. High
interaction (high error) means that the effect of
conditions is not consistent across "subjects."
Since the error term is an interaction, its df is
the product of the subjects' dfs (n-1) and the
conditions' dfs (a-1) df error (N -1)(a-1)?
http//www.davidmlane.com/hyperstat/within-subject
s.html
20SS between groups SSBG
- There is still one term left SSBG for the
between-group variance - SSBG SST SSM SSR
- 1705.868 554.125 1048.375
- 103.37
- Field calls it between group variance but in
this case it is actually the variation between
the subjects, here, the essays. - We do not need this value to calculate the
F-ratio for the main effect, but only later for
the effect sizes.
21Summary Partitioning variance for repeated ANOVA
SST total Sum of squares 1705.868
SSW within participants SS 1602.5
SS between groups (here essays)? 103.368
SSR Error Variance (subjects x essays)? 1048.375
SSM Effect of Experiment (4 lecturers)? 554.125
Model and residual variance both arise from
within subjects
22Mean squares MS
- The MS are the SS divided by their respective df
- MSM SSM 554.125 184.708
- dfM 3
- MSR SSR 1048.375 49.923
- dfR 21
- MSBG SSBG SSBG 103.37 14.77
- dfBG n-1 8-1
dfM k-1 4-13
dfR (k-1)(n-1)? (interaction!)? 3x7 21
dfBG n-1 8-17
23The F-ratio
- The F-ratio compares the amount of explained
variance MSM and the amount of error variance
MSR. - F MSM 184.708 3.7
- MSR 49.923
- Now we have to look up in the F-table whether
this value, given its df's (3 in the numerator
and 21 in the denominator), is significant. - Fobserved gt Fcritical ? yes, effect, p lt 0.05
- 3.7 gt 3.07
- ? It looks like there is a significant effect of
the 4 lecturers rating the 8 essays differently.
24One-way repeated-measures ANOVA using SPSS(using
Tutormarks.sav)?
- How to organize the data
- Each row represents 1 subject (here essay)?
- Each column represents 1 treatment level (here 4
tutors)?
25Defining the factors
- Analyze ? General Linear Model ? Repeated
Measures
Call factor1 'tutor'
There are 4 levels of 'tutor'
add
Click 'define'
26Within-subjects Variables
SPSS asks you which are the levels of your
within- subjects-variable Carry the 4 tutors to
the 4 Question marks
27Defining contrasts for repeated measures
Per default, a 'polynominal' contrast will be
carried out, but we choose a 'repeated' contrast
which compares each level with the one before.
Select the new contrast and press 'Change'
28post-hoc tests not directly available for
repeated measures
29Post hoc tests through the Options dialogue box
- Before carrying out Post hoc tests, it is
important to check whether the assumption of
sphericity has been met - If yes, you can use Tukey's post hoc test
- If not, Bonferroni's post hoc test can be used
- Even better as Tukey's is Games-Howell
- The PostHoc dialog box does not enable PostHoc
tests. They have to be requested through the
Options dialog box.
30The options dialogue box
Carry the within-subjects variable 'tutor' to the
'display means' window
Chose 'Bonferroni' as an adjustment for a post
hoc test
Request 'Descriptive Statistics' and a 'Transro
Matrix' which provides coding values for the
contrasts
Request 'Descriptive Statistics' and a 'Transfo
Matrix' which provides coding values for the
contrasts
Finally, press OK
31Output for Repeated Measures ANOVA Descriptives
The 2nd descriptive tells us the means, SD, and
n of the 4 levels of the within-subj var
The 1st descriptive simply tells us how many
levels the within- subjects var had (4) and how
they are called.
32Output for Repeated Measures ANOVA Test of
Sphericity
Mauchly's test of sphericity tells us whether the
differences between the variances for all 4
levels of the within-subjects variable are
similar here, the test is , hence sphericity is
violated!
Correction measures are provided when sphericity
is violated. Their lower bound is (1/k-1), in
our case 1/4-1) .333. The Greenhouse-Geisser
measure of .558 is closer to this lower bound
than to the upper bound of 1 (perfect
sphericity). This measure indicates a deviation
from sphericity. The correction values(G-G and
H-F)are multiplied with the df's of the F-ratios
and alter the significance level (see next
output)?
33The Main ANOVA
Different df's in dependence of the
various adjustment values
F MSM MSR
SSM
MSM
ns
SSR
3 x 0.558 1.673 3x .712 2.137
3 x 0.558 1.673 3x .712 2.137
MSR
- If sphericity cannot be assumed, we have to look
at the next 2 rows (Greenhouse-Geisser,
Huynh-Feldt). The original df has to be
multiplied with the adjustment values (see
previous slide). This changes the significance
level. - Note G-Geisser is too conservative (n.s.), while
Huynh-Feldt is quite liberal ()?
34What to do if sphericity is violated?
- We are faced with the situation of violated
sphericity. One of the alternative measures,
Greenhouse-Geisser is n.s., the other,
Huynh-Feldt is . - As a compromise, the mean between both may be
taken, which is (0.063 0.047)/2 0.055. - Since this average significance level is not
significant, we should reject the hypothesis and
regard the differences between the 4 tutors as
not significant. - ? Note The properly chosen F-value decides over
a significant or a non-significant result and
over making a Type 1 error or not. - However, since the effect is marginally
significant, it is good to check the effect size.
35What to do if sphericity is violated?
- An alternative is to use multivariate statistics
which does not make the assumption of sphericity.
SPSS provides those in the output as well.
All multi- variate test statistics are n.s.
Given that the multivariate test statistics are
n.s. as well, the Null-Hypothesis should be
accepted There is no overall difference between
the 4 lectures.
36Difference between Univariate and Multivariate
approach to Repeared Measures ANOVA
Source Mary Beth Oliver 2003 at http//www.perso
nal.psu.edu/users/s/x/sxl325/Comm20597E20Data20
Analysis/Lecture20Notes/Week201020--20Repeated
20Measures20--20Lecture.pdf
37Contrasts
- Although the main effect Tutor is n.s., we may
want to look at the single contrasts
coding matrix for contrasts
The 'Repeated' contrast compares 1st contrast
Field vs. Smith 2nd contrast Smith vs.
Scrote 3rd contrast Scrote vs. Death
38Contrasts
The 'Repeated' contrast reveals 1st contrast
Field vs. Smith 2nd contrast Smith vs.
Scrote n.s. 3rd contrast Scrote vs. Death
n.s. ? Although we have one contrast, it should
be ignored, given the overall n.s. main
effect. Rather, the study should be repeated,
with more subjects (essays).
39Post hoc tests
- The only post hoc test is between Dr. Field and
Dr. Smith. - Why isn't the comparison between Dr. Field and
Dr. Death , given that their means are even
further apart? - ? Due to sphericity There is a high correlation
between Dr. Field's and Dr. Smith's grades (low
variability), whereas there is high variability
between Dr. Field's and Dr. Death's grading.
40Correlations between the 4 lecturers
- Actually, Dr. Field's and Dr. Death's grades are
also highly correlated only negatively!
41Effect sizes for repeated measures (?2)Overall
effect
- ?2 (k-1/nk) (MSM MSR)
- MSR (MSBG -MSR)/k (k-1/nk) (MSM MSR)
- (4-1/8x4) (184.71 - 49.92)
- 49.92 (14.77 -49.92)/4 (4-1/8x4)
(184.71 49.92) -
- 12.64 .24
- 53.77
- ? The overall effect is rather small.
- (Note Here, the MSBG come into play)?
42Effect sizes for the single comparisons
- r ? F (1,dfR)
- F (1,dfR) dfR
- rField vs. Smith ??18.18/18.18 7
.85 large -
- rSmith vs. Scrote ??.15/.15 7
.14 small -
- rScrote vs. Death ? 3.44/ 3.44 7
.57 large -
43Reporting one-way repeated measures ANOVA
(Field_2005_p454)?
- The results show that the mark of an essay was
not significantly affected by the lecturer that
marked it, F(1.67, 11.71) 3.7, p gt .05. - In more detail
- Mauchly's test indicated that the assumption of
sphericity had been violated (?2 (5) 11.63, p
lt .05) therefore, degrees of freedom were
corrected using Greenhouse-Geisser estimates of
sphericity (?? .56). The results show that the
mark of an essay was not significantly affected
by the lecturer that marked it, F(1.67, 11.71)
3.7, p gt .05,?2 .24. - No further details are reported, since the ANOVA
is n.s.
44Repeated measures with several independent
variables(using Attitude.sav)?
hops
malt
- Study We want to find out the relation between
positive, negative, and neutral imagery in
advertising on the rating of three different
kinds of drink (beer, wine, water)? - 2 factors (indep var)
- Kind of drink (beer, wine, water)?
- Kind of imagery (positive, neutral, negative)?
- Dependent variable rating of drink (from 100
over 0 to 100)? - All levels of both factors were completely
crossed over. All participants rated all
combinations.
http//www.kneipp.de/uploads/pics/Kn_Pfl_ein_Hopfe
n004_470.jpg
www.fotogalerie.f-knieper.de/.../wein-160.jpg
www.uibk.ac.at/.../news/images/wasser400x306.jpg
45Total cross-over of factor levels ? 3 x 3 9
conditions
46Data entry 'Attitude.sav'
Each condition 1 column
Each subject 1 row
male
Note that we do not code 'gender' as a between
subjects dummy variable! That is we disregard the
possible effect of gender
female
47Analyze ? General Linear Model ? Repeated
Measures ? Define Factors
Name the two factors here first 'drink'
then 'imagery' (only 8 characters)?
As a last step, click on 'Define' to tell
SPSS which variables correspond to which of the
3 levels of the 2 factors
Specify of levels here (3)?
Add the factors to the window
48The main dialog box
SPSS wants to know which of the variables from
the left hand side correspond to which of the
factor combinations
Note In a repeated measures design we do not
have coding variables we identify the
conditions in this main dialog box
Highlight all 9 variables and transfer them to
the main window
49Thinking ahead the contrasts
- For later contrasts, we should arrange the
variables so that the control condition for each
factor is either the first or last level. - Here 'water' is the control for the 'drink'
factor - 'neutral' is the control for the 'imagery' factor
'Neutral' comes last (x,3)? since 'neutral' is
the control condition for 'imagery'
'Neutral' comes last (x,3)? since 'neutral' is
the control condition for 'imagery'
'Neutral' comes last (x,3)? since 'neutral' is
the control condition for 'imagery'
'water' comes last (3,x)? since 'water' is
the control condition for 'drink'
50Thinking ahead the contrasts
- Which kind of contrast is suited?
- Orthogonal Helmert contrast is NOT suited, since
the two imagery conditions (/-) should have
effects in the opposite direction and therefore
will cancel each other out. - Non-orthogonal simple effect is better suited it
compares beer (1) and wine (2) to water (3, last)?
51Contrasts
- Specify 'Simple' contrasts for both factors
'drink' and 'imagery'. Click on 'change' each
time. - Speficy the reference category as 'last'
52Graphing Interactions ? Plots
- Transfer 'drink' to the horizontal axis and
'imagery' to the separate lines. Click 'Add'. - In the plot, then, the values for pos/neg/neutral
imagery will be plotted for beer/wind/water. - (You can also do it the other way round, have
'imagery' on the horizontal and 'drink' on the
vertical axis. You can also do both.)?
53post-hoc tests not directly available for
repeated measures
54Options
Transfer the 2 factors and their interaction to
the Display means window
For comparing the main effects the
Bonferroni correction should be chosen
The 'Transformation Matrix' is important for
interpretation of the contrasts
- That's it!
- Run the analysis
55Output for Factorial repeated-measures
ANOVADescriptives
In this table, you can verify whether you
have entered your variables and their levels in
the correct way
Mean and SD for the various conditions
56Sphericity(whether the variances between the
levels of a factor are similar)?
Sphericity for 'Drink' and 'Imagery' is violated,
i.e., the variances between the levels of these
factors are not similar. ? Corrected F-values
have to be chosen. Sphericity for the Interaction
is given.
57Repeated ANOVA main output
Main effect 'Drink' (corrected F- value)?
Main effect 'Imagery' (corrected F- value)?
Interaction 'Drink''Imagery'
The corrected df's of the conservative
'Greenhouse-Geisser' value should be reported.
58Main effect 'drink'
- Main effect of drink Irrespective of kind of
imagery, people rated some types of drink
differently.
'Estimates' output
Part of the Test of Within subjects effects
The 'Estimates' output gives us the Estimated
Marginal means for the three levels of drink
1beer, 2wine, 3water. (Bonferroni-corrected)? E
xpl. The mean of 'beer' 11.833, is derived
from the values of 'beer' for each level of
'imagery (21.05 4.45 10.00)/3 11.833
59Plot Main effect 'drink'(in SPSS)?
- Beer and wine are rated higher than water.
60Main effect 'drink' , qualified by the pairwise
comparisons
- What underlies the main effect 'drink'?
beer
wine
water
The main effect 'drink' is brought about by the
difference between levels 2 and 3, i.e., wine
and water. Why isn't the difference between beer
and water ? ? because the SD of the beer is very
high whereas the SD of wine is very small. Small
SD's lead to large differences in means. The
Bonferroni correction controls for the error rate.
61Corrected vs. Uncorrected error rate(Bonferroni
vs. None (LSD) in the Options dialog box)?
Uncorrected error rate wine vs. Water is
(0.00)? beer vs. Water is (0.022)?
Corrected error rate only wine vs. Water is
(0.001)?
62Main effect Imagery
Part of the Test of Within subjects effects
- Main effect of imagery Irrespective of kind of
drink, people rated drinks differently due to
different types of imagery
Estimated Marginal means
The 'Estimates' output gives us the Estimated
Marginal means for the three levels of imagery
1positive, 2negative, 3neutral.
(Bonferroni-corrected)?
Expl. The mean 'positive' 21.267, is derived
from the values of 'positive' for each level of
drink (21.05 25.35 17.4)/3 21.27
63Plot Main effect 'imagery'(in SPSS)?
- 'Positive' has the highest rating, 'neutral' is
second and 'negative' comes last.
64Main effect 'imagery' , qualified by the pairwise
comparisons
pos
neg
neutr
65Interaction drinkimagery
- There is a interaction between drink x imagery
(F (4,76) p gt 0.001)?
This means that the kind of imagery (/-/neutr)
has a different effect depending on the kind of
drink.
The 'Estimates' output gives us the Estimated
Marginal means for the three levels of imagery
against the three levels of drink.
66The interaction plot
Negative imagery for beer still produced
ratings
The interaction plot is derived from the
Estimates
Wine suffers under negative imagery in particular
- The interaction is conveyed by the differential
effect of negative imagery on beer (still
postive). Also, negative imagery has a
particularly strong effect on wine (strongly
negative).
67Contrasts
- Drink
- 1.Beer vs. Water (was n.s. With Bonferroni
correction!)? - 2. Wine vs. Water (had been before, as
well)? - Imagery
- 1. Positive vs. Neutral
- 2. Negative vs. Neutral
68Alternative to contrasts'Simple effect
analysis'(using Simple.Effects.Attitude.sps)?
Our 9 conditions
Our 2 factors
Effect of imagery for the 3 levels
beer/wine/ water.
Click 'Run'
- In a 'simple effects' analysis we can look at the
effect of one independent variable at individual
levels of the other independent variable. - 'Simple effects' have to be specified via syntax.
69Output of 'Simple Effects'MANOVA
Effect of imagery on level 1 of 'drink'
(beer)
Effect of imagery on level 2 of 'drink'
(wine)
Effect of imagery on level 3 of 'drink'
(water)
- ??All levels of 'drink' (beer, wine, water) are
affected by 'imagery' (pos/neg/neutr)?
70Various contrasts
Contrast table
Beer vs. Water neg vs. Neutr imagery , no
parallel lines interaction
Beer vs. Water vs. Neutr imagery n.s.,
parallel lines no interaction
Wine vs. Water vs. Neutr imagery n.s.,
parallel lines no interaction
Wine vs. Water neg vs. neutr imagery , no
parallel lines Interaction
71Limits of contrasts
- Here, we have not exhausted all
- possible contrast. But still some useful
inferences could be drawn - Positive images raised the ratings of all
products, irrespective of what it was beer,
wine, or water. - Negative images had a greater effect on wine
(steep decline) and a lesser effect on beer
(modest decline)? - ? Interpreting interactions is a tedious
business. Do not give up. Creep into your data
and find out about them. They are highly
interesting!
72Effect sizes for factorial repeated measuress
ANOVA
DfRn-1 20-119
r ? F (1,dfR) F (1,dfR) dfR
rBeer vs. Water ?? 6.22/6.22 19
.50 rWine vs. Water ??
18.61/18.61 19 .70
rPositive vs. Neutral ?
142.19/142.19 19 .94
rNegative vs. Neutral ? 47.07/47.07
19 .84
Large effects
Huge effects
73Effect sizes Interactions
- RBeer vs. Water, positive vs. Neutral ?
1.58/1.58 19 .28 -
- RBeer vs. Water, negative vs. Neutral ?
6.75/6.75 19 .51 -
- RWine vs. Water, positive vs. Neutral ?
.24/.24 19 .11 -
- RWine vs. Water, negative vs. Neutral ?
26.91/26.91 19 .77 -
Small to medium and big effects
74Reporting Factorial repeated measures ANOVA
schematically (Field_2005_p480)?
- Mauchly's test indicated violated sphericity for
the main effects drink ?2 (2) 23.75, plt.001,
and imagery, ?2 (2) 7.42, plt .05. Therefore,
Greenhouse-Geisser-corrected degrees of freedom
were used. - Main effect of drink, F (1.15, 21.93) 5.11.
- Main effect of imagery, F (1.50, 28.4) 122.57.
- Significant interaction drinkimagery, F (4,76)
17.16. Contrasts revealed that negative imagery
(as compared to neutral) lowered scores
significantly more in water than it did for beer,
and lowered scores significatly more for wine
than it did for water.