Why use multilevel modelling

1
Why use multilevel modelling?
Clio Day
Jon Rasbash GSOE November 2006
2
A simple question
How much of the variability in pupil attainment
is attributable to schools level factors and how
much to pupil level factors?
First of all we have to define what we mean by
variability
3
A toy example first we calculate the mean
Two schools each with two pupils.
Overall mean (32(-1)(-4))/40
4
Calculating the variance
The total variance is the sum of the squares of
the departures of the observations around mean
divided by the sample size(4)
(94116)/47.5
5
The variance of the school means around the
overall mean
The variance of the school means around the
overall mean
(2.52(-2.5)2)/26.25
Total variance 7.5
6
The variance of the pupils scores around their
schools mean
The variance of the pupils scores around their
schools mean
((3-2.5)2 (2-2.5)2 (-1-(-2.5))2
(-4-(-2.5))2 )/4 1.25
The variance of the school means around overall
mean (2.52(-2.5)2)/26.25
Total variance 7.56.251.25
7
Returning to our question
How much of the variability in pupil attainment
is attributable to schools level factors and how
much to pupil level factors?
In terms of our toy example we can now say
6.25/7.5 82 of the total variation of pupils
attainment is attributable to school level factors
1.25/7.5 18 of the total variation of pupils
attainment is attributable to pupil level factors
8
Now lets do the same thing on real data(65
schools4000 pupils)
Overall mean0 (attainment scaled to have 0 mean
overall)
Total variation 1
Variance of school means around overall mean0.15
Variance of pupils attainment scores around
school mean0.85
85 of variability due to pupil level factors,
15 due to school level factors
9
Estimating parameters of distributions
The multilevel model assumes the school means
and the pupils departures around their school
means are Normally distributed.
The Normal distribution has two parameters the
mean and the variance. Our model estimates
N(0,0.85) for pupil within school effects and
N(0,0.15) for school effects
We gain a great deal of modelling power and
flexibility by making these Normality assumption
10
(No Transcript)
11
Can we explain the variation at the school and
pupil levels?
With educational data we typically want to take
account of pupil intake ability when they enter
school. In our data set the plot looks like this
12
Can we explain the variation at the school and
pupil levels?
What might happen to the between school and
between pupil within school variances when we
correct for prior ability?
Both the between school and between pupil within
school variation will be reduced when we model
mean attainment as a function of prior ability
13
And on the real world data
Recall that the between school and between pupil
variances before taking account of prior ability
were 0.15 and 0.85 respectively.
After taking account of prior ability between
school and between pupil variances are reduced to
0.092 an 0.566.
So accounting for prior ability explains 39 and
35 of between school and between pupil variation.
The effect of taking account of prior attainment
is important but not entirely surprising. School
variability is partly determined by school intake
profile and pupil prior attainment is a good
predictor of subsequent attainment.
14
Obvious next questions
Are there other school level variables that can
explain why schools differ?
Does school gender(mixed school, boy school, girl
school) explain some of the between school
variation?
If we (additionally to prior ability) allow the
mean to be a function of school gender the
between school variance is reduced by 13.
15
Focussing on school gender differences
16
Problems with traditional techniques
How much between school variation is there? What
school level predictor variables can explain some
of this variation?
This line of enquiry is powerful. Traditional
statistical analysis techniques can not pursue
this exploratory avenue.
Estimate the between school variability OR
Estimate school level predictors such as school
gender
But not both
17
It gets worse
If we fit a single level model we get incorrect
uncertainty intervals leading to incorrect
inferences
This is because the single level model ignores
the clustering effect of schools
mixed school boy school girl
school
The distributional assumptions made by the
multilevel model allows the estimation of between
school variance and school level predictors.
18
Variance is our business
We have modelled mean attainment as a function
of prior ability and school gender
We have simultaneously modelled the total
variation in attainment as function of school and
pupil levels.
Traditional modelling techniques are unable to
partition the variation in this way and just
estimate a single term and refer to it as error.
We think this variation is not error, it contains
a lot of interesting structure. Multilevel
modelling is a great tool for exploring the
structure in the error term.
19
Is there a family effect?
Recent studies in developmental psychology and
behavioural genetics(BG) emphasise non-shared
environment and genetic influences are much more
important in explaining childrens adjustment
than shared environment has led to a focus on
non-shared environment.(Plomin et al, 1994
TurkheimerWaldron, 2000)
Multilevel modelling can replicate the BG
analysis. It can also extend them to more
reasonably represent the complexities of family
structures and processes. When this is done
persistent family effects are found.
20
(No Transcript)
21
10 schools two scenarios
22
Is there a family effect?
Recent studies in developmental psychology and
behavioural genetics emphasise non-shared
environment and genetic influences are much more
important in explaining childrens adjustment
than shared environment has led to a focus on
non-shared environment.(Plomin et al, 1994
TurkheimerWaldron, 2000)
My collaborators from psychology Jenny
Jenkins(Toronto University) and Tom
OConnor(Rochester University) were concerned
that perhaps the analytic techniques being used
might have some simplifying assumptions that made
it difficult to pick up the shared family
context. They were interested to see if applying
multilevel models, with the recognised strengths
in exploring contextual effects might turn up
some different findings.
23
Two analyses
1. Understanding the sources of differential
parenting the role of child and family level
effects. Jenny Jenkins, Jon Rasbash and Tom
OConnor Developmental Psychology 2003(1) 99-113
2. Applying social network models to within
family processes. Currently being written up for
publication.
24
Differential parental treatment

• One key aspect of the non-shared environment that
  has been investigated is differential parental
  treatment of siblings.
• Differential treatment predicts differences in
  sibling adjustment
• What are the sources of differential treatment?
• Child specific/non-shared age, temperament,
  biological relatedness
• Can family level shared environmental factors
  influence differential treatment?

25
The Stress/Resources Hypothesis
Do family contexts(shared environment) increase
or decrease the extent to which children within
the same family are treated differently?
Parents have a finite amount of resources in
terms of time, attention, patience and support to
give their children. In families in which most of
these resources are devoted to coping with
economic stress, depression and/or marital
conflict, parents may become less consciously or
intentionally equitable and more driven by
preferences or child characteristics in their
childrearing efforts. Henderson et al
1996.This is the hypothesis we wish to test. We
operationalised the stress/resources hypothesis
using four contextual variables socioeconomic
status, single parenthood, large family size, and
marital conflict
26
Modelling the mean and variance simultaneously
We show a possible pattern of how the mean,
within family variance and between family
variance might behave as functions of HSES in the
schematic diagram below.
Here are 5 families of increasing HSES(in the
actual data set there are 3900 families.
We can fit a linear function of SES to the mean.
positive parenting
The family means now vary around the dashed trend
line. This is now the between family variation
which is pretty constant wrt HSES
HSES
However, the within family variation(measure of
differential parenting) decreases with HSES
this supports the SR hypothesis.
27
Conclusion on differential parental treatment

• We have found strong support for the
  stress/resources hypothesis. That is although
  differential parenting is a child specific factor
  that drives differential adjustment, differential
  parenting itself is influenced by family factors
  such as HSES.
• This challenges the current tendency in
  developmental psychology and behavioural genetics
  to focus on child specific factors.
• Multilevel models which model both the mean and
  the variability simultaneously are needed to
  uncover these relationships.

28
(No Transcript)
29
Deconstructing relationships what determines how
people get on within a family?
family Culture
the individual
the dyad(the two people relating)
genes
30
Applying models from social network theory to
family data
Non-Shared Environment Adolescent
Development(NEAD) data set, Reiss et al(1994).

• 2 wave longitudinal family study, designed for
  testing hypothesis about genetic and
  environmental effects
• 277 full-sib pairs, 109 half-sib pairs, 130
  unrelated pairs, 93 DZ twins and 99 MZ twins,
  aged between 9 and 18 years
• Wave 2 followed 3 years after wave 1 and any
  families where the older sib was older than 18
  were not followed up.
• A wide range of self-report, parental-report and
  observer variables were collected.
• All families had 2 parents and 2 kids of the same
  sex.
• We focus here on data on relationship quality
  collected by observers.

31
Within family structure
We start with 12 relationship scores in each
family. These can be classified
partner
dyad
actor
Family 1
Dyad d1 d2
d3 d4 d5
d6
Relationship c1?c2 c1?m c1?f c2?c1 c2?m
c2?f m?c1 m?c2 m?f f?c1 f?c2 f?m
Partner c1
c2 m
f
This model is the multilevel social relations
model-SnijdersKenny(1999)
32
Useful diagrams for thinking about multilevel
structure
The relationship scores are contained within a
cross classification of actor, dyad and partner
and all of this structure is nested within
families. This can structure can be shown
diagramatically with
A unit diagram one node per unit
A classification diagram with one node per
classification
33
Interpretation of variance components
Familythe extent to which family level factors
effect all the relationships in a family. Actor
the extent to which individuals act similarly
across relationships with other family
members(actor stability, trait-like
behaviour) Partner We actually have two traits
operating, in addition to the trait of common
acting to other family members we also have the
trait of elicitation from other family members.
The greater the partner variance component the
greater the evidence for such a trait
operating. Dyad The extent to which relationship
quality is specific to the dyad. A high dyad
random effect means that the relationship score
from joe-gtfred is similar of that from fred-gtjoe.
In social network theory this is known as
reciprocity. Reciprocity is a context specific
effect(non trait-like) Relationship residual
variation across relationships in relationship
quality.
34
Results of SRM more detail
Table shows variance partition coefficients
For positivity 44 of the variablity is
attributable to actors indicating that
individuals act in a consistent way across
relationships with other family members. There is
a strong actor trait component to positivity.
For negativity 0.41 of the variability is
attributable to dyad. Indicating the dyad is an
important structure in determining negativity in
relationships. There is a strong context specific
component to negativity.
There is little evidence of an elicitation or
partner trait for either response.
At the family level there are stronger effects
for negativity than positivity.
35
Modelling the mean relationship quality in terms
of role
The basic unit, a relationship, has an actor and
a partner. Actors and partners are classified
into the roles of children, mothers and fathers
by the two categorical variables actor_role and
partner_role.
We use child as the reference category for
actor_role and partner_role variables.
36
Including actor and partner roles-positivity
Modelling actor and partner role drops likelihood
by over 1000 units with 4df.
The effect is dominated by the actor role
categories. With mothers and then fathers being
much more positive as actors than the reference
category child.
These actor_role role variables explain over 50
of the actor level variance.
Adding interactions between actor_role and
partner-role does not improve the model.
Since we have explained actor level variance this
means actor role explains the some of the trait
component of relationship positivity.
37
Graphing actor and partner role effects for
positivity
The graph shows actor_role having a big effect on
relationship quality and partner role having a
marginal effect.
actor child actor m
actor f
38
Including actor and partner roles-negativity
Now an interaction is required between actor_role
and partner_role. Note the interaction categories
a_mothp_moth and a_fathp_fath structurally do
not exist.
Modelling actor and partner role and the
interaction drops the loglike by 500 units with
6df.
Note the main drop in the variance occurs at the
dyad level which reduces by 15. This means
modelling actor and partner roles has explained
context specific variation in relationship
quality for negativity.
39
Graphing actor and partner role effects for
negativity
With respect to actor and partner roles the main
context specific effects for relationship quality
occur in relationships where the child is an
actor..
Whether the partner is another child, a mother or
a father greatly effects the negativity of the
predicted relationship quality
actor child actor m
actor f
A possible psychological explanation for this
pattern is that negativity is high stakes
behaviour. The amount of negativity a child feels
safe to express is determined by the
power/authority of the partner.
Note that parents are trait-like wrt actor
negativity effects.
40
Genetic effects
Individuals exhibit some trait-like behaviour for
both relationship positivity and negativity. With
individuals exhibiting stronger trait-like
behaviour for relationship positivity.
Such trait-like behaviour may have a genetic
component.
The standard behavioural genetics model for
children within families estimates shared
environment(family), non-shared
environment(individual) and genetic components of
variation.
Our structure is more complex in that the lowest
level is not the individual but a relationship
between two individuals. Also we have a dyad
component of variation and the individual
component of variation is split into actor and
partner components.
However, we can extend the basic BG model (which
incorporates some questionable assumptions) to
our structure. The extended model gives
heritabilities (genetic variance)/(total
variance) of 0.42 and 0.16 for positivity and
negativity respectively. The actor and partner
variance components were reduced with the
inclusion of genetic effects but the family
variance component was undiminished.
41
Stability of effects over time
The data has two waves where the same
relationships were measured three years later.
This allows us to explore the stability of
family, actor, partner, dyad and relationship
effects over time.
We can operationalise the longitudinal structure
by fitting a multivariate response social
relations model where the first response is the
time 1 relationship score and the second the time
2 relationship score.
We simultaneously estimate all variance
components for each response
and the following correlations
42
Stability results of two bivariate SRM
The basic patterns of the vpcs found in wave 1
are repeated in wave 2 for both positivity and
negativity.
Family effects are very stable over time for both
positivity (?12 0.77) and negativity (?120.8).
Family effects are a bit stronger for negativity.
Actor effects are stronger for positivity than
negativity but stability across time is high for
both actor behaviours(0.87 and 0.67)
Dyad effects are much stronger for negativity
than positivity. But the stability of dyad
effects for both behaviours is lower than actor,
partner and family effect stabilities. Dyads are
more stable for negativity than positivity.
43
A comment on family effects
Developmental psychology and behavioural
genetics, .(Plomin et al, 1994
TurkheimerWaldron, 2000). Have suggested that
after taking account of genetic and individual
level factors there is scant evidence for family
level effects. Our work shows strong family level
effects, that persist over time, even when
genetic, actor, partner, dyad and relationship
level variance components are included in the
model. Part of the previous failure to find
family effects may be the analytical strategy of
breaking down families into series of overlapping
dyads and analyising each dyad separately. This
strategy is probably in part determined by the
methodology available to the researchers.
44
A comment on dyad effects for relationship
negativity
For relationship negativity we saw large dyad
effects and relatively low stability over
time. This means that at wave 1 there is a large
within family variability in dyad negativity and
likewise at wave 2. However the dyads which are
most and least negative within the family are to
an extent switching around. The next step is to
see if we can find some systematic pattern to
these dyadic dynamics for relationship negativity.
45
Alspac data an example of highly complex
multilevel structure
All the children born in the Avon area in 1990
followed up longitudinally
Many measurements made including educational
attainment measures
Children span 3 school year cohorts(say
1994,1995,1996)
Suppose we wish to model development of numeracy
over the schooling period. We may have the
following attainment measures on a child
m1 m2 m3 m4 m5 m6 m7
m8 primary school secondary
school
46
Structure for primary schools

• Measurement occasions within pupils

• At each occasion there may be a different teacher

• Pupils are nested within primary school cohorts

• All this structure is nested within primary school

• Pupils are nested within residential areas

47
A mixture of nested and crossed relationships
Nodes directly connected by a single arrow are
nested, otherwise nodes are cross-classified. For
example, measurement occasions are nested within
pupils. However, cohort are cross-classified with
primary teachers, that is teachers teach more
than one cohort and a cohort is taught by more
than one teacher.
48
Multiple membership
It is reasonable to suppose the attainment of a
child in a particualr year is influenced not only
by the current teacher, but also by teachers in
previous years. That is measurements occasions
are multiple members of teachers.
We represent this in the classification diagram
by using a double arrow.
49
What happens if pupils move area?
Classification diagram without pupils moving
residential areas
If pupils move area, then pupils are no longer
nested within areas. Pupils and areas are
cross-classified. Also it is reasonable to
suppose that pupils measured attainments are
effected by the areas they have previously lived
in. So measurement occasions are multiple members
of areas
Classification diagram where pupils move between
residential areas
BUT
50
If pupils move area they will also move schools
Classification diagram where pupils move between
areas but not schools
If pupils move schools they are no longer nested
within primary school or primary school cohort.
Also we can expect, for the mobile pupils, both
their previous and current cohort and school to
effect measured attainments
Classification diagram where pupils move between
schools and areas
51
If pupils move area they will also move schools
cntd
And secondary schools
We could also extend the above model to take
account of Secondary school, secondary school
cohort and secondary school teachers.
52
So why use multilevel models?
It gives the correct answers for the standard
errors of regression coefficients(in the presence
of clustering). Thereby protecting against
incorrect inferences(school gender example).
Modelling the variance(in addition to the mean)
gives a framework that allows a greater range of
questions. For example, how does variability in
parental treatment of sibs partition between and
within families? Does the within family variance
change as a function of social class?(As in the
differential parenting example)
Multilevel models extend to handle situations
where there are multiple classifications arranged
in nested, crossed and multiple membership
relations. For example in the social relations
model with relationship score, actor, partner,
dyad, family and genetic effects.
53
Other predictor variables
Remember we are partitioning the variability in
attainment over time between primary school,
residential area, pupil, p. school cohort,
teacher and occasion. We also have predictor
variables for these classifications, eg pupil
social class, teacher training, school budget and
so on. We can introduce these predictor variables
to see to what extent they explain the
partitioned variability.