Multivariate distributions

1
Multivariate distributions
2
The Normal distribution
3
1.The Normal distribution parameters m and s
(or s2)

• Comment If m 0 and s 1 the distribution is
  called the standard normal distribution

4
The probability density of the normal
distribution
If a random variable, X, has a normal
distribution with mean m and variance s2 then we
will write
5
The multivariate Normal distribution
6

• Let

a random vector
Let
a vector of constants (the mean vector)
7

• Let

a p p positive definite matrix
8
Definition

• The matrix A is positive semi definite if

Further the matrix A is positive definite if
9
Suppose that the joint density of the random
vector
The random vector, x1, x2, xp is said
to have a p-variate normal distribution with mean
vector and covariance matrix S We will
write
10
Example the Bivariate Normal distribution
with
and
11
Now
and
12
(No Transcript)
13
Hence
where
14
Note
is constant when
is constant. This is true when x1, x2 lie on an
ellipse centered at m1, m2 .
15
(No Transcript)
16
Surface Plots of the bivariate Normal distribution
17
Contour Plots of the bivariate Normal distribution
18
Scatter Plots of data from the bivariate Normal
distribution
19
Trivariate Normal distribution - Contour map
x3
mean vector
x2
x1
20
Trivariate Normal distribution
x3
x2
x1
21
Trivariate Normal distribution
x3
x2
x1
22
Trivariate Normal distribution
x3
x2
x1
23
example

• In the following study data was collected for a
  sample of n 183 females on the variables

• Age,
• Height (Ht),
• Weight (Wt),
• Birth control pill use (Bpl - 1no pill, 2pill)
• and the following Blood Chemistry measurements
• Cholesterol (Chl),
• Albumin (Abl),
• Calcium (Ca) and
• Uric Acid (UA). The data are tabulated next page

24
The data
25
(No Transcript)
26
Alb, Chl, Bp
27
Marginal and Conditional distributions
28
Theorem (Marginal distributions for the
Multivariate Normal distribution)
have p-variate Normal distribution
with mean vector
and Covariance matrix
Then the marginal distribution of is
qi-variate Normal distribution (q1 q, q2 p -
q)
with mean vector
and Covariance matrix
29
Theorem (Conditional distributions for the
Multivariate Normal distribution)
have p-variate Normal distribution
with mean vector
and Covariance matrix
Then the conditional distribution of given
is qi-variate Normal distribution
with mean vector
and Covariance matrix
30
Proof (of Previous two theorems)
is
The joint density of
,
and
where
31
where
,
and
32
also
and
,
33
,
34
(No Transcript)
35
The marginal distribution of is
36
The conditional distribution of given
is
37
is called the matrix of partial variances and
covariances.
is called the partial covariance (variance if i
j) between xi and xj given x1, , xq.
is called the partial correlation between xi and
xj given x1, , xq.
38
is called the matrix of regression coefficients
for predicting xq1, xq2, , xp from x1, , xq.
Mean vector of xq1, xq2, , xp given x1, ,
xqis
39
Example
Suppose that
Is 4-variate normal with
40
The marginal distribution of
is bivariate normal with
The marginal distribution of
is trivariate normal with
41
Find the conditional distribution of
given
Now
and
42
(No Transcript)
43
The matrix of regression coefficients for
predicting x3, x4 from x1, x2.
44
(No Transcript)
45
Thus the conditional distribution of
given
is bivariate Normal with mean vector
And partial covariance matrix
46
Using SPSS
Note The use of another statistical package such
as Minitab is similar to using SPSS
47

• The first step is to input the data.

The data is usually contained in some type of
file.

• Text files
• Excel files
• Other types of files

48
After starting the SSPS program the following
dialogue box appears
49
If you select Opening an existing file and press
OK the following dialogue box appears
50
Once you selected the file and its type
51
The following dialogue box appears
52
If the variable names are in the file ask it to
read the names. If you do not specify the Range
the program will identify the Range
Once you click OK, two windows will appear
53
A window containing the output
54
The other containing the data
55
To perform any statistical Analysis select the
Analyze menu
56
To compute correlations select Correlate then
BivariateTo compute partial correlations select
Correlate then Partial
57
for Bivariate correlation the following dialogue
appears
58
the output for Bivariate correlation
59
for partial correlation the following dialogue
appears
60
the output for partial correlation
- - - P A R T I A L C O R R E L A T I O N C
O E F F I C I E N T S - - - Controlling for..
AGE HT WT CHL
ALB CA UA CHL 1.0000
.1299 .2957 .2338 (
0) ( 178) ( 178) ( 178)
P . P .082 P .000 P .002 ALB
.1299 1.0000 .4778 .1226
( 178) ( 0) ( 178) ( 178)
P .082 P . P .000 P
.101 CA .2957 .4778 1.0000
.1737 ( 178) ( 178) (
0) ( 178) P .000 P .000
P . P .020 UA .2338
.1226 .1737 1.0000 ( 178)
( 178) ( 178) ( 0) P
.002 P .101 P .020 P . (Coefficient /
(D.F.) / 2-tailed Significance) " . " is printed
if a coefficient cannot be computed
61
Compare these with the bivariate correlation
62
Partial Correlations
CHL ALB CA
UA CHL 1.0000 .1299 .2957
.2338 ALB .1299 1.0000
.4778 .1226 CA .2957
.4778 1.0000 .1737 UA
.2338 .1226 .1737 1.0000
Bivariate Correlations
63

• In the last example the bivariate and partial
  correlations were roughly in agreement.
• This is not necessarily the case in all stuations
• An Example
• The following data was collected on the following
  three variables

• Age
• Calcium Intake in diet (CAI)
• Bone Mass density (BMI)

64
The data
65
Bivariate correlations
66
Partial correlations
67
Scatter plot CAI vs BMI (r -0.447)
68
(No Transcript)
69
3D Plot
Age, CAI and BMI
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
Independence
74
Note two vectors, , are
independent if
Then the conditional distribution of given
is equal to the marginal distribution of
If is multivariate Normal with
mean vector
and Covariance matrix
Then the two vectors, , are
independent if
75
The components of the vector, , are
independent if
s ij 0 for all i and j (i ? j )
i. e. S is a diagonal matrix
76
Transformations
77
Transformations
Theorem Let x1, x2,, xn denote random variables
with joint probability density function f(x1,
x2,, xn ) Let u1 h1(x1, x2,, xn).
u2 h2(x1, x2,, xn).
?
un hn(x1, x2,, xn).
define an invertible transformation from the xs
to the us
78
Then the joint probability density function of
u1, u2,, un is given by
where
Jacobian of the transformation
79
Example
Suppose that u1, u2 are independent with a
uniform distribution from 0 to 1 Find the
distribution of
Solving for u1 and u2 we get the inverse
transformation
80
also
and
Hence
81
The Jacobian of the transformation
82
(No Transcript)
83
The joint density of u1, u2 is f(u1, u2) f1
(u1) f2(u2)
Hence the joint density of z1 and z2 is
84
Thus z1 and z2 are independent Standard normal.
The transformation
is useful for converting uniform RVs into
independent standard normal RVs
85
Example
Suppose that x1, x2 are independent with density
functions f1 (x1) and f2(x2) Find the
distribution of u1 x1 x2
u2 x1 - x2
Solving for x1 and x2 we get the inverse
transformation
86
The Jacobian of the transformation
87
The joint density of x1, x2 is f(x1, x2) f1
(x1) f2(x2) Hence the joint density of u1 and u2
is
88

Theorem Let x1, x2,, xn denote random variables
with joint probability density function f(x1,
x2,, xn ) Let u1 a11x1 a12x2 a1nxn c1
u2 a21x1 a22x2 a2nxn c2
?
un an1 x1 an2 x2 annxn cn
define an invertible linear transformation from
the xs to the us
89
Then the joint probability density function of
u1, u2,, un is given by
where
90
then
has a p-variate normal distribution
with mean vector
and covariance matrix
91
then
has a p-variate normal distribution
with mean vector
and covariance matrix
92
Proof
then
93
since
and
Also
and
hence
QED
94
Theorem (Linear transformations of Normal RVs)
Suppose that The random vector, has a p-variate
normal distribution with mean vector and
covariance matrix S
with mean vector
and covariance matrix
95
proof
Let B be a (p - q) p matrix so that
is invertible.
then
is pvariate normal with mean vector
and covariance matrix
96
Thus the marginal distribution of
is qvariate normal with mean vector
and covariance matrix
97
Recall Definition of eigenvector, eigenvalue

• Let A be an n n matrix
• Let

then l is called an eigenvalue of A and
and is called an eigenvector of A and
98
Thereom If the matrix A is symmetric with
distinct eigenvalues, l1, , ln, with
corresponding eigenvectors
Assume
99
Then and covariance matrix S is positive
definite.
Suppose l1, , lp are the eigenvalues of S
corresponding eigenvectors of unit length
Note l1 gt 0, , lp gt 0
100
Let
101
(No Transcript)
102
(No Transcript)
103
then
has a p-variate normal distribution
with mean vector
and covariance matrix
104
Thus the components of
are independent normal with mean 0 and variance 1.
and
Has a c2 distribution with p degrees of freedom