? Random Variables and Distributions

1
CHAPTER 3

2
CHAPTER 3
Overview

• ? Random Variables and Distributions
• ? Continuous Distributions
• ? The Distribution Function

3

• ? Bivariate Distributions
• ? Marginal Distributions
• ? Conditional Distributions
• ? Functions of a Random Variable

4
So We are Skipping

• Sec3.7 Multivariate Distributions
• Sec3.9 Functions of Two or More Random
  Variables

5
Section 3.1Random Variables and Discrete
Distributions
6
Definition

• A Random Variable X is a real-valued function
  defined on the sample space S of an
  experiment. X S ? R
• That is, for every outcome in S is associated
  with a real number.

7
Examples

• Toss a coin twice, the sample space is
  SHH,HT,TH,TT.
• We can define a random variable on this sample
  space in many ways, for example1) let X be the
  number of heads obtained.2) let Y be the number
  of heads minus the number of tails.

8
More Examples

• 3) Toss a coin repeatedly until we see a head.
• The number of tosses needed until we observe a
  head for the first time is a random variable.

9

• 4) Consider the experiment of throwing a dart
  at a rectangular wall.Let X be the x-coordinate
  of the location hit by the dart.
• 5) Suppose that a school bus arrives between 730
  and 745 am.We can regard the exact arrival time
  of the bus as a random variable.

10
Definitions

• A random Variable is a called discrete if it can
  only assume a finite or countable number of
  events.
• A random Variable is a called continuous if its
  values fill in an entire interval (possibly of
  infinite length)

11
Distribution of a Discrete random Variable

• The distribution of a discrete random variable is
  a formula or table that lists all the possible
  values that the random variable can take,
  together with the corresponding probabilities.

12
Requirements

• A probability distribution must satisfy
• 1) P(Xi) 0 for every possible i
• 2) ?x P(Xx) equals one.

13
Example 1 The Uniform Distribution On the
Integers

• Consider a set of integers, say 1,2,3,,N.
• The uniform distribution on this set assigns
  the same weight/probability to each outcome,
  namely 1/N.
• A random variable that has this distribution is
  referred to as a discrete uniform r.v.

14
Example 2 The Binomial Distribution

• Definitions
• an experiment that results in one of two possible
  values is called a Bernoulli experiment. We will
  refer to this experiment as a trial.
• We will refer to the two possible outcomes of a
  single Bernoulli trial as S (for Success) or F
  (for Failure)
• The probabilities of S and F are p and q.
• A Bernoulli process is a process of repeating a
  Bernoulli trial many times independently.

15
Definition

• Consider a Bernoulli process with a fixed number
  of trials n
• The number of successes X in these n trials is a
  random variable, which we call binomial
• We write X b(n,p).
• Formula P(Xx)Cn,x pxqn-x
• Where x0,1,2,n

16
Geometric Distribution

• Consider a Bernoulli Process that goes on and on
  until a success is encountered for the first
  time.
• The number of trials X, including the last
  success, we needed to make is called a geometric
  random variable.
• Formula P(Xx) pqx-1, x1,2,3,

17
Negative Binomial Distribution

• Consider a Bernoulli process again.
• We will stop the process as soon as we
  encounter a success for the kth time.
• The number of trials needed is a random variable,
  which we call a negative binomial.
• FormulaP(Xx)Cx-1,k-1pkqx-k x k,k1,.

18
Hypergeometric Distribution

• A box contains N items, of which D are defective.
  We select k items randomly, where n min( D, N -
  D).
• The number of defective items X we randomly
  picked among the k items selected is called a
  hypergeometric random variable.
• Possible values of X 0,1,,n
• Probability Function
• P(x) (D choose x)(N-D choose n-x)/(N choose n)

19
Example

• Consider a well-shuffled regular deck of cards.
  Draw the cards successively and without
  replacement. Let X be the number of cards drawn
  by the time when the first ace appears.
• Is X geometric?
• Repeat the problem so that X is the number of
  cards when the fourth ace appears.
• Is X negative binomial?

20

• Section 3.2
• Continuous Distributions

21
The Probability Density Function

• Recall for a continuous random variable X the
  probability P(Xx)0 for any specific value x.
• So in the continuous case we calculate the
  probability that X falls in some set or
  interval.
• The right object that characterizes probabilities
  in the continuous case is the density f(x) of the
  random variable (probability density function or
  p.d.f. for short).
• f(x) 0
• ?-8, 8 f(x)dx 1

22
The Uniform Distribution on an Interval a,b

• This distribution assigns the same probability to
  intervals with the same width.
• It is a generalization of the discrete uniform
  probability function which assigns the same
  probability to all possible values.
• The p.d.f. is given by 1/(b-a) for x in a,b and
  zero elsewhere.
• Interpretation the mass is uniformly / evenly
  distributed in the interval a,b.

23
Example

• A random variable X has the f(x) cx2 for 1 x
  2 and zero otherwise.
• Find the value of the constant c and sketch the
  p.d.f.
• Find the value of P(X gt 2).
• The constant c is called the normalizing
  constant, and if it exists it is unique.

24

• Section 3.3
• The Distribution Function

25
Definition

• The Distribution function of a random variable is
  defined for every real number asF(x) P(X x)
• F(x) is called the cumulative distribution
  function.
• It is defined (as we will see) for all types of
  random variables, discrete and continuous.
• Properties
• F(-8)0, F(8)1.
• F(x) is nondecreasing (as x increases)
• F(x) is right-continuous, i.e. F(x)F(x)

26
Useful Facts about the Distribution Function

• P(Xgtx)1-F(x)
• P(Xltx) F(x-)
• P(X x) F(x) - F(x-).

27
Relation with the p.d.f.

• Consider a distribution function F(x) that has a
  derivative
• Hence F is necessarily the distribution of a
  continuous random variable.
• the p.d.f. is defined to be the derivative of
  F(x).

28
The Quantile Function

• If F(x) is continuous, then it is always
  increasing and so it is invertible.
• The inverse function xF-1(p) 0 p 1 is
  called the quantile function, it provides the pth
  quantile of the distribution.
• EXAMPLE the median of the distribution is the
  0.5th quantile m F-1(1/2).
• If F(x) is the d.f. of a discrete r.v. then
• F-1(p) is defined to be the least value x such
  that F(x) p.

29

• Section 3.4
• Bivariate Distributions

30
Section Abstract

• We generalize the concept of distribution of a
  single random variable to the case of two r.v.s
  where we speak of their joint distribution.
• We do so by introducing
• The joint p.f. of two discrete r.v.s
• The joint p.d.f. of two continuous r.v.s
• The joint d.f. for any two r.v.s

31
Discrete Joint Distributions

• Definition by example
• A subcommittee of five members is to be formed
  randomly from among a committee of 10 democrats,
  8 republicans and 2 independent members.
• Let X and Y be the numbers of democrats and
  republicans chosen to serve on the committee,
  respectively. Their joint p.f. is
• P(x,y)P(X x,Y y)
• C10,xC8,yC2,5-x-y / C20,5
• X 0,1,,5 y 0,1,,5 xy 5.

32
Requirements

• P(x,y) 0
• Sx Sy p(x,y)1 where the double sum is taken
  over all possible values of (x,y) in the
  xy-plane.
• The joint probability describes the joint
  behavior of two random variables.

33
Another Example of Joint p.f.

• Throw a die once, let X be the outcome. Throw a
  fair coin X times and let Y be the number of
  heads. Find P(x,y)?
• Solution
• The possible X values range from 1 to 6.
• The possible Y values range from 0 to 6.
• When X 2 (e.g.) Y 0,1 or 2.
• P(2,0) P(X2,Y0) P(X2)P(Y0X2)
• (1/6)(1/2)21/24.

34
Contd Table of p(x,y)
y x 0 1 2 3 4 5 6 sum
1 1/12 1/12 0 0 0 0 0 1/6
2 1/24 2/24 1/24 0 0 0 0 1/6
3 1/48 3/48 3/48 1/48 0 0 0 1/6
4 1/96 4/96 6/96 4/96 1/96 0 0 1/6
5 1/192 5/192 10/192 10/192 5/192 1/192 0 1/6
6 1/384 6/384 15/384 20/384 15/384 6/384 1/384 1/6
sum 63/384 120/384 99/384 64/384 29/384 8/384 1/384
35
Die and Coin Continued

• Questions
• Find P(Xgt4,Ygt3)
• Find P(Y3)
• Solution
• P(Xgt4,Ygt3)P(5,4)P(5,5) P(6,4)P(6,5)P(6,6)
• P(Y3)P(1,3) P(2,3) P(3,3) P(4,3) P(5,3)
  P(6,3)
• 0 0 P(3,3) P(4,3) P(5,3)
  P(6,3)

36
Continuous Joint Distributions

• A real valued function f(x,y) is a joint
  probability density function (p.d.f.) if
• f(x,y) 0
• ? ? f(x,y)dxdy1 where integration is performed
  over the entire xy plane.
• A p.d.f. describes the joint behavior of two
  continuous random variables
• For any region A in the xy plane, the probability
  that (X,Y) falls within A is the volume of the
  solid under the graph of f(x,y) and above the
  region A.
• Hence the problem of finding the probability is a
  question about finding a double integral.

37
Examples

• Find the constant c that renders the function
  f(x,y) into a joint p.d.f.
• f(x,y) cxy2 0 x 1 0 y 1 AND
  zero elsewhere
• f(x,y) cxy2 0 x y 1 AND zero
  elsewhereFind P(Xgt3/4) in each case.

38
Will the Chicken Be Safe?

• A farmer wants to build a triangular pen for his
  chickens. He sends his son out to cut the lumber
  and the boy, without taking any thought as to the
  ultimate purpose, makes two cuts at two points
  selected at random. What are the chances that the
  resulting three pieces can be used to form a
  triangular pen?

39
Bivariate Distribution Function

• F(x,y) P(Xx, Yy)
• F(-8,-8)0 F(8,8)1
• F is nondecreasing in both x and y.
• Calculating Probability that (X,Y) belongs to a
  given rectangle Using d.f.
• P(altXb and cltYd)
• P(altXb and Yd) - P(altXb and Yc)
• P(Xb and Yd)- P(Xa and Yd)
• -P(Xb and Yc)- P(Xa and Yc)
• F(b,d)-F(a,d)-F(b,c)F(a,c)

40
Calculating joint p.d.f. from joint d.f.

• If both X and Y are continuous
• Alternatively, if F(x,y) has partial derivatives
  in both x and y,
• Then the joint p.d.f. is
• f(x,y) ?F(x,y)/?x?y
• Conversely,
• F(x,y)??f(s,t)dsdt
• where the integrals are from (-8,y) and
• (-8,x) respectively.

41

• Section 3.5
• Marginal Distributions

42
Definition

• Let X,Y have the joint distribution F(x,y).
• i) The marginal distribution of X is given by
• FX(x) F(x,8)
• ii) The marginal distribution of Y is given by
• FY(y)F(8,y).
• That is, the marginal d.f. of X (resp.Y) is the
  same as the univariate d.f. of X (resp. Y)
  recovered from the joint d.f. of X and Y.

43
Discrete Marginals

• Let p(x,y) be the joint p.f. of X and Y.
• The marginal p.f.s of X and Y are denoted by
  pX(x) and pY(y) respectively, where pX(x)
  ?yp(x,y)
• pY(y) ?xp(x,y)

44
Example Table of p(x,y)
x y 0 1 2 3 4 5 6 pX(x)
1 1/12 1/12 0 0 0 0 0 1/6
2 1/24 2/24 1/24 0 0 0 0 1/6
3 1/48 3/48 3/48 1/48 0 0 0 1/6
4 1/96 4/96 6/96 4/96 1/96 0 0 1/6
5 1/192 5/192 10/192 10/192 5/192 1/192 0 1/6
6 1/384 6/384 15/384 20/384 15/384 6/384 1/384 1/6
pY(y) 63/384 120/384 99/384 64/384 29/384 8/384 1/384 1
45
Two Continuous Random Variables

• X and Y continuous r.v. with joint p.d.f.
  f(x,y).
• The marginal p.d.f. of X is
• fX(x) -8?8f(x,y)dy
• The marginal p.d.f. of Y is
• fY(y) -8?8f(x,y)dx

46
Independent Random Variables

• Def. Two Random Variables are independent iff
  their joint d.f. is the product of their marginal
  d.f.s
• F(x,y) FX(x)FY(y)
• Equivalently, X and Y are indep. iff
• f(x,y) fX(x)fY(y)
• That is, X and Y are indep if any two events A
  and B determined by X and Y respectively are
  independent events
• e.g. A Xgt2 B0 lt Y 12.

47
Checking two discrete r.v. for Independence

• From the table, (or formula) verify thatP(Xx,
  Yy) P(Xx) P(Yy)
• If this is the case for all possibilities, then
• X and Y are independent
• If this equation is violated at least once then X
  and Y are dependent
• REMARK in the case of indep. The rows are all
  proportional (and so are the columns!)

48
Discrete Example 1 dependent
x y 0 1 2 3 4 5 6 pX(x)
1 1/12 1/12 0 0 0 0 0 1/6
2 1/24 2/24 1/24 0 0 0 0 1/6
3 1/48 3/48 3/48 1/48 0 0 0 1/6
4 1/96 4/96 6/96 4/96 1/96 0 0 1/6
5 1/192 5/192 10/192 10/192 5/192 1/192 0 1/6
6 1/384 6/384 15/384 20/384 15/384 6/384 1/384 1/6
pY(y) 63/384 120/384 99/384 64/384 29/384 8/384 1/384 1
49
Discrete Example 2
X \ y 1 2 3 4 total
1 .06 .02 .04 .08 .20
2 .15 .05 .10 .20 .50
3 .09 .03 .06 .12 .30
Total .30 .10 .20 .40 1.00
50
Checking two continuous r.v. for Independence

• No table available ?, only formula of the joint
  p.d.f.
• Get marginal p.d.f. of both X and Y, that is
  fX(x) and fY(y).
• Verify that f(x,y) fX(x)fY(y)

51

• Section 3.6
• Conditional Distributions

52
Section Abstract

• Now that we have defined the concepts of marginal
  distributions and independent random variables,
  we proceed to generalize the concept of
  conditional probability to conditional
  distributions.
• That is, we will be able to speak about the
  distribution of a r.v. X given that of another
  r.v. Y just as we defined probability of event A
  given that event B occurred.

53

• Discrete CaseP(XxYy)
• Pr(Xx,Yy)/P(Yy)
• p(x,y)/pY(y).Provided that pY(y) ? 0.
• Continuous Case
• Condit. Dist. Of X given Yg1(xy) f(x,y) /
  fY(y)
• Condit. Dist. Of Y given X
• g2(yx) f(x,y) / fX(x)

54
Example Conditional Distribution-Discrete Case

• Each student at a high school was classified
  according to his/her year in school and according
  to the number of times that he/she had visited a
  certain museum.
• If a student selected randomly is a junior, what
  is the prob. That he/she has visited the museum
  exactly once?
• If a student selected randomly has visited the
  museum twice, what is the probability that she is
  a senior?

Never Once More than once
Fresh-man 0.08 0.10 0.04
Sophomore 0.04 0.10 0.04
Juniors 0.04 0.20 0.09
Seniors 0.02 0.15 0.10
55
Example Conditional Distribution-Cont. Case

• Let (X,Y) be a point selected at random from the
  inside of the circle
• (x-1)2y24
• Find the conditional density of X given Y.
• Are X and Y independent...by the way?

56

• Section 3.8
• Functions of a Random Variable

57
Section Abstract

• In this section we learn the general methodology
  of how to find the distribution/density of a
  random variable Y defined on top of another
  random variable X whose distribution is known to
  us.
• That is, if Yr(X), where r(.) is a given
  function, then how is the r.v. Y distributed?

58
Transforming a Discrete R.V.

• X is a discrete r.v. that takes the values
  -2,-1,0,1,and 2 according to the table.
• We define Y X2.
• Y takes the values 0,1 and 4 with the
  probabilities given in lower table.

X -2 -1 0 1 2
p(x) 0.2 0.1 0.3 0.15 0.25
y 0 1 4
p(y) 0.3 0.25 0.45
59
Transforming Discrete r.v. in General

• Let Yr(X) be a deterministic function from R to
  R.
• First note that if r(.) is one to one onto
  function then Y will have the same distribution
  as X, on the set of new values of Y.
• Otherwise, r(.) has the effect of grouping
  different values of X as one value of Y
• g(y) Pr(Yy) Pr(r(X) y) ? f(x)
• where the sum is taken over all x such that r(x)
  y.

60
Transforming a Continuous Distribution

• Let X be a uniform random variable U0,1, and
  let Y X2 . How do we find the p.d.f. g(y) of Y?
• Outline
• First we will find the d.f. G(y) of Y in terms of
  the d.f. F(x) of Xwhich we know!
• Then we will differentiate G(y) to discover the
  density g(y).
• Contd

61
Contd. Transforming a Continuous Distribution

• G(y) Pr(Y y)
• Pr(X2 y)
• Pr(0 X vy)
• F(vy) - F(0)
• vy - (0) (because F(x) x for the uniform
  U0,1).
• Hence we obtain G(y) vy
• Differentiating we get
• g(y) 1/2vy for y in 0,1 and zero elsewhere