Nuts and Bolts cont'

1
Nuts and Bolts cont.

• 1. Calculus Review
• 2. Random Variables and Probability Density
  Functions (PDFs)
• 3. Cumulative Distribution Functions (CDFs)

2
Introduction to Calculus

• - Differentiation
• - Partial Differentiation
• - Basic Tools of Integration
• - Multiple Integration

3
The concept of a derivative
The derivative of a function f is the function f
defined by

• In class, we may also use the notation dy / dx.
• - Thinking carefully about the equation, we see
  that the derivative provides the measure of the
  slope of a function.
• To see this, slope ?y/?x
• Rewritten in terms of the above equation, slope
  ?f(x) /?x
• The change in f due to an arbitrarily small shift
  in x (as measured by h) is f( x h ) - f( x )
• The change in x is measured by h.

4
Rules of Differentiation

• Constant Rule If f(x) k, then f(x) 0.
• e.g. Suppose f(x) 3. What is f(x)?
• Power Function Rule. If f(x) cxn, then f(x)
  cnxn-1
• e.g. Suppose f(x) 3x2, what is f(x)?
• Sum-Difference Rule.
• If f(x) g(x) h(x), then f(x) g(x)
  h(x)
• e.g. Suppose f(x) x2 3x3, what is f(x)?
• e.g. Suppose f(x) 17 4x, what is f(x)?

5
More rules of Differentiation

• Product Rule
• If f(x) g(x)h(x), then f(x) g(x)h(x)
  g(x)h(x)
• e.g. Suppose f(x) (4x3)(5-x2). What is f(x)?
• Quotient Rule
• If f(x) g(x) / h(x), then f(x)
  g(x)h(x)-g(x)h(x) / h(x)2
• e.g. Suppose f(x) 2x2 / (x-2). What is f(x)?

6
More rules of differentiation

• Log Rule If f(x) ln( g(x) ), then f(x)
  g(x) / g(x)
• e.g. Suppose f(x) ln(x). What is f(x)?
• e.g. Suppose f(x) log(3x x2). What is f(x)?
• Exponential-Function Rule
• If f(x) eg(x), then f(x) g(x)eg(x)
• e.g. Suppose f(x) e3x, what is f(x)?

7
The concept of a partial derivative

• Suppose you have a function y f(x1,,xn). Then
  the partial derivative is the equation

The partial derivative of f with respect to x1 is
the effect of an infinitely small shift in x1
(again measured by h) on f. - It is the
multivariate extension of the concept of the
slope. To compute a partial derivative with
respect to x1, simply treat x2, , xn as
constants, and apply the standard rules of
differentiation
8
Examples of Partial Derivatives

• Identify the partial derivatives of f with
  respect to x1 and x2 for the following equations?
• Example 1. f(x1,x2) 4x12 3x2.
• Example 2. f(x1,x2) ln(x1x2).

9
Integration

• Whereas the concept of a derivative stemmed from
  the need to compute the slope of a function f(x),
  integral calculus emerged from the need to
  identify the area between a function f(x) and the
  x-axis.
• For example, suppose you wanted to know the area
  under the function f(x) 2 on the range from 0
  to 10. Then, a numerical solution for this
  integral obviously exists, equals to 20.

f(x)
2
x
0
10
10
Integration cont.

• The integral of f(x) is defined as F(x) ? f(x)
  dx.
• For reasons that remain obscure to me, F(x)
  represents the anti-derivative of the function
  f(x). In other words,
• F(x) f(x)
• Therefore, a set of rules symmetric to those we
  used for differentiation apply to integration.

11
Rules of Integration

• Rule 1) ? a dx ax c
• e.g. What is ? 2 dx 2x c ?
• Rule 2) ? a f(x) dx a? f(x) dx
• e.g. What is ? 17 x3 dx ?
• Rule 3) ? (uv) dx ? u dx ? v dx ux vx
  c
• e.g. What is ? (413) dx ?
• Rule 4) ? xn dx xn1 / (n1) c
• e.g. What is ? x3 dx ?
• Note that for each of these rules, we must add a
  constant of integration.
• To compute the area under the curve between two
  boundaries, we substitute the maximum boundary
  into the equation and subtract that number from
  the quantity found when we substitute the minimum
  boundary into the equation.

12
Multiple Integrals

• Sometimes, rather than computing the area under a
  curve, you will want to compute the volume, or
  even the hyper-volume. In these cases, you will
  need to integrate over multiple dimensions.
• For a function f(x,y), its integral F(x,y) is
  defined
• F(x,y) ?? f(x,y) dx dy
• To solve for this double integral is pretty easy
  conceptually. It is legitimate to rewrite the
  equation
• F(x,y) ?? f(x,y) dx dy ? ? f(x,y) dx dy
• So, all you have to do is compute the integral of
  f(x,y) with respect to x and then with respect to
  y. If there are limits of integration for x,
  these can be substitute in after the first
  integral is calculated to make the calculation
  even easier.
• And for a function f(x, y, , z), the integral
  F(x,y,,z) is defined F(x,y,,z) ???
  f(x,y,z) dx dydz

13
Example of Solving A Multiple Integral

• What is the multiple integral of f(x,y) x2y ?
• Answer ?? f(x,y) ?? x2y dx dy ?
• ?? x2y dx dy ? x3y/3 c dy x3y2/6 cy
  d

14
Univariate Probability Models

• Definition of a random variable
• PDFs
• CDFs

15
Random Variables and Discrete Distributions

• Let S define the sample space for an experiment.
• A random variable is a real-valued function that
  is defined on the space S.
• ? This definition is obtuse. It simply means
  that each element of S can be represented by some
  number.
• ? note that for a real-valued function y f(x),
  real-valued just means that y is a number between
  -? and ?.
• Example If your experiment was to flip a fair
  coin, the sample space would be heads or tails. A
  random variable X would be the assignment of 1 to
  the heads outcome and 0 to the tails outcome.

16
The Distribution of a Random Variable

• If we have a probability distribution over the
  sample space, we may also have a probability
  distribution for the random variable.
• Example 1 If we flip a fair coin once, our
  sample space is a head (X1) or a tail (X0). The
  probability distribution states that Pr(X1).5
  and Pr(X0).5
• Example 2 If we flip a fair coin twice, there
  are four possible outcomes in our sample space.
  The probability distribution states that
  Pr(X2).25 Pr(X1).5 Pr(X0).25.

17
Probability Density Functions (pdfs)The
Discrete Case

• For a discrete random variable X, the pdf of X is
  the function f such that for any x
• f(x) Pr(X x).
• In other words, the pdf states the probability of
  observing each possible value of X. If x is not a
  possible value for X, then f(x) 0.
• Example Suppose that the random variable X
  represents a fair coin toss where the probability
  of heads (X1) .5. Then the pdf of X is
• f(x) .5x (.5)1-x for x 1,0
• f(x) 0 otherwise

18
Probability Density Functions (pdfs)The
Continuous Case

• For a continuous random variable X, the pdf of X
  is the function f such that for any x
• f(x) Pr(X x).
• If x is outside the sample space, Pr(x) 0.
• A peculiar property of pdfs is that the
  probability of any x equals zero. Why?
• Consequently, we must identify the probability of
  a range of values of X. Thus,
• Pr(X?A) ?A f(x) dx
• Further, we know that ?Real Line f(x) dx 1

19
More on continuous pdfs

• Example 1. Suppose that X is a random variable
  that follows a uniform pdf between zero and one.
• What is the pdf of X?
• Example 2. Suppose that X is a random variable
  that follows a uniform pdf between zero and five.
• What is the pdf of X?

20
More on continuous pdfs

• Example 1. Suppose that X is a random variable
  that follows a uniform pdf between zero and one.
• f(x) 1 if 0 ? x ? 1
• f(x) 0 otherwise
• Example 2. Suppose that X is a random variable
  that follows a uniform pdf between zero and five.
• f(x) 1/5 if 0 ? x ? 5
• f(x) 0 otherwise

21
Cumulative Distribution Functions (cdfs)

• The cumulative distribution function F of a
  random variable X is a function defined for each
  x as follows
• F(x) Pr( X ? x )
• The cdf simply states the probability that the
  random variable takes a value less than x.
• Properties of the CDF (draw figures to
  illustrate)
• P1. If x1 lt x2, F(x1) lt F(x2)
• P2. For any given x, Pr(X gt x) 1 F(x)
• P3. For any given x, Pr(x1 lt X lt x2) F(x2)-
  F(x1)

22
Cumulative Distribution Function Example

• Example Suppose that the random variable X
  represents a fair coin toss where the probability
  of heads (X1) .5.
• What is the cdf of X?

23
Cumulative Distribution Function Example

• Example Suppose that the random variable X
  represents a fair coin toss where the probability
  of heads (X1) .5.
• What is the cdf of X?
• F(x) 0 if x lt 0
• F(x) .5 if 0 ? x lt 1
• F(x) 1 if x ? 1

1
X
0
-1
0
1
2
24
CDFs for Discrete Variables

• The CDF for a discrete random variable X can be
  written
• F(xj) ?i?j f(xi)

This is the pdf for X
Because of the use of summations, it is clear
that the cdf for a discrete random variable is
discontinuous.
25
CDFs for Continuous Variables

• The CDF for a continuous random variable X can be
  written
• F(x) ?-? to x f(t) dt

This is the pdf for X
Notice how the notation has become cumbersome,
with ts standing in for xs.
26
Cumulative Distribution Function Example

• Example. Suppose that X is a random variable that
  follows a uniform pdf between zero and five.
• f(x) 1/5 if 0 ? x ? 5
• f(x) 0 otherwise
• What is the cumulative distribution function
  F(x)?
• What is the probability that X lt 2 ?

27
Cumulative Distribution Function Example

• Example. Suppose that X is a random variable that
  follows a uniform pdf between zero and five.
• f(x) 1/5 if 0 ? x ? 5
• f(x) 0 otherwise
• What is the cumulative distribution function?
• F(x) ?0 to X 1/5 dt
• F(x) t/5 0 to X x/5 0 x/5

What is the probability that X lt 2? F(2) 2/5
.4