Probability

1
Probability

• Sample Space (S)
• Collection of all possible outcomes of a random
  experiment
• Sample Point
• Each outcome of the experiment (or)
• element in the sample space
• Events are Collection of sample points
• Ex Rolling a die (six sample points), Odd number
  thrown in a die (three sample point a subset),
  tossing a coin (two sample points head,tail)

2
Probability

• Null Event (No Sample Point)
• Union (of A and B)
• Event which contains all points in A and B
• Intersection (of A and B)
• Event that contains points common to A and B
• Law of Large Numbers
• N number of times the random experiment is
  repeated
• NA- number of times event A occurred

3
Probability

• Properties

4
Probability

• Conditional Probability
• Probability of B conditioned by the fact that A
  has occurred
• The two events are statistically independent if

5
Probability

• Bernoullis Trials
• Same experiment repeated n times to find the
  probability of a particular event occurring
  exactly k times

6
Random Signals

• Associated with certain amount of uncertainty and
  unpredictability. Higher the uncertainty about a
  signal, higher the information content.
• For example, temperature or rainfall in a city
• thermal noise
• Information is quantified statistically
• (in terms of average (mean), variance, etc.)
• Generation
• Toss a coin 6 times and count the number of heads
  
• x(n) is the signal whose value is the number of
  heads on the nth trial

7
Random Signals

• Mean
• Median Middle or most central item in an ordered
  set of numbers
• Mode Maxxi
• Variance
• Standard Deviation
• measure of spread or deviation from the mean

8
Random Variables

• Probability is a numerical measure of the outcome
  of the random experiment
• Random variable is a numerical description of the
  outcome of a random experiment, i.e., arbitrarily
  assigned real numbers to events or sample points
• Can be discrete or continuous
• For example head is assigned 1
• tail is assigned 1 or 0

9
Random Variables

• Cumulative Distribution Function (CDF)
• Properties
• Probability Density Function (PDF)
• Properties

10
Important Distributions

• Binary distribution (Bernoulli distribution)
• Random variable has a binary distribution
• Partitions the sample space into two distinct
  subsets A and B
• All elements in A are mapped into one number say
  1 and B to another number say 0.

11
Important Distributions

• Binomial Distribution
• Perform binary experiment n times with outcome
  X1,X2,Xn, if , then X has
  binomial distribution

12
Important Distributions

• Uniform Distribution
• Random variable is equally likely
• Equally Weighted pdf

13
Important Distributions

• Poisson Distribution
• Random Variable is Poisson distributed
  with parameter m with
• Approximation to binomial with p ltlt 1,
  and k ltlt 1, then

14
Important Distributions

• Gaussian Distribution
• Normalized Gaussian pdf - N(0,1)
• Zero mean, Unit Variance

15
Important Distributions

• Normalized Gaussian pdf

16
Joint and Conditional PDFs

• For two random variables X and Y

17
Joint and Conditional PDFs

• Marginal pdfs
• Conditional pdfs

18
Expectation and Moments

• Centralized Moment
• Second centralized moment is variance

19
Expectations and Moments

• (i,j) joint moment between random variables X and
  Y

20
Expectations and Moments

• (i,j) joint central moment

21
Expectations and Moments

• Auto-covariance
• Characteristic Function (moment generator)

22
Random Process

• If a random variable X is a function of another
  variable, say time t, x(t) is called random
  process
• Collection of all possible waveforms is called
  the ensemble
• Individual waveform is called a sample function
• Outcome of a random experiment is a sample
  function for random process instead of a single
  value in the case of random variable

23
Random Process

• Random Process X(.,.) is a function of time
  variable t and sample point variable s
• Each sample point (s) identifies a function of
  time X(.,s) referred as sample function
• Each time point (t) identifies a function of
  sample points X(t,.), i.e., a random variable
• Random or Stochastic Processes can be
• continuous or discrete time process
• continuous or discrete amplitude process

24
Random Process

• Ensemble statistic Ensemble average at a
  particular time
• Temporal average for a sample function
• Random Process Classifications
• Stationary Process Statistical characteristics
  of the sample function do not change with time
  (time-invariant)

25
Random Process

• Second Order joint pdf
• Autocorrelation is a function of only time
  difference
• Wide Sense (or Weak) Stationary
• Independent of time up to second order only
• Ergodic Process
• Ensemble average time average

26
Random Process

• Mean
• Mean of the random process at time t is the mean
  of the random variable X(t)
• Autocorrelation
• Auto-covariance

27
Random Process

• Cross Correlation and covariance
• Power Density Spectrum

28
Random Process

• Total Average Power