Statistics and Probability in DSP

In DSP, Probability is used to describe the process that generated the acquired signals, and Statistics are used to characterize the signals. The 2 approaches work by measuring some parameters from the acquired signals.  

The 2 most important statistics values are the mean and the standard deviation.

The mean is a measure of the average of all samples of the signal. The standard deviation is a measure of the  average of how the signal samples deviate from the mean. The square of the standard deviation is called variance. In denoising applications, the square of the mean could be addressed as the energy of the useful signal part and the variance as the noisy (undesired) signal part.

In DSP, it is important to make the difference between the characteristics of the acquired signals and the ones  of the underlying process. In other words, a underlying process may describe a model of a expected case and a acquired signal, a sample of that model. A process which characteristics vary other time is called a   nonstationary process, otherwise a stationary process.

Other process characteristics are:

• Histogram: Group the samples of a acquired signal that have the same value. it represents he number of occurences of a value in the signal 
  
• The probability mass function (pmf) is the ideal histogram representation computed by assuming a infinite number of sample, this is the histogram of the underlying process. The pmf is computable for discrete signals. The pmf has a discrete form.
  
• The probability distribution function or probability density function (pdf) is the pmf version for continuous signal. In difference the values of the pdf are normalized by the sample number, providing probability values lying between 0 and 1 and the area under the curve sum to one (1). The pdf has a continuous shape.
  
• Binning is used in case to fix a convenient number of bins (groups) in a histogram in order to have a reasonable representation.
  
• Cumulative distribution function (cmf) represents the probability that a value of the pdf lies between a certain range of values.  
• A very important distribution in DSP is the normal distribution, also called Gauss distribution or simply Gaussian. The Gaussian has a bell like shape with tails that fall quick to zero. The Gaussian function is fully charaterized by the mean and the standard deviation which are respectively the center position and the width of the Gaussian. Since the Gaussian can not be integrated mathematically, a lookup table of its cmf is available for a standard Gaussian of mean 0 and variance 1. Its typical PHI function allows the computation of the probability for any range of values.
  
•  Another important distribution is the uniform distribution, where each value has the same theoretical probability to occur. 
• For experiments purposes, it may be necessary to simulate noisy signals or environments. "Random Number Generators" are useful to this task.
  
• In many applications, the parameters of the distribution are estimated. The estimator is the method uses to estimate the value. The estimated value generally differ from the true value due to differents reasons: system calibration, operation noise, etc. The error is qualified in term of accuracy and precision.
• The accuracy refers to systematic errors, while precision refers to random errors. 

@uthor: Joan MOUBA