Library of Congress Cataloging-in-Publication Data
Andrews, Larry C.
Field guide to probability, random processes, and random data analysis / Larry C. Andrews, Ronald L. Phillips.
p. cm. – (Field guide series)
Includes bibliographical references and index.
ISBN 978-0-8194-8701-8
1. Mathematical analysis. 2. Probabilities. 3. Random data (Statistics) I. Phillips, Ronald L. II. Title.
QA300.A5583 2012
519.2—dc23
2011051386
Published by
SPIE
P.O. Box 10
Bellingham, Washington 98227-0010 USA
Phone: +1.360.676.3290
Fax: +1.360.647.1445
Email: books@spie.org
Web: http://spie.org
The content of this book reflects the work and thoughts of the author. Every effort has been made to publish reliable and accurate information herein, but the publisher is not responsible for the validity of the information or for any outcomes resulting from reliance thereon.
Printed in the United States of America.
First printing
Introduction to the Series
Welcome to the SPIE Field Guides—a series of publications written directly for the practicing engineer or scientist. Many textbooks and professional reference books cover optical principles and techniques in depth. The aim of the SPIE Field Guides is to distill this information, providing readers with a handy desk or briefcase reference that provides basic, essential information about optical principles, techniques, or phenomena, including definitions and descriptions, key equations, illustrations, application examples, design considerations, and additional resources. A significant effort will be made to provide a consistent notation and style between volumes in the series.
Each SPIE Field Guide addresses a major field of optical science and technology. The concept of these Field Guides is a format-intensive presentation based on figures and equations supplemented by concise explanations. In most cases, this modular approach places a single topic on a page, and provides full coverage of that topic on that page. Highlights, insights, and rules of thumb are displayed in sidebars to the main text. The appendices at the end of each Field Guide provide additional information such as related material outside the main scope of the volume, key mathematical relationships, and alternative methods. While complete in their coverage, the concise presentation may not be appropriate for those new to the field.
The SPIE Field Guides are intended to be living documents. The modular page-based presentation format allows them to be easily updated and expanded. We are interested in your suggestions for new Field Guide topics as well as what material should be added to an individual volume to make these Field Guides more useful to you. Please contact us at fieldguides@SPIE.org.
John E. Greivenkamp, Series Editor
College of Optical Sciences
The University of Arizona
The Field Guide Series
Keep information at your fingertips with all of the titles in the Field Guide Series:
Field Guide to
Adaptive Optics, Robert Tyson & Benjamin Frazier
Atmospheric Optics, Larry Andrews
Binoculars and Scopes, Paul Yoder, Jr. & Daniel Vukobratovich
Diffractive Optics, Yakov G. Soskind
Geometrical Optics, John Greivenkamp
Illumination, Angelo Arecchi, Tahar Messadi, & John Koshel
Infrared Systems, Detectors, and FPAs, Second Edition, Arnold Daniels
Interferometric Optical Testing, Eric Goodwin & Jim Wyant
Laser Pulse Generation, Rüdiger Paschotta
Lasers, Rüdiger Paschotta
Microscopy, Tomasz Tkaczyk
Optical Fabrication, Ray Williamson
Optical Fiber Technology, Rüdiger Paschotta
Optical Lithography, Chris Mack
Optical Thin Films, Ronald Willey
Polarization, Edward Collett
Radiometry, Barbara Grant
Special Functions for Engineers, Larry Andrews
Spectroscopy, David Ball
Visual and Ophthalmic Optics, Jim Schwiegerling
Field Guide to Probability, Random Processes, and Random Data Analysis
Developed in basic courses in engineering and science, mathematical theory usually involves deterministic phenomena. Such is the case for solving a differential equation that describes a linear system where both input and output are deterministic quantities. In practice, however, the input to a linear system, such as imaging or radar systems, can contain a “random” quantity that yields uncertainty about the output. Such systems must be treated by probabilistic rather than deterministic methods. For this reason, probability theory and random-process theory have become indispensable tools in the mathematical analysis of these kinds of engineering systems.
Topics included in this Field Guide are basic probability theory, random processes, random fields, and random data analysis. The analysis of random data is less well known than the other topics, particularly some of the tests for stationarity, periodicity, and normality.
Much of the material is condensed from the authors’ earlier text Mathematical Techniques for Engineers and Scientists (SPIE Press, 2003). As is the case for other volumes in this series, it is assumed that the reader has some basic knowledge of the subject.
Larry C. Andrews
Professor Emeritus
Townes Laser Institute
CREOL College of Optics
University of Central Florida
Ronald L. Phillips
Professor Emeritus
Townes Laser Institute
CREOL College of Optics
University of Central Florida
Table of Contents
Glossary of Symbols and Notation x
Probability: One Random Variable 1
Terms and Axioms 2
Random Variables and Cumulative Distribution 3
Probability Density Function 4
Expected Value: Moments 5
Example: Expected Value 6
Expected Value: Characteristic Function 7
Gaussian or Normal Distribution 8
Other Examples of PDFs: Continuous RV 9
Other Examples of PDFs: Discrete RV 12
Chebyshev Inequality 13
Law of Large Numbers 14
Functions of One RV 15
Example: Square-Law Device 16
Example: Half-Wave Rectifier 17
Conditional Probabilities 18
Conditional Probability: Independent Events 19
Conditional CDF and PDF 20
Conditional Expected Values 21
Example: Conditional Expected Value 22
Probability: Two Random Variables 23
Joint and Marginal Cumulative Distributions 24
Joint and Marginal Density Functions 25
Conditional Distributions and Density Functions 26
Example: Conditional PDF 27
Principle of Maximum Likelihood 28
Independent RVs 29
Expected Value: Moments 30
Example: Expected Value 31
Bivariate Gaussian Distribution 32
Example: Rician Distribution 33
Functions of Two RVs 34
Sum of Two RVs 35
Product and Quotient of Two RVs 36
Conditional Expectations and Mean-Square Estimation 37
Sums of N Complex Random Variables 38
Central Limit Theorem 39
Example: Central Limit Theorem 40
Phases Uniformly Distributed on (−π, π) 41
Phases Not Uniformly Distributed on (−π, π) 42
Example: Phases Uniformly Distributed on (− α, α) 43
Central Limit Theorem Does Not Apply 45
Example: Non-Gaussian Limit 46
Random Processes 48
Random Processes Terminology 49
First- and Second-Order Statistics 50
Stationary Random Processes 51
Autocorrelation and Autocovariance Functions 52
Wide-Sense Stationary Process 53
Example: Correlation and PDF 54
Time Averages and Ergodicity 55
Structure Functions 56
Cross-Correlation and Cross-Covariance Functions 57
Power Spectral Density 58
Example: PSD 59
PSD Estimation 60
Bivariate Gaussian Processes 61
Multivariate Gaussian Processes 62
Examples of Covariance Function and PSD 63
Interpretations of Statistical Averages 64
Random Fields 65
Random Fields Terminology 66
Mean and Spatial Covariance Functions 67
1D and 3D Spatial Power Spectrums 68
2D Spatial Power Spectrum 69
Structure Functions 70
Example: PSD 71
Transformations of Random Processes 72
Memoryless Nonlinear Transformations 73
Linear Systems 74
Expected Values of a Linear System 75
Example: White Noise 76
Detection Devices 77
Zero-Crossing Problem 78
Random Data Analysis 79
Tests for Stationarity, Periodicity, and Normality 80
Nonstationary Data Analysis for Mean 81
Analysis for Single Time Record 82
Runs Test for Stationarity 83
Equation Summary 85
Biography 90
Index 91
Glossary of Symbols and Notation
a, x, u, etc.
Random variable, process, or field
Bu(R)
Autocovariance or covariance function of random field
Cx(τ)
Autocovariance or covariance function of random process
C xy(τ)
Cross-covariance function
CDF
Cumulative distribution function
Cov
Covariance
Dx(τ)
Structure function
E[.]
Expectation operator
E[g(x)|A]
Conditional expectation operator
fx(x), fx(x, t)
Probability density function
fx(x|A)
Conditional probability density
Fx(x), Fx(x, t)
Cumulative distribution function
Fx(x|A)
Conditional cumulative distribution function
Generalized hypergeometric function
h(t)
Impulse response function
H(ω)
Transfer function
Ip(x)
Modified Bessel function of the first kind
Jp(x)
Bessel function of the first kind
Kp(x)
Modified Bessel function of the second kind
m, m(t)
Mean value
mk
k’th standard statistical moment
n!
Factorial function
Probability density function
Pr
Probability
Pr(B|A)
Conditional probability
PSD
Power spectral density
RV
Random variable
Rx(τ)
Autocorrelation or correlation function
Rxy(τ)
Cross-correlation function
Long-time-average correlation function
Sx(ω), Su(κ)
Power spectral density function
U(x−a)
Unit step function
Var
Variance
Var[x|A]
Conditional variance
Time average
Complex conjugate of z
γ(c, x)
Incomplete gamma function
Γ(x)
Gamma function
δ(x−a)
Dirac delta function (impulse function)
μk
k’th central statistical moment
Estimator of mean value
Variance
τ
Time difference t2−t1
Φx(s)
Characteristic function
||
Absolute value
∈
Belonging to
Binomial coefficient
〈 〉
Ensemble average
{ }
Event
Intersection
