Ergodic process

In econometrics and signal processing, a stochastic process is said to be ergodic if its statistical properties can be deduced from a single, sufficiently long, random sample of the process. The reasoning is that any collection of random samples from a process must represent the average statistical properties of the entire process. In other words, regardless of what the individual samples are, a birds-eye view of the collection of samples must represent the whole process. Conversely, a process that is not ergodic is a process that changes erratically at an inconsistent rate.^[1]

Specific definitions

One can discuss the ergodicity of various statistics of a stochastic process. For example, a wide-sense stationary process $X(t)$ has constant mean

\mu _{X}=E[X(t)]

,

and autocovariance

r_{X}(\tau )=E[(X(t)-\mu _{X})(X(t+\tau )-\mu _{X})]

,

that depends only on the lag $\tau$ and not on time $t$ . The properties $\mu _{X}$ and $r_{X}(\tau )$ are ensemble averages not time averages.

The process $X(t)$ is said to be mean-ergodic^[2] or mean-square ergodic in the first moment^[3] if the time average estimate

{\hat {\mu }}_{X}={\frac {1}{T}}\int _{{0}}^{{T}}X(t)\,dt

converges in squared mean to the ensemble average $\mu _{X}$ as $T\rightarrow \infty$ .

Likewise, the process is said to be autocovariance-ergodic or mean-square ergodic in the second moment^[3] if the time average estimate

{\hat {r}}_{X}(\tau )={\frac {1}{T}}\int _{0}^{T}[X(t+\tau )-\mu _{X}][X(t)-\mu _{X}]\,dt

converges in squared mean to the ensemble average $r_{X}(\tau )$ , as $T\rightarrow \infty$ . A process which is ergodic in the mean and autocovariance is sometimes called ergodic in the wide sense.^[3]

An important example of an ergodic processes is the stationary Gaussian process with continuous spectrum.

Discrete-time random processes

The notion of ergodicity also applies to discrete-time random processes $X[n]$ for integer $n$ .

A discrete-time random process $X[n]$ is ergodic in mean if

{\hat {\mu }}_{X}={\frac {1}{N}}\sum _{{n=1}}^{{N}}X[n]

converges in squared mean to the ensemble average $E[X]$ , as $N\rightarrow \infty$ .

Examples of non-ergodic random processes

An unbiased random walk is non-ergodic. Its expectation value is zero at all times, whereas its time average is a random variable with divergent variance.

Suppose that we have two coins: one coin is fair and the other has two heads. We choose (at random) one of the coins first, and then perform a sequence of independent tosses of our selected coin. Let X[n] denote the outcome of the nth toss, with 1 for heads and 0 for tails. Then the ensemble average is ½ (½ + 1) = ¾; yet the long-term average is ½ for the fair coin and 1 for the two-headed coin. So the long term time-average is either 1/2 or 1. Hence, this random process is not ergodic in mean.

Notes

↑ Originally due to L. Boltzmann. See part 2 of Vorlesungen über Gastheorie. Leipzig: J. A. Barth. 1898. OCLC 01712811. ('Ergoden' on p.89 in the 1923 reprint.) It was used to prove equipartition of energy in the kinetic theory of gases
↑ Papoulis, p.428
1 2 3 Porat, p.14

References

Porat, B. (1994). Digital Processing of Random Signals: Theory & Methods. Prentice Hall. p. 14. ISBN 0-13-063751-3.
Papoulis, Athanasios (1991). Probability, random variables, and stochastic processes. New York: McGraw-Hill. pp. 427–442. ISBN 0-07-048477-5.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Originally due to L. Boltzmann. See part 2 of Vorlesungen über Gastheorie. Leipzig: J. A. Barth. 1898. OCLC 01712811. ('Ergoden' on p.89 in the 1923 reprint.) It was used to prove equipartition of energy in the kinetic theory of gases

[2] Papoulis, p.428

[porat-3] 1 2 3 Porat, p.14