Taylor expansions for the moments of functions of random variables

In probability theory, it is possible to approximate the moments of a function f of a random variable X using Taylor expansions, provided that f is sufficiently differentiable and that the moments of X are finite.

First moment

{\begin{aligned}\operatorname {E} \left[f(X)\right]&{}=\operatorname {E} \left[f\left(\mu _{X}+\left(X-\mu _{X}\right)\right)\right]\\&{}\approx \operatorname {E} \left[f(\mu _{X})+f'(\mu _{X})\left(X-\mu _{X}\right)+{\frac {1}{2}}f''(\mu _{X})\left(X-\mu _{X}\right)^{2}\right].\end{aligned}}

Since $E[X-\mu _{X}]=0,$ the second term disappears. Also $E[(X-\mu _{X})^{2}]$ is $\sigma_X^2$ . Therefore,

\operatorname {E}\left[f(X)\right]\approx f(\mu _{X})+{\frac {f''(\mu _{X})}{2}}\sigma _{X}^{2}

where $\mu _{X}$ and $\sigma _{X}^{2}$ are the mean and variance of X respectively.^[1]

It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,

\operatorname {E}\left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {E}\left[X\right]}{\operatorname {E}\left[Y\right]}}-{\frac {\operatorname {cov}\left[X,Y\right]}{\operatorname {E}\left[Y\right]^{2}}}+{\frac {\operatorname {E}\left[X\right]}{\operatorname {E}\left[Y\right]^{3}}}\operatorname {var}\left[Y\right]

Second moment

Similarly,^[1]

\operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}

The above is using a first order approximation unlike for the method used in estimating the first moment. It will be a poor approximation in cases where $f(X)$ is highly non-linear. This is a special case of the delta method. For example,

\operatorname {var}\left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {var}\left[X\right]}{\operatorname {E}\left[Y\right]^{2}}}-{\frac {2\operatorname {E}\left[X\right]}{\operatorname {E}\left[Y\right]^{3}}}\operatorname {cov}\left[X,Y\right]+{\frac {\operatorname {E}\left[X\right]^{2}}{\operatorname {E}\left[Y\right]^{4}}}\operatorname {var}\left[Y\right].

The second order approximation is^[2]:

\operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]-{\frac {\left(f''(\operatorname {E} \left[X\right])\right)^{2}}{4}}\left(\operatorname {var} \left[X\right]\right)^{2}=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}-{\frac {1}{4}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}

Notes

1 2 Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005.
↑ Hendeby, Gustaf; Gustafsson, Fredrik. "ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS" (PDF). Retrieved 5 October 2017.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[Benaroya-1] 1 2 Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005.

[HendebyGustafsson-2] Hendeby, Gustaf; Gustafsson, Fredrik. "ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS" (PDF). Retrieved 5 October 2017.

Taylor expansions for the moments of functions of random variables

First moment

Second moment

See also

Notes