Marginal distribution

Given a known joint distribution of two discrete random variables, say, X and Y, the marginal distribution of either variable--X for example--is the probability distribution of X when the values of Y are not taken into consideration. This can be calculated by summing the joint probability distribution over all values of Y. Naturally, the converse is also true: we can obtain the marginal distribution for Y by summing over the separate values of X.

${\displaystyle p_{X}(x_{i})=\sum _{j}p(x_{i},y_{j})}$ , and ${\displaystyle p_{Y}(y_{j})=\sum _{i}p(x_{i},y_{j})}$

A marginal probability can always be written as an expected value:

${\displaystyle p_{X}(x)=\int _{y}p_{X\mid Y}(x\mid y)\,p_{Y}(y)\,\mathrm {d} y=\operatorname {E} _{Y}[p_{X\mid Y}(x\mid y)]\;.}$

Intuitively, the marginal probability of X is computed by examining the conditional probability of X given a particular value of Y, and then averaging this conditional probability over the distribution of all values of Y.

This follows from the definition of expected value (after applying the law of the unconscious statistician)

${\displaystyle \operatorname {E} _{Y}[f(Y)]=\int _{y}f(y)p_{Y}(y)\,\mathrm {d} y.}$

Therefore, marginalization provides the rule for the transformation of the probability distribution of a random variable Y and another random variable X = g(Y):

${\displaystyle p_{X}(x)=\int _{y}p_{X\mid Y}(x\mid y)\,p_{Y}(y)\,\mathrm {d} y=\int _{y}\delta {\big (}x-g(y){\big )}\,p_{Y}(y)\,\mathrm {d} y.}$

Given two continuous random variables X and Y whose joint distribution is known, then marginal probability density function can be obtained by integrating the joint probability distribution, ${\displaystyle f}$, over Y, and vice versa. That is

${\displaystyle f_{X}(x)=\int _{c}^{d}f(x,y)dy,}$ and ${\displaystyle f_{Y}(y)=\int _{a}^{b}f(x,y)dx}$

where ${\displaystyle x\in [a,b]}$, and ${\displaystyle y\in [c,d]}$.

Finding the marginal cumulative distribution function from the joint cumulative distribution function is easy. Recall that

${\displaystyle F(x,y)=P(X\leq x,Y\leq y)}$ for discrete random variables,

${\displaystyle F(x,y)=\int _{a}^{x}\int _{c}^{y}f(x',y')\,dy'dx'}$ for continuous random variables,

If X and Y jointly take values on [a, b] × [c, d] then

${\displaystyle F_{X}(x)=F(x,d)}$ ${\displaystyle F_{Y}(y)=F(b,y)}$

If d is ∞, then this becomes a limit ${\displaystyle F_{X}(x)=\lim _{y\to \infty }F(x,y)}$ . Likewise for ${\displaystyle F_{Y}(y)}$.

Marginal distribution functions play an important role in the characterization of independence between random variables: two random variables are independent if and only if their joint distribution function is equal to the product of their marginal distribution functions,[2]

${\displaystyle P(X\leq x,Y\leq y)=P(X\leq x)P(Y\leq y)}$ for discrete random variables,

${\displaystyle f(x,y)=f_{X}(x)f_{Y}(y)}$ for continuous random variables,

that is,

${\displaystyle F(x,y)=F_{X}(x)F_{Y}(y)}$

for all possible values x and y.[3]

• Discrete random variables

Let X and Y be two discrete random variables having joint distributions (See Table.2),

we can easily conclude from this table that

${\displaystyle P_{X}(X\leq x_{2})P_{Y}(Y\leq y_{1})={\Big (}{\frac {2}{3}}+{\frac {1}{6}}{\Big )}\times {\frac {1}{2}}={\frac {5}{12}}=P(X\leq x_{2},Y\leq y_{1})}$, which is the same as

${\displaystyle F(x_{2},y_{1})={\frac {1}{12}}+{\frac {4}{12}}={\frac {5}{12}}={\Big (}{\frac {1}{6}}+{\frac {2}{3}}{\Big )}\times {\frac {1}{2}}=F_{X}(x_{2})F_{Y}(y_{1})}$.

Thus, discrete random variables X and Y are independent.

• continuous random variables[2]

Let X and Y be two random variables having marginal distribution functions

${\displaystyle F_{X}(x)={\begin{cases}0&{\text{if }}x<0\\1-exp(-x)&{\text{if }}x\geq 0\end{cases}}}$

${\displaystyle F_{Y}(y)={\begin{cases}0&{\text{if }}y<0\\1-exp(-y)&{\text{if }}y\geq 0\end{cases}}}$

and joint distribution function

${\displaystyle F_{X,Y}(x,y)={\begin{cases}0&{\text{if }}x<0{\text{ or }}y<0\\1-exp(-x)-exp(-y)+exp(-x-y)&{\text{if }}x\geq 0{\text{ and }}y\geq 0\end{cases}}}$

It is easy to check that

${\displaystyle F_{X,Y}(x,y)=F_{X}(x)F_{Y}(y)}$

The marginal probability is the probability that one of the variables takes a particular value: ${\displaystyle p_{X}(x)=P(X=x)}$, disregarding what the values of the other variables are. In essence, we are calculating the probability of one independent variable. A conditional probability is the probability that an event will occur given that another specific event has already occurred. We say that we are placing a condition on the larger distribution of data, or that the calculation for one variable is dependent on another variable.[4]

The relationship between marginal distribution is usually described by saying that the conditional distribution is the joint distribution divided by the marginal distribution.[5] That is,

${\displaystyle p_{Y|X}(y|x)=P(Y=y|X=x)={\frac {P(X=x,Y=y)}{P_{X}(x)}}}$ for discrete random variables,

${\displaystyle f_{Y|X}(y|x)={\frac {f_{X,Y}(x,y)}{f_{X}(x)}}}$ for continuous random variables.

Suppose we are trying to understand the relationship in a classroom of 200 students between the amount of time studied (X) and the percent correct (Y).[6] We can assume X and Y are discrete random variables representing the amount of time studied and the percent correct, respectively. Then the joint distribution of X and Y can simply be described by listing all the possible values of p(x_i,y_j), as shown in Table.3.

If we want to study how many students who got a score below 20 in the test, we need to calculate the marginal distribution. To translate it into a statistical problem, we can derive the equation: ${\displaystyle p_{Y}(y_{1})=P_{Y}(Y=y_{1})=\sum _{i=1}^{4}P(x_{i},y_{1})={\frac {2}{200}}+{\frac {8}{200}}={\frac {10}{200}}}$,which means, 5% of the students get a score lower than 20 in the test, that is, 10 students.

In another case, if we want to study the probability that the students studied more than 60 minutes but got a score lower than 20, we need to calculate the conditional distribution. Here, the given condition is that those students studied more than 60 minutes, namely, ${\displaystyle p_{X}(x_{4})=P_{X}(X=x_{4})={\frac {70}{200}}}$. According to the equation given above, we can calculate that ${\displaystyle p_{Y|X}(y_{1}|x_{4})=P(Y=y_{1}|X=x_{4})={\frac {P(X=x_{4},Y=y_{1})}{P(X=x_{4})}}={\frac {8}{70}}={\frac {4}{35}}}$.