Markov property

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be Markovian or a Markov process. The most famous Markov process is a Markov chain. Brownian motion is another well-known Markov process.

Let ${\displaystyle (\Omega ,{\mathcal {F}},P)}$ be a probability space with a filtration ${\displaystyle ({\mathcal {F}}_{s},\ s\in I)}$, for some (totally ordered) index set ${\displaystyle I}$; and let ${\displaystyle (S,{\mathcal {S}})}$ be a measurable space. A ${\displaystyle (S,{\mathcal {S}})}$-valued stochastic process ${\displaystyle X=\{X_{t}:\Omega \to S\}_{t\in I}}$ adapted to the filtration is said to possess the Markov property if, for each ${\displaystyle A\in {\mathcal {S}}}$ and each ${\displaystyle s,t\in I}$ with ${\displaystyle s<t}$,

${\displaystyle P(X_{t}\in A\mid {\mathcal {F}}_{s})=P(X_{t}\in A\mid X_{s}).}$[3]

In the case where ${\displaystyle S}$ is a discrete set with the discrete sigma algebra and ${\displaystyle I=\mathbb {N} }$, this can be reformulated as follows:

${\displaystyle P(X_{n}=x_{n}\mid X_{n-1}=x_{n-1},\dots ,X_{0}=x_{0})=P(X_{n}=x_{n}\mid X_{n-1}=x_{n-1}).}$

Alternatively, the Markov property can be formulated as follows.

${\displaystyle \operatorname {E} [f(X_{t})\mid {\mathcal {F}}_{s}]=\operatorname {E} [f(X_{t})\mid \sigma (X_{s})]}$

for all ${\displaystyle t\geq s\geq 0}$ and ${\displaystyle f:S\rightarrow \mathbb {R} }$ bounded and measurable.[4]

Suppose that ${\displaystyle X=(X_{t}:t\geq 0)}$ is a stochastic process on a probability space ${\displaystyle (\Omega ,{\mathcal {F}},P)}$ with natural filtration ${\displaystyle \{{\mathcal {F}}_{t}\}_{t\geq 0}}$. For any ${\displaystyle t\geq 0}$, we can define the germ sigma algebra ${\displaystyle {\mathcal {F}}_{t+}}$ to be the intersection of all ${\displaystyle {\mathcal {F}}_{s}}$ for ${\displaystyle s>t}$. Then for any stopping time ${\displaystyle \tau }$ on ${\displaystyle \Omega }$, we can define

${\displaystyle {\mathcal {F}}_{\tau ^{+}}=\{A\in {\mathcal {F}}:\{\tau =t\}\cap A\in {\mathcal {F}}_{t+},\,\forall t\geq 0\}}$.

Then ${\displaystyle X}$ is said to have the strong Markov property if, for each stopping time ${\displaystyle \tau }$, conditioned on the event ${\displaystyle \{\tau <\infty \}}$, we have that for each ${\displaystyle t\geq 0}$, ${\displaystyle X_{\tau +t}}$ is independent of ${\displaystyle {\mathcal {F}}_{\tau ^{+}}}$ given ${\displaystyle X_{\tau }}$.

The strong Markov property implies the ordinary Markov property, since by taking the stopping time ${\displaystyle \tau =t}$, the ordinary Markov property can be deduced.[5]

In the fields of predictive modelling and probabilistic forecasting, the Markov property is considered desirable since it may enable the reasoning and resolution of the problem that otherwise would not be possible to be resolved because of its intractability. Such a model is known as a Markov model.

Assume that an urn contains two red balls and one green ball. One ball was drawn yesterday, one ball was drawn today, and the final ball will be drawn tomorrow. All of the draws are "without replacement".

Suppose you know that today's ball was red, but you have no information about yesterday's ball. The chance that tomorrow's ball will be red is 1/2. That's because the only two remaining outcomes for this random experiment are:

On the other hand, if you know that both today and yesterday's balls were red, then you are guaranteed to get a green ball tomorrow.

This discrepancy shows that the probability distribution for tomorrow's color depends not only on the present value, but is also affected by information about the past. This stochastic process of observed colors doesn't have the Markov property. Using the same experiment above, if sampling "without replacement" is changed to sampling "with replacement," the process of observed colors will have the Markov property.[6]

An application of the Markov property in a generalized form is in Markov chain Monte Carlo computations in the context of Bayesian statistics.

• Causal Markov condition
• Chapman–Kolmogorov equation
• Hysteresis
• Markov blanket
• Markov chain
• Markov decision process
• Markov model