Markov kernel

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.^[1]

Formal definition

Let $(X,{\mathcal {A}}),(Y,{\mathcal {B}})$ be measurable spaces. A Markov kernel with source $(X,{\mathcal {A}})$ and target $(Y,{\mathcal {B}})$ is a map $\kappa :X\times {\mathcal {B}}\to [0,1]$ with the following properties:

The map $x\mapsto \kappa (x,B)$ is ${\mathcal {A}}$ -measurable for every $B\in {\mathcal {B}}.$
The map $B\mapsto \kappa (x,B)$ is a probability measure on $(Y,{\mathcal {B}})$ for every $x\in X$ .

In other words it associates to each point $x\in X$ a probability measure $\kappa (x,\cdot )$ on $(Y,{\mathcal {B}})$ such that, for every measurable set $B\in {\mathcal {B}}$ , the map $x\mapsto \kappa (x,B)$ is measurable with respect to the $\sigma$ -algebra ${\mathcal A}.$ ^[2]

Examples

Simple random walk

Take $X=Y=\mathbb {Z} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$ (the power set of $\mathbb {Z}$ ), then the Markov kernel $\kappa$ with

\kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} ),

where $\mathbf {1}$ is the indicator function, describes the transition rule for the random walk on $\mathbb {Z} .$

Galton–Watson process

Take $X=Y=\mathbb {N} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} ),$ then

\kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&x=0\\\Pr(\xi _{1}+\cdots +\xi _{x}\in B)&x\neq 0\\\end{cases}}

with i.i.d. random variables $\xi _{i}$ .

General Markov processes with finite state space

Take $X=Y,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$ and $|X|=|Y|=n,$ then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1\leq i,j\leq n}$ with

\forall i\in X:\qquad \sum _{j\in Y}K_{ij}=1.

In the convention of Markov kernels we write

\kappa (i,B)=\sum _{j\in B}K_{ij},\qquad \forall i\in X,\quad \forall B\in {\mathcal {B}}

.

Construction of a Markov kernel

If $\nu$ is a finite measure on $(Y,{\mathcal {B}})$ and $k:X\times Y\to \mathbb {R} _{+}$ is a measurable function with respect to the product $\sigma$ -algebra ${\mathcal {A}}\otimes {\mathcal {B}}$ and has the property

\forall x\in X\qquad \int _{Y}k(x,y)\nu (\mathrm {d} y)=1,

then the mapping

{\begin{cases}\kappa :X\times {\mathcal {B}}\to [0,1]\\\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y)\end{cases}}

defines a Markov kernel.^[3]

Properties

Semidirect product

Let $(X,{\mathcal {A}},P)$ be a probability space and $\kappa$ a Markov kernel from $(X,{\mathcal {A}})$ to some $(Y,{\mathcal {B}})$ . Then there exists a unique measure $Q$ on $(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$ , such that:

Q(A\times B)=\int _{A}\kappa (x,B)\,dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}.

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ -valued random variable on the measure space $(\Omega ,{\mathcal {F}},P)$ and ${\mathcal {G}}\subseteq {\mathcal {F}}$ a sub- $\sigma$ -algebra. Then there exists a Markov kernel $\kappa$ from $(\Omega ,{\mathcal {G}})$ to $(S,Y)$ , such that $\kappa (\cdot ,B)$ is a version of the conditional expectation $\mathbb {E} [\mathbf {1} _{\{X\in B\}}\mid {\mathcal {G}}]$ for every $B\in Y$ , i.e.

P(X\in B\mid {\mathcal {G}})=\mathbb {E} \left[\mathbf {1} _{\{X\in B\}}\mid {\mathcal {G}}\right]=\kappa (\omega ,B),\qquad P{\text{-a.s.}}\,\,\forall B\in {\mathcal {G}}.

It is called regular conditional distribution of $X$ given ${\mathcal {G}}$ and is not uniquely defined.

Generalizations

Transition kernels generalize Markov kernels in the sense that the map

B\mapsto \kappa (x,B)

is not necessarily a probability measure but can be any type of measure.

References

↑ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8.
↑ Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.
↑ Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.

Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9

§36. Kernels and semigroups of kernels

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8.

[2] Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.

[3] Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.