Data processing inequality

The Data processing inequality is an information theoretic concept which states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.^[1] As explained by Kinney and Atwal, the DPI means that information is generally lost (never gained) when transmitted through a noisy channel.^[2]

Example

Let $X\rightarrow Y\rightarrow Z$ be a Markov chain; then,
$I(x;y)\geqslant I(x;z)$ with
$I(x;y)=I(x;z)$ if and only if $X\rightarrow Z\rightarrow Y$
where $I(x;y)$ is the mutual information.

References

↑ Beaudry, Normand (2012), "An intuitive proof of the data processing inequality", Quantum Information & Computation, 12 (5–6): 432–441, arXiv:1107.0740, Bibcode:2011arXiv1107.0740B
↑ Kinney; Atwal (Mar 2014). "Equitability, mutual information, and the maximal information coefficient". Proc Natl Acad Sci U S A. 111: 3354–9. arXiv:1301.7745. Bibcode:2014PNAS..111.3354K. doi:10.1073/pnas.1309933111. PMC 3948249. PMID 24550517.

External links

http://www.scholarpedia.org/article/Mutual_information

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[BeaudryArxiv-1] Beaudry, Normand (2012), "An intuitive proof of the data processing inequality", Quantum Information & Computation, 12 (5–6): 432–441, arXiv:1107.0740, Bibcode:2011arXiv1107.0740B

[2] Kinney; Atwal (Mar 2014). "Equitability, mutual information, and the maximal information coefficient". Proc Natl Acad Sci U S A. 111: 3354–9. arXiv:1301.7745. Bibcode:2014PNAS..111.3354K. doi:10.1073/pnas.1309933111. PMC 3948249. PMID 24550517.

Data processing inequality

Example

See also

References

External links