Maximum entropy

Entropy is a concept that originated in thermodynamics, and later, via statistical mechanics, motivated entire branches of information theory, statistics, and machine learning.

Maximum entropy is the state of a physical system at greatest disorder or a statistical model of least encoded information, these being important theoretical analogs.

Maximum entropy may refer to:

Physics

• Maximum entropy thermodynamics
• The law of maximum entropy production
• Maximum entropy spectral estimation

Statistics

• The principle of maximum entropy
• The maximum entropy probability distribution
• Maximum entropy estimation, akin to maximum likelihood estimation

Machine learning

• Entropy maximization, an approach in convex optimization
• Maximum entropy classifier

See also

• Second law of thermodynamics (also known as "the law of increasing entropy")