在物理學中，“熵”一詞具有重要的物理意義，表示系統的“無序”程度。在數學中，使用更抽象的定義。變數 
的（夏農）熵定義為
![H(X)=-sum_(x)P(x)log_2[P(x)]](/images/equations/Entropy/NumberedEquation1.svg) |
位元，其中 
是 X 處於狀態 
的機率，並且如果 
，則 
定義為 0。變數 
, ..., 
的聯合熵則定義為
![H(X_1,...,X_n)=-sum_(x_1)...sum_(x_n)P(x_1,...,x_n)log_2[P(x_1,...,x_n)].](/images/equations/Entropy/NumberedEquation2.svg) |
熵
另請參閱
微分熵, 資訊理論, 柯爾莫哥洛夫熵, 最大熵方法, 度量熵, 互資訊, 納特, 奧恩斯坦定理, 冗餘, 相對熵, 拓撲熵使用 探索
參考文獻
Ellis, R. S. Entropy, Large Deviations, and Statistical Mechanics. New York: Springer-Verlag, 1985.Havil, J. "A Measure of Uncertainty." §14.1 in Gamma: Exploring Euler's Constant. Princeton, NJ: Princeton University Press, pp. 139-145, 2003.Khinchin, A. I. Mathematical Foundations of Information Theory. New York: Dover, 1957.Lasota, A. and Mackey, M. C. Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, 2nd ed. New York: Springer-Verlag, 1994.Ott, E. "Entropies." §4.5 in Chaos in Dynamical Systems. New York: Cambridge University Press, pp. 138-144, 1993.Rothstein, J. "Information, Measurement, and Quantum Mechanics." Science 114, 171-175, 1951.Schnakenberg, J. "Network Theory of Microscopic and Macroscopic Behavior of Master Equation Systems." Rev. Mod. Phys. 48, 571-585, 1976.Shannon, C. E. "A Mathematical Theory of Communication." The Bell System Technical J. 27, 379-423 and 623-656, July and Oct. 1948. http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf.Shannon, C. E. and Weaver, W. Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1963.在 中被引用
熵請引用為
Eric W. Weisstein "熵。" 來自 —— 資源。 https://mathworld.tw/Entropy.html