INFORMATION & COMMUNICATION

discrete information entropy

measures the average uncertainty of each output of a random source.

H(X)=−∑xp(x)log⁡2p(x)H(X)=-\sum_x p(x)\log_2p(x)

symbols, variables and units

p(x): probability; H: bit; agreed to 0 log 0=0.

applicable conditions and boundaries

Discrete distribution; continuous differential entropy differs from this definition and can be negative.

formula source code

The following is a copyable LaTeX expression.

H(X)=-\sum_x p(x)\log_2p(x)

Reference and Extended Learning

MIT OpenCourseWare · Information Theory

is organized according to model definition and assumptions. Please check actual conditions and original literature before engineering, research and clinical use.

entropyinformation entropy

How can this knowledge be incorporated into high-end products?

Relevant scientific figures and methodological contributions

Same subject formula

Go to Free Science Tool Library ↗