Shannon Entropy Calculator (text)

Understanding the Shannon entropy calculator — concept and principle

Shannon entropy measures how unpredictable the symbols of a message are, in bits per symbol. This calculator counts how often each character appears in your text and computes the entropy of that frequency distribution. It is used in information theory courses, to estimate how compressible data is and to compare how varied strings are.

Frequent letters carry fewer bits; the weighted average is the entropy.

How it is calculated

H = −Σ p(x) × log₂ p(x), summed over every distinct character x

where p(x) = (count of x) ÷ (total characters). The calculator reports three values:

Entropy (bits/symbol): H itself.
Minimum bits for message: H × number of characters, the lower bound for encoding this exact text with a code built on these frequencies.
Unique symbols: how many different characters appear (spaces and punctuation count).

The principle

Claude Shannon defined the information of an event with probability p as log₂(1/p) bits: a certain event (p = 1) tells you nothing, while a rare one tells you a lot. Entropy is the average information per symbol. It is largest when all symbols are equally likely (log₂ of the number of symbols) and zero when only one symbol occurs. Shannon's source coding theorem says no lossless code can use fewer than H bits per symbol on average, which is why entropy sets the limit for compression. This calculator uses a zero-order model: it looks only at single-character frequencies and ignores order and context.

Worked example

Text: "hello world" has 11 characters.
Counts: l = 3, o = 2, and h, e, space, w, r, d = 1 each, so 8 unique symbols.
Terms: −(3/11) log₂(3/11) = 0.5112; −(2/11) log₂(2/11) = 0.4472; each single character gives −(1/11) log₂(1/11) = 0.3145, and six of them give 1.8870.
Sum: 0.5112 + 0.4472 + 1.8870 = 2.845350937 bits/symbol.
Minimum bits: 2.845350937 × 11 = 31.2988603 bits, compared with 88 bits in 8-bit ASCII.

How to read the result

The maximum possible value is log₂(unique symbols); for 8 symbols that is 3 bits, so 2.85 is close to uniform. English prose usually gives about 4 to 4.5 bits per character under this single-letter model, while random bytes approach 8. A low value means a few characters dominate and the text compresses well.

Common mistakes

Treating zero-order entropy as the true information content; real language has strong context, so its entropy rate is much lower.
Reading high entropy of a short string as proof of randomness; small samples give unreliable frequencies.
Forgetting that case, spaces and punctuation are separate symbols.

Keep in mind: This estimate is computed from your sample's own frequencies. It does not measure the security of a password or key, which depends on how it was generated, not on how its characters happen to be distributed.

Frequently asked questions

What is the maximum Shannon entropy for text?

log₂ of the alphabet size: about 4.7 bits for 26 letters and 8 bits for arbitrary bytes.

Can a file be compressed below its entropy?

Not below the true entropy of its source, but real compressors beat this zero-order figure by exploiting repeated patterns.

Sources: C. E. Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal, 1948 · T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley

Checked 8 Nov 2026. Standard formulas and RFC/IEEE definitions, computed locally in your browser — nothing you type is sent anywhere.

Your numbers