A theorem from information theory that is a simple consequence of the weak law of large numbers. It states that if a set of values X_1, X_2, ..., X_n is drawn independently from a random variable X distributed according to P(x), then the joint probability P(X_1, ..., X_n) satisfies -1/n log_2 P(X_1, X_2, ..., X_n)->H(X), where H(X) is the entropy of the random variable X.

entropy