Image Coding
H(p) ? 0 with H(p) = 0 only for p = 0 or p = 1.
H(p1,p2, . . . , pM) ? logbM with equality iff pi = 1/M for i = 1, 2, . . . , M.
Considering two pixels (or images) X with gray levels {xi} and Y with gray levels {yi}, the average joint information or joint entropy is defined as
Then H(X,Y) ? H(X) + H(Y), with equality if X and Y are statistically independent.