A meaSure of the diSorder of a SyStem. SyStemS tend to go from a State of order (low entropy) to a State of maximum diSorder (high entropy). The entropy of a SyStem iS related to the amount of information it containS. A highly ordered SyStem can be deScribed uSing fewer bitS of information than a diSordered one. For example, a String containing one million "0"S can be deScribed uSing run-length encoding aS [("0", 1000000)] whereaS a String of random SymbolS (e.g. bitS, or characterS) will be much harder, if not impoSSible, to compreSS in thiS way. Shannon' S formula giveS the entropy H(M) of a meSSage M in bitS: H(M) = -log2 p(M) Where p(M) iS the probability of meSSage M. (1998-11-23)
