arXiv:0809.3540v1 [cond-mat.stat-mech] 22 Sep 2008
A Note on the Equivalence of Gibbs Free Energy and Information
David Ford, Physics NPS∗
Department of Physics, Naval Post Graduate School, Monterey, California
(Dated: September 22, 2008)
The minimization of Gibbs free energy is based on the changes in work and free energy that occur
in a physical or chemical system. The maximization of mutual information, the capacity, of a noisy
channel is determined based on the marginal probabilities and conditional entropies associated with a
communications system. As different as the procedures might first appear, through the exploration
of a simple, “dual use” Ising model, it is seen that the two concepts are in fact the same. In
particular, the case of a binary symmetric channel is calculated in detail.
In 1876 J.W. Gibbs  proposed a refinement
to the notion of Helmholtz free energy. The util-
ity of the refinement rests on the observation
that when attempting to determine the max-
imum amount of work that may be extracted
from a thermodynamic process, some of that
work may already be accounted for, for exam-
ple, work against the atmosphere −p∆V . The
suggestion was that perhaps the actual quan-
tity of interest is the maximum amount of work
other than atmospheric that may be extracted.
This general concept, equipped with a system
dependent form for the work term ,,, is
well known and widely used in physics, physical
chemistry and many engineering disciplines.
Some seventy years later C. Shannon and W.
Weaver  provide a sound theoretical frame-
work for determining the maximum amount
of information that may be transmitted from
sender to receiver through a noisy channel.
Their concept, the channel capacity, calculated
by extremization of differences of various com-
munication system entropies, is reminiscent of
statistical physics  but apparently different.
The purpose of this letter is to show how these
two concepts are in fact the same. To accom-
plish this it is useful to have a simple example
that belongs to both the physics and communi-
cations theory traditions. The Ising model for
magnetic systems is a good candidate. The liter-
ature on the topic is vast. Physical applications
of the model relevant to the present purpose in-
clude , ,. Relevant applications of the
model in communications include ,,.
After a brief review of the Ising model, it will be
shown that the real space renormalization pro-
cedure  connects the Gibbs free energy and
the channel capacity in a natural way.
As noted in the introduction, the Ising model
is of interest in both physics and communica-
tions. The states described by the model are a
sequence of electron spin up (↑) or spin down
(↓) states which are easily translated to the 1’s
and 0’s of a bit stream. The simplest communi-
cations scheme is one in which the transmitted
bits are statistically independent of each other
but coupled to the bits received. The sequence
FIG. 1: A simple send/receive Ising model. The sent
bits, σ1, σ3, σ3 (odd numbered) are independent of
each other but coupled to their received bits σ2, σ4,
σ6 (even numbered).
 T.L. Hill, “ Statistical Mechanics”, McGraw-
Hill Book Co., New York, 1956
 K. Huang, “Statistical Mechanics”, Wiley, 1987
 D. Ford, “Surfaces of Constant Temperature
in Time,” http://www.arxiv.org/abs/cond-
 D. Ford, “Application of Thermodynamics to
the Reduction of Data Generated by a Non-
Standard System,” http://arxiv.org/abs/cond-