ArticlePDF Available

Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t

Authors:
  • Associated General Contractors of America
AUTHOR COPY
informal payments that do not appear in official
statistics.
Perhaps the biggest harm done by politically
administered prices occurs because they prevent
entrepreneurial physicians and hospitals from
bundling and repricing their services in ways that
benefit patients. Even though a Medicare patient
may have diabetes, congestive heart failure, and
high blood pressure, Medicare will pay the full fees
for treating only one condition per visit. Patients
and physicians who find it more efficient to take
care of everything in one visit may not do so unless
the physician can afford to give his time away.
When third-party payers do not control pricing
or practice, as is the case in markets for cosmetic
surgery, vision correction surgery, and cash-only
surgical procedures, physicians interact directly with
price-sensitive customers. Over time, they have ad-
justed their services in order to attract patients.
Prices have plummeted. The real price of LASIK
surgery declined by 30 percent in the last decade.
Patients heal faster, outcomes have improved, and
advances in technology have propelled an expanded
range of treatment options. Further evidence of
the importance of freeing suppliers comes from the
rapid growth of primary-care retail clinics, the price
and quality competition in the segment of the U.S.
hospital market that caters to cash-paying foreign-
ers, and the development of markets for interna-
tional medical tourism.
The chapters on how health insurance works,
and the considerations guiding the design of
optimal health insurance, are among the most
valuable aspects of this book. They review
the idiosyncratic history of the development
of current health coverage, discuss optimal terms
of entry into an existing insurance pool, the
optimal terms of renewal, and considerations
surrounding the proper allocation of financial
responsibility between third-party insurance and
self-insurance. In general, third-party payment
worksbestwhenitisdeployedforcasesin
which the occurrence of medical expenditure is
not under individual control, the price of third-
party insurance is consequently low, and the
exercise of individual choice does not create risks
for others.
The final chapters emphasize the importance of
a “Do-No-Harm” approach to health policy. They
detail the Medicare and Medicaid reforms that
would be dictated by a concern for providing
proper economic incentives, and discuss the pro-
visions of the new health-care law that discourage
work, discriminate against the sick, enable fraud,
and reduce incentives to innovate.
Goodman adeptly presents the case for basing
health-care reform on the principles of economics.
Readers interested in the current health-care policy
debate will also find Priceless an excellent guide to
its major contours.
Linda Gorman
Health Care Policy Center,
Independence Institute
The Signal and the Noise: Why So
Many Predictions Fail—But Some
Don’t
By Nate Silver. 2013. New York, NY:
Penguin Press. Pp. 534. $27.95 hardcover.
Business Economics (2013) 48, 82–84.
doi:10.1057/be.2012.33
Why should business economists read a book
about seismology, climate change, Texas
hold ‘em poker and flu epidemics? First, because
Nate Silver tells entertaining and compelling stories
about all of these subjects. Second, because all of
these discussions, and many more, contain useful
messages for economic modelers, forecasters, and
analysts.
The first chapter, “A Catastrophic Failure of
Prediction,” covers ground close to home—the
failure of the bond-rating agencies and many
economists to see the bursting of the housing
bubble and subsequent recession, even after it was
under way. Silver follows this account with a dis-
cussion of political polling and punditry and tales
of successes in predicting which baseball players
would do well in the major leagues. (Silver is well-
known now for his New York Times column and
blog, “FiveThirtyEight,” which focuses on election
polls, but first achieved recognition for inventing
a system of analyzing baseball statistics.)
As Silver explains in the Introduction, his
purpose in drawing on these diverse fields is to “get
BOOK REVIEWS
82
AUTHOR COPY
you thinking about some of the most fundamental
questions that underlie the prediction problem.
How can we apply our judgment to the data—
without succumbing to our biases? When does
market competition make forecasts better—and
how can it make them worse? How do we reconcile
theneedtousethepastasaguidewithour
recognition that the future may be different?” (p. 16).
Each of these questions gets thoughtful treatment,
particularly in a chapter on economic forecasting,
“How to Drown in Three Feet of Water.”
In every field he delves into, Silver dives more
than three feet deep, sources his research metic-
ulously, and adds his own analyses. For instance, in
text and endnotes, he presents several plots and
statistical tests he performed on forecasters’ pre-
dictions regarding unemployment and real GDP.
(The book contains over 1,000 endnotes, encom-
passing a huge range of citations from scholarly
books and journals, as well as popular sources.)
He also weaves in accounts of interviews he con-
ducted with both notable and less heralded names.
(Among the economists: George Akerlof, Hal
Varian, Jan Hatzius, Lawrence Summers, Jeffrey
Sachs, and Robin Hanson.)
In Silver’s view, economists fail to make clear
that their forecasts rely on data that they know—or
should know—are uncertain and likely to be
revised. Even when they offer a range of outcomes,
they do not assign a high enough probability
to extreme (but not unprecedented) results. “In
December 2007, economists in the Wall Street
Journal forecasting panel predicted only a 38 per-
cent likelihood of a recession over the next year.
This was remarkable because, the data would later
reveal, the economy was already in recession at the
time. [emphasis in original] The economists in
another panel, the Survey of Professional Fore-
casters, thought there was less than a 1 in 500
chance that the economy would crash as badly as
it did” (p. 33).
After reviewing how widely and often economic
forecasts miss the mark, Silver acknowledges, “This
property of overconfident predictions has been
identified in many other fields, including medical
research, political science, finance, and psychol-
ogyy.But economists, perhaps, have fewer excuses
than those in other professionsy.Economic fore-
casters get more feedback than people in most
other professions, but they haven’t chosen to cor-
rect for their bias toward overconfidence” (p. 183).
He asserts, “one can plot the error made in annual
GDP predictions by the Survey of Professional
Forecasters against a time trend and find that there
has been no overall improvement since 1968”
(endnote 51, p. 481).
Silver is also critical of economists who use
models with an excessive number of explanatory
variables, saying, “they can talk themselves into
believing thatyanything that has any semblance
of economic meaning” can be a useful leading
indicator. “With so many economic variables to
pick from, you’re sure to find something that fits
the noise in the past data well. It’s much harder to
find something that identifies the signal; variables
that are leading indicators in one economic cycle
often turn out to be lagging ones in the next”
(pp. 186–187).
Unfortunately, Silver’s messages for econo-
mists are somewhat mixed. One is that “A fore-
caster should almost never ignore data, especially
when she is studying rare events like recessionsy,
about which there isn’t very much data to begin
with. Ignoring data is often a tip-off that the
forecaster is overconfident, or is overfitting her
model—that she is interested in showing off rather
than trying to be accurate” (p. 191). But just three
paragraphs later, he concedes that a “rationale
you’ll sometimes hear for throwing out data is that
there has been some sort of fundamental shift in the
problem you are trying to solve. Sometimes these
arguments are valid to a certain extent: the Ameri-
can economy is a constantly evolving thing and
periodically undergoes structural shiftsyAn eco-
nomic model conditioned on the notion that nothing
major will change is a useless one. But anticipating
these turning points is not easy” (pp. 192–193).
In discussing the often large revisions in data
on which economic forecasts are based, Silver de-
livers another mixed message. “So we should have
some sympathy for economic forecasters” (p. 194).
He immediately tempers this sympathy in an end-
note to the same sentence: “Although economists
often do not give enough attention to the distinc-
tion between real time and revised data when they
present their forecasts” (endnote 50, p. 481).
Perhaps surprisingly, after showing the wide
errors and lack of improvement over time in
economists’ forecasts, Silver advises, “If you’re
looking for an economic forecast, the best place to
turn is the average or aggregate prediction rather
than that of any one economist. My research into
the Survey of Professional Forecasters suggests
that these aggregate forecasts are about 20 percent
more accurate than the typical individual’s forecast
at predicting GDP, 10 percent better at predicting
BOOK REVIEWS
83
AUTHOR COPY
unemployment and 30 percent better at predicting
inflation” (pp. 197–198).
Without exonerating forecasters, Silver also
recognizes their customers “have to be better con-
sumers of forecasts. In the case of economic fore-
casting, that might mean turning the spotlight away
from charlatans with ‘black box’ models full of
random assortments of leading indicators toward
people like Jan Hatzius of Goldman Sachs who are
actually talking economic substance. It might also
mean placing more emphasis on the noisiness of
economic indicators and economic forecasts. Per-
haps initial estimates of GDP should be reported
with margins of errory.More broadly, it means
recognizing that the amount of confidence someone
expresses in a prediction is not a good indication of
its accuracy—to the contrary, these qualities are
often inversely correlated. Danger lurks in the
economy and elsewhere, when we discourage fore-
casters from making a full and explicit account of
the risks inherent in the world around us” (p. 203).
Silver’s concluding advice: Think probabi-
listically, not as if only one outcome is certain.
Know what your prior beliefs are and where they
are coming from, so you can guard against, or at
least recognize, biases in your forecasts. Make a
lot of forecasts—it’s the only way to get better.
Recognize that “our bias is to think we are better at
prediction than we really are. [May we be] a little
more modest about our forecasting abilities, and a
little less likely to repeat our mistakes” (p. 454).
Readers hoping for a Silver bullet for produc-
ing bull’s-eye forecasts may be disappointed. But
at least they’ll have enjoyed a thought-provoking
read, not just a serving of Silver platitudes.
Ken Simonson
Associated General Contractors of America
BOOK REVIEWS
84
Article
Full-text available
In our fast-forward times, the special issue ‘Back to the Future: Telling and Taming Anticipatory Media Visions and Technologies’ examines the future-making capacity of networked services and digital data. Its contributions ask about the role media play in forecasting the future and their part in bringing it about. And they are interested in the expectations and anticipatory visions that accompany the formation and spread of new media. Along these lines, the eight articles in this special issue explore the future-making dimension of new media. As a whole, they provide an empirically grounded analysis of the ways media reconfigure the relations and distances among present, past, and future times. The contributions delineate imaginaries of futures related to digital media. Furthermore, they attend to interventions into the plans and efforts of making futures and they inquire about the creation of differently vast and (un)certain horizons of expectation. Together, the articles share the assumption that mediated futures are actively accomplished and enacted; they do not simply appear or wait for us to arrive in them.
ResearchGate has not been able to resolve any references for this publication.