Content uploaded by J. Scott Armstrong
Author content
All content in this area was uploaded by J. Scott Armstrong on Feb 09, 2021
Content may be subject to copyright.
Global Warming:
Forecasts by Scientists
versus
Scientific Forecasts
J. Scott Armstrong
The Wharton School
University of Pennsylvania
September 13, 2007
Conclusions
Predictions of global warming are
forecasts by scientists –not
scientific forecasts.
We have been unable to find a
scientific forecast to support global
warming.
Background
•Professor at the Wharton School since 1968
•A founder of:
–Journal of Forecasting
–International Journal of Forecasting
–forecastingprinciples.com
•Author of:
–Long-range Forecasting (1978; 1985) and
over 70 papers on forecasting
•Editor of
–Principles of Forecasting (handbook)
•Details and full-text of papers at
jscottarmstrong.com
Our contribution
Green & Armstrong, “Global Warming:
Forecasts by Scientists versus Scientific
Forecasts” forthcoming in Energy and
Environment*
• We are experts on forecasting methods, not
climate
• Neither of us has received funding for our
research on climate forecasting
• We have received sponsorship from the
International Institute of Forecasters for
publicpolicyforecasting.com and for
theclimatebet.com
*available at publicpolicyforecasting.com
Global climate does change—but
can we forecast the changes?
“A trend is a trend,
But the question is, will it bend?
Will it alter is course
Through some unforeseen force
And come to a premature end?”
Alec Cairncross 1969
Scientific forecasts
•“Forecasts derived using evidence-
based methods”
•Research over half a century has
produced many evidence-based findings
•Summarized as principles at
forecastingprinciples.com, a free and
easily accessible site, and in Principles
of Forecasting (Armstrong 2001)
Examples of Principles
•Experts’ unaided judgments have little
value in forecasting over time
•Agreement among experts is weakly
related to accuracy—but highly related
to confidence.
•Complex models harm forecast
accuracy
•Uncertainty calls for conservatism
The Forecasting Problem
For policy recommendations based on
global warming, forecasts must be
accurate for each of the following
areas:
1. Long-term temperature change
2. Effects of temperature changes
3. Effects of feasible policy changes
Forecasts of global
temperature change
•Green & Armstrong (2007) looked
primarily at forecasts of medium to
long-term temperature change.
Forecasts from climate modelers
•Some climate modelers claim that their
models do not make forecasts
•However, climate modelers do make forecasts:
The word “forecast” and its derivatives occurred 37
times, and “predict” and its derivatives occurred 90
times in the body of Chapter 8 of the 2007 IPCC
report.
•They use models to express their judgments—
what goes in and what comes out. Thus, they
make “expert forecasts.”
Other scientists see climate models as
expert opinions written in mathematics
•Pilkey & Pilkey-Jarvis (2007) concluded long-
term climate forecasts were based only on the
opinions of the scientists expressed in complex
mathematical terms—and use this quotation:
–“Today’s scientists have substituted mathematics for
experiments, and they wander off through equation
after equation and eventually build a structure which
has no relation to reality.”
–(Nikola Telsa, inventor and electrical engineer, 1934).
Are Experts’ Forecasts of
Global Climate Useful?
•Poor forecasts easy to find
1924: MacMillan reports signs of new ice age
1974: A major cooling widely considered to be
inevitable
•Seer-sucker theory proposed in 1978
–No matter how much evidence exists that seers do
not exist, seers will find suckers.
•Tetlo c k (2005) f o u nd s u p port f or t h i s th e o r y
with evaluation of 82,000 forecasts over 20
years.
Prior Reviews of Climate
Forecasting
•1985: Climate scientists ignored judgmental
forecasting principles. Forecasts then were
based on expert opinion (Delphi) studies.
•2006: Stern Review made no reference to
scientific forecasting
** No prior reviews of forecasting methodology
for climate models
Use of scientific literature
•Google searches in April 2007
revealed no citations of scientific
literature on forecasting in global
warming literature
Identifying key papers on
climate change
Sent requests to 240 climate experts (70% were
IPCC authors or reviewers),
“We want to know which forecasts people regard
as the most credible and how those forecasts were
derived…
–In your opinion, which scientific article is
the source of the most credible forecasts of
global average temperatures over the rest of
this century?”
51 people sent responses, of which
42 included references, of which
30 referred to latest IPCC report
Scientific Literature
in IPCC Chapter 8
Of the roughly 650 references cited in
IPCC 8, none had any obvious
relationship to evidence-based
forecasting methods
Forecasting audit standards
1. All elements of the forecasting process
are examined
2. Each principle supported by evidence
3. Ratings against each principle assessed
independently by two or more people
4. Ratings identify errors of omission and
of commission
5. Full disclosure of the ratings
Audit of IPCC Chapter 8
•Of the 140 principles in the Forecasting
Audit, we judged 127 relevant
•Each author rated the forecasting
methods independently, then we
resolved differences
•We were able to rate 89 principles of
which 72 principles were violated
Some important violations
1.4 Use only methods that are
better than a naïve model
7.1 Use simple forecasting
methods
9.3 Do not use fit
13.26 Test on ou t-of-sample data
Full disclosure & open
peer review
•Our audit is fully-disclosed at
publicpolicyforecasting.com
•Others invited to apply the
Forecasting Audit to Ch. 8—or to
another climate forecasting paper
and publish on the site.
•We welcome commentary and
open peer review on our paper.
No scientific forecast to date
• Climate is complex and poorly
understood.
• Much uncertainty
• Key (IPPC) forecasts violate
important principles.
In such conditions, climate models
are expected to be inferior to the
simple naïve model, which
assumes complete ignorance.
Is the naïve model best?
•Possibly not.
•Based on prior research, I would
recommend testing such methods as
–Extrapolation of (very) long-term trends
–Naïve model with drift
–Rule-based forecasting
–Simple models with well-founded causal
relationships (assuming these can be
identified & causal variables can be
forecasted)
–Combining forecasts from different methods
The Global
Warming Challenge
•Claims have been made that the Earth will
warm rapidly.
•These are not based on scientific forecasting
methods. Thus, the challenge:
Predict global mean temp over 10 years.
-Al Gore selects any current climate model
-Scott Armstrong will assume no change
Each deposits $10,000 in a trust fund in Dec.
2007. Value to winner’s charity in 2018.
Purpose of the Challenge
•While I expect the naïve method to be more
accurate, winning is not what’s important.
•A Gore/Armstrong collaboration can yield
benefits for public policy by fostering the use
of science in forecasting for public policy
with:
–evidence-based forecasting principles,
–comparative tests among a variety of forecasting
methods (going beyond the naïve method),
–proper validation tests (going beyond the simple test
in the challenge)
Updates on challenge provided at…
theclimatebet.com
Along with latest version of paper
1. Long-term temperature change *
2. Effects of changes
3. Effects of feasible policy changes
A failure in any of the three
problems negates public policy
recommendations
Forecasts of global warming
may be harmful if accepted
•Misallocation of resources away from
uses which would do more to make
peoples lives better
•Rejection of policy options due to false
premises:
–Evaluate policy recommendations on their
merits (e.g., energy taxes)—assessing both
intended and unintended consequences
Standards for
publicpolicyforecasting.com
•Researchers can publish audits.
•Content
–Full or partial evidence-based audits
–Full disclosure of the audit
•Moderated to avoid
–“advertisements”
–ad hominem arguments
•Attributed contributions with contact,
bio, and potential bias.
Public policy should be based
on scientific forecasting
Forecasts from expert judgment are
of no value
Forecasting should be judged
against evidence-based principles—
which are easy to find and freely
available
Conclusions
•We have been unable to find a single scientific
forecast to support global warming
•Forecasts by climate experts are of no value.
•Climate will change in the future, but…
–To da te , the most se nsib le f orecast i s fo r “no change”
because we are not sure of direction or magnitude
–Scientific forecasting methods can be tested to see if
any are more accurate than the “no change” forecast.