To: Senator James M. Inhofe
From: Drs. J. Scott Armstrong and Kesten C. Green
Re: Your Request for an Analysis of the U.S. Environmental Protection Agency’s
Advanced Notice of Proposed Rulemaking for Greenhouse Gases
Scientific understanding about the Earth’s climate is tentative at best. As a result of uncertainties
over what causes climate to change and how and when, there are rival theories and arguments
among scientists about how to interpret the evidence.
Rather than join these arguments, we have examined the processes that have been used to analyze
the available data in order to derive forecasts of climate over the 21st Century. We have concluded
that the forecasting process reported on by the Intergovernmental Panel on Climate Change
(IPCC) lacks a scientific basis.
1. No scientific forecasts of the changes in the Earth’s climate. Currently, the only forecasts are
those based on the opinions of some scientists. Computer modeling was used to create
scenarios (i.e., stories) to represent the scientists’ opinions about what might happen. The
models were not intended as forecasting models (Trenberth 2007) and they have not been
validated for that purpose. Since the publication of our paper, no one has provided evidence
to refute our claim that there are no scientific forecasts to support global warming.
We conducted an audit of the procedures described in the IPCC report and found that they
clearly violated 72 scientific principles of forecasting (Green and Armstrong 2008). (No
justification was provided for any of these violations.) For important forecasts, we can see no
reason why any principle should be violated. We draw analogies to flying an aircraft or
building a bridge or performing heart surgery—given the potential cost of errors, it is not
permissible to violate principles.
2. Improper peer review process. To our knowledge, papers claiming to forecast global
warming have not been subject to peer review by experts in scientific forecasting.
3. Complexity and uncertainty of climate render expert opinions invalid for forecasting. Expert
opinions are an inappropriate forecasting method in situations that involve high complexity
and high uncertainty. This conclusion is based on over eight decades of research. Armstrong
(1978) provided a review of the evidence and this was supported by Tetlock’s (2005) study
that involved 82,361 forecasts by 284 experts over two decades.
Long-term climate changes are highly complex due to the many factors that affect climate
and to their interactions. Uncertainty about long-term climate changes is high due to a lack of
good knowledge about such things as:
a) causes of climate change,
b) direction, lag time, and effect size of causal factors related to climate change,
c) effects of changing temperatures, and
d) costs and benefits of alternative actions to deal with climate changes (e.g., CO2
Given these conditions, expert opinions are not appropriate for long-term climate predictions.
4. Forecasts are needed for the effects of climate change. Even if it were possible to forecast
climate changes, it would still be necessary to forecast the effects of climate changes. In other
words, in what ways might the effects be beneficial or harmful? Here again, we have been
unable to find any scientific forecasts—as opposed to speculation—despite our appeals for
We addressed this issue with respect to studies involving the possible classification of
polar bears as threatened or endangered (Armstrong, Green, and Soon 2008). In our audits of
two key papers to support the polar bear listing, 41 principles were clearly violated by the
authors of one paper and 61 by the authors of the other. It is not proper from a scientific or
from a practical viewpoint to violate any principles. Again, there was no sign that the
forecasters realized that they were making mistakes.
5. Forecasts are needed of the costs and benefits of alternative actions that might be taken to
combat climate change. Assuming that climate change could be accurately forecast, it would
be necessary to forecast the costs and benefits of actions taken to reduce harmful effects, and
to compare the net benefit with other feasible policies including taking no action. Here again
we have been unable to find any scientific forecasts despite our appeals for such studies.
6. To justify using a climate forecasting model, one would need to test it against a relevant
naïve model. We used the Forecasting Method Selection Tree to help determine which
method is most appropriate for forecasting long-term climate change. A copy of the Tree is
attached as Appendix 1. It is drawn from comparative empirical studies from all areas of
forecasting. It suggests that extrapolation is appropriate, and we chose a naïve (no change)
model as an appropriate benchmark. A forecasting model should not be used unless it can be
shown to provide forecasts that are more accurate than those from this naïve model, as it
would otherwise increase error. In Green, Armstrong and Soon (2008), we show that the
mean absolute error of 108 naïve forecasts for 50 years in the future was 0.24°C.
7. The climate system is stable. To assess stability, we examined the errors from naïve forecasts
for up to 100 years into the future. Using the U.K. Met Office Hadley Centre’s data, we
started with 1850 and used that year’s average temperature as our forecast for the next 100
years. We then calculated the errors for each forecast horizon from 1 to 100. We repeated the
process using the average temperature in 1851 as our naïve forecast for the next 100 years,
and so on. This “successive updating” continued until year 2006, when we forecasted a single
year ahead. This provided 157 one-year-ahead forecasts, 156 two-year-ahead and so on to 58
We then examined how many forecasts were further than 0.5°C from the observed
value. Fewer than 13% of forecasts of up to 65-years-ahead had absolute errors larger than
0.5°C. For longer horizons, fewer than 33% had absolute errors larger than 0.5°C. Given the
remarkable stability of global mean temperature, it is unlikely that there would be any
practical benefits from a forecasting method that provided more accurate forecasts.
8. Be conservative and avoid the precautionary principle. One of the primary scientific
principles in forecasting is to be conservative in the darkness of uncertainty. This principle
also argues for the use of the naive no-change extrapolation. Some have argued for the
precautionary principle as a way to be conservative. It is a political, not a scientific principle.
As we explain in our essay in Appendix 2, it is actually an anti-scientific principle in that it
attempts to make decisions without using rational analyses. Instead, cost/benefit analyses are
appropriate given the available evidence which suggests that temperature is just as likely to
go up as down. However, these analyses should be supported by scientific forecasts.
Armstrong, J.S., (1978, 1985), Long-range Forecasting. New York: John Wiley. (In full text at
Armstrong, J.S., K. C. Green, W. Soon, (2008), "Polar bear population forecasts: a public-policy
forecasting audit," Interfaces, 38, 382-405. (Working paper with commentary in full text at
Green, K.C. & J.S. Armstrong (2007), “Global warming: Forecasts by scientists versus scientific
forecasts,” Energy & Environment, 18, 997-1022. (Working paper in full text at
Green, K. C., J.S. Armstrong, W. Soon (2009), " Validity of Climate Change Forecasting for
Public Policy Decision Making," forthcoming, International Journal of Forecasting (Working
paper in full text at http://publicpolicyforecasting.com)
Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know?
Princeton Univ. Press.
Trenberth, K. (2007). Predictions of climate. [Retrieved June 2, 2008 from
Information on the authors
We are experts in scientific forecasting methods. Dr. Armstrong has been working in the field for
48 years. He is a founder of the International Journal of Forecasting, Journal of Forecasting,
International Institute of Forecasters, and International Symposium on Forecasting, and the author
of Long-range Forecasting (1978, 1985), the Principles of Forecasting Handbook, and over 70
papers on forecasting. Dr. Green has developed two important new forecasting methods and has
published seven articles on forecasting. His first article was accompanied by six commentaries
and was awarded Best Paper of 2002-2003 by the International Journal of Forecasting. Dr Green
established publicpolicyforecasting.com to promote the use of scientific forecasting methods to
help improve public policy decision making. Along with Dr. Armstrong, he is a director of
Original submission: November 20, 2008; reformatted and updated January 26, 2009
Appendix 1: Forecasting methods selection Tree
Uncertainty, the Precautionary Principle, and Climate Change
Kesten C. Green & J. Scott Armstrong
August 9, 2008
The precautionary principle is a political principle, not a scientific one. The principle is used to urge the
cessation or avoidance of a human activity in situations of uncertainty, just in case that activity might cause
harm to human health or the natural environment. There is an interesting discussion of the history of the
term in Wikipedia.
In practice, the precautionary principle is invoked when an interest group identifies an issue that can help it
to achieve its objectives. If the interest group is successful in its efforts to raise fears about the issue, the
application of the scientific method is rejected and a new orthodoxy is imposed. Government dictates
follow. People who dissent from the orthodox view are vilified, ostracized, and may have their livelihoods
taken away from them.
Consider the case of “climate change”. Warnings of dangerous manmade global warming from scientists,
politicians, and celebrities have received much publicity. They admonish us to dramatically reduce
emissions of CO2 in order to prevent disaster over the course of the 21st Century. Efforts have been made to
stifle a scientific approach to the issue. In an article titled “Veteran climate scientist says 'lock up the oil
men'”, James Hanson, who heads the NASA Goddard Institute for Space Studies, was quoted as suggesting
that those who promote the ideas of global warming skeptics should be “put on trial for high crimes against
humanity.” The skeptics themselves have been ejected from, for example, State Climatologist positions and
prevented from publishing research in mainstream journals, and they and their views are routinely attacked.
Much complexity and uncertainty surround climate change. The cumulative empirical evidence on proper
forecasting procedures suggests that the most appropriate method in this case is naïve extrapolation. In
simple terms, this means to forecast no change. Of course there will be change, but with current knowledge
there is no more reason to expect warming than to expect cooling.
As we describe in our paper, we have been unable to find any forecast derived from evidence-based
(scientific) forecasting methods that supports the contention that the world faces dangerous manmade
Appeals for urgent curtailment of human activity “just in case” are often couched in ways that imply that
industrial societies are inherently sinful, rather than that there might be a problem to be dealt with. Indeed,
interpretation of the precautionary principle is subjective and it is arguable that it is being misapplied to the
issue of climate change.
Firstly, even if forecasts of increasing temperatures turned out to be accurate, predicted temperatures and
other conditions are within the range of variations that have been experienced in the past. There is no
evidence that the natural environment “prefers” relatively cool to relatively warm average temperatures. In
fact, life in general prefers warmth.
Secondly, curtailing human activity would harm people’s health by making them poorer than they would
otherwise have been. This is likely to be the case even if curtailing human activity happened to reduce
global average temperatures. When the situation is framed in this way, the precautionary principle dictates
that it is policies to curtail economically efficient human activity that should themselves be curtailed.
The outlook for the climate over the 21st Century is highly uncertain. There is a word in the English
language to express high uncertainty. That word is “ignorance”. And ignorance is not a basis for
responsible government action. We should expect our politicians to have the courage to resist interest
groups’ calls for action in the face of ignorance.
The precautionary principle brings to mind the slogan on the Ministry of Truth building in George Orwell’s
1984: “Ignorance is Strength.” Instead of this political principle, we hope that politicians will turn to
scientific principles for making public policy.