Technical ReportPDF Available

Analysis of the U.S. Environmental Protection Agency's Advanced Notice of Proposed Rulemaking for Greenhouse Gases Statement

Authors:

Abstract

Scientific understanding about the Earth's climate is tentative at best. As a result of uncertainties over what causes climate to change and how and when, there are rival theories and arguments among scientists about how to interpret the evidence. Rather than join these arguments, we have examined the processes that have been used to analyze the available data in order to derive forecasts of climate over the 21 st Century. We have concluded that the forecasting process reported on by the Intergovernmental Panel on Climate Change (IPCC) lacks a scientific basis...
To: Senator James M. Inhofe
From: Drs. J. Scott Armstrong and Kesten C. Green
Re: Your Request for an Analysis of the U.S. Environmental Protection Agency’s
Advanced Notice of Proposed Rulemaking for Greenhouse Gases
Statement
Scientific understanding about the Earth’s climate is tentative at best. As a result of uncertainties
over what causes climate to change and how and when, there are rival theories and arguments
among scientists about how to interpret the evidence.
Rather than join these arguments, we have examined the processes that have been used to analyze
the available data in order to derive forecasts of climate over the 21st Century. We have concluded
that the forecasting process reported on by the Intergovernmental Panel on Climate Change
(IPCC) lacks a scientific basis.
1. No scientific forecasts of the changes in the Earth’s climate. Currently, the only forecasts are
those based on the opinions of some scientists. Computer modeling was used to create
scenarios (i.e., stories) to represent the scientists’ opinions about what might happen. The
models were not intended as forecasting models (Trenberth 2007) and they have not been
validated for that purpose. Since the publication of our paper, no one has provided evidence
to refute our claim that there are no scientific forecasts to support global warming.
We conducted an audit of the procedures described in the IPCC report and found that they
clearly violated 72 scientific principles of forecasting (Green and Armstrong 2008). (No
justification was provided for any of these violations.) For important forecasts, we can see no
reason why any principle should be violated. We draw analogies to flying an aircraft or
building a bridge or performing heart surgerygiven the potential cost of errors, it is not
permissible to violate principles.
2. Improper peer review process. To our knowledge, papers claiming to forecast global
warming have not been subject to peer review by experts in scientific forecasting.
3. Complexity and uncertainty of climate render expert opinions invalid for forecasting. Expert
opinions are an inappropriate forecasting method in situations that involve high complexity
and high uncertainty. This conclusion is based on over eight decades of research. Armstrong
(1978) provided a review of the evidence and this was supported by Tetlock’s (2005) study
that involved 82,361 forecasts by 284 experts over two decades.
Long-term climate changes are highly complex due to the many factors that affect climate
and to their interactions. Uncertainty about long-term climate changes is high due to a lack of
good knowledge about such things as:
a) causes of climate change,
b) direction, lag time, and effect size of causal factors related to climate change,
c) effects of changing temperatures, and
2
d) costs and benefits of alternative actions to deal with climate changes (e.g., CO2
markets).
Given these conditions, expert opinions are not appropriate for long-term climate predictions.
4. Forecasts are needed for the effects of climate change. Even if it were possible to forecast
climate changes, it would still be necessary to forecast the effects of climate changes. In other
words, in what ways might the effects be beneficial or harmful? Here again, we have been
unable to find any scientific forecastsas opposed to speculationdespite our appeals for
such studies.
We addressed this issue with respect to studies involving the possible classification of
polar bears as threatened or endangered (Armstrong, Green, and Soon 2008). In our audits of
two key papers to support the polar bear listing, 41 principles were clearly violated by the
authors of one paper and 61 by the authors of the other. It is not proper from a scientific or
from a practical viewpoint to violate any principles. Again, there was no sign that the
forecasters realized that they were making mistakes.
5. Forecasts are needed of the costs and benefits of alternative actions that might be taken to
combat climate change. Assuming that climate change could be accurately forecast, it would
be necessary to forecast the costs and benefits of actions taken to reduce harmful effects, and
to compare the net benefit with other feasible policies including taking no action. Here again
we have been unable to find any scientific forecasts despite our appeals for such studies.
6. To justify using a climate forecasting model, one would need to test it against a relevant
naïve model. We used the Forecasting Method Selection Tree to help determine which
method is most appropriate for forecasting long-term climate change. A copy of the Tree is
attached as Appendix 1. It is drawn from comparative empirical studies from all areas of
forecasting. It suggests that extrapolation is appropriate, and we chose a naïve (no change)
model as an appropriate benchmark. A forecasting model should not be used unless it can be
shown to provide forecasts that are more accurate than those from this naïve model, as it
would otherwise increase error. In Green, Armstrong and Soon (2008), we show that the
mean absolute error of 108 naïve forecasts for 50 years in the future was 0.24°C.
7. The climate system is stable. To assess stability, we examined the errors from naïve forecasts
for up to 100 years into the future. Using the U.K. Met Office Hadley Centre’s data, we
started with 1850 and used that year’s average temperature as our forecast for the next 100
years. We then calculated the errors for each forecast horizon from 1 to 100. We repeated the
process using the average temperature in 1851 as our naïve forecast for the next 100 years,
and so on. This “successive updating” continued until year 2006, when we forecasted a single
year ahead. This provided 157 one-year-ahead forecasts, 156 two-year-ahead and so on to 58
100-year-ahead forecasts.
We then examined how many forecasts were further than 0.5°C from the observed
value. Fewer than 13% of forecasts of up to 65-years-ahead had absolute errors larger than
0.5°C. For longer horizons, fewer than 33% had absolute errors larger than 0.5°C. Given the
remarkable stability of global mean temperature, it is unlikely that there would be any
practical benefits from a forecasting method that provided more accurate forecasts.
8. Be conservative and avoid the precautionary principle. One of the primary scientific
principles in forecasting is to be conservative in the darkness of uncertainty. This principle
also argues for the use of the naive no-change extrapolation. Some have argued for the
3
precautionary principle as a way to be conservative. It is a political, not a scientific principle.
As we explain in our essay in Appendix 2, it is actually an anti-scientific principle in that it
attempts to make decisions without using rational analyses. Instead, cost/benefit analyses are
appropriate given the available evidence which suggests that temperature is just as likely to
go up as down. However, these analyses should be supported by scientific forecasts.
References
Armstrong, J.S., (1978, 1985), Long-range Forecasting. New York: John Wiley. (In full text at
http://jscottarmstrong.com)
Armstrong, J.S., K. C. Green, W. Soon, (2008), "Polar bear population forecasts: a public-policy
forecasting audit," Interfaces, 38, 382-405. (Working paper with commentary in full text at
http://publicpolicyforecasting.com)
Green, K.C. & J.S. Armstrong (2007), “Global warming: Forecasts by scientists versus scientific
forecasts,” Energy & Environment, 18, 997-1022. (Working paper in full text at
http://publicpolicyforecasting.com)
Green, K. C., J.S. Armstrong, W. Soon (2009), " Validity of Climate Change Forecasting for
Public Policy Decision Making," forthcoming, International Journal of Forecasting (Working
paper in full text at http://publicpolicyforecasting.com)
Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know?
Princeton Univ. Press.
Trenberth, K. (2007). Predictions of climate. [Retrieved June 2, 2008 from
http://blogs.nature.com/climatefeedback/2007/06/predictions_of_climate.html].
Information on the authors
We are experts in scientific forecasting methods. Dr. Armstrong has been working in the field for
48 years. He is a founder of the International Journal of Forecasting, Journal of Forecasting,
International Institute of Forecasters, and International Symposium on Forecasting, and the author
of Long-range Forecasting (1978, 1985), the Principles of Forecasting Handbook, and over 70
papers on forecasting. Dr. Green has developed two important new forecasting methods and has
published seven articles on forecasting. His first article was accompanied by six commentaries
and was awarded Best Paper of 2002-2003 by the International Journal of Forecasting. Dr Green
established publicpolicyforecasting.com to promote the use of scientific forecasting methods to
help improve public policy decision making. Along with Dr. Armstrong, he is a director of
forecastingprinciples.com.
Original submission: November 20, 2008; reformatted and updated January 26, 2009
4
Appendix 1: Forecasting methods selection Tree
5
Appendix 2
Uncertainty, the Precautionary Principle, and Climate Change
Kesten C. Green & J. Scott Armstrong
August 9, 2008
The precautionary principle is a political principle, not a scientific one. The principle is used to urge the
cessation or avoidance of a human activity in situations of uncertainty, just in case that activity might cause
harm to human health or the natural environment. There is an interesting discussion of the history of the
term in Wikipedia.
In practice, the precautionary principle is invoked when an interest group identifies an issue that can help it
to achieve its objectives. If the interest group is successful in its efforts to raise fears about the issue, the
application of the scientific method is rejected and a new orthodoxy is imposed. Government dictates
follow. People who dissent from the orthodox view are vilified, ostracized, and may have their livelihoods
taken away from them.
Consider the case of “climate change”. Warnings of dangerous manmade global warming from scientists,
politicians, and celebrities have received much publicity. They admonish us to dramatically reduce
emissions of CO2 in order to prevent disaster over the course of the 21st Century. Efforts have been made to
stifle a scientific approach to the issue. In an article titled “Veteran climate scientist says 'lock up the oil
men'”, James Hanson, who heads the NASA Goddard Institute for Space Studies, was quoted as suggesting
that those who promote the ideas of global warming skeptics should be “put on trial for high crimes against
humanity.” The skeptics themselves have been ejected from, for example, State Climatologist positions and
prevented from publishing research in mainstream journals, and they and their views are routinely attacked.
Much complexity and uncertainty surround climate change. The cumulative empirical evidence on proper
forecasting procedures suggests that the most appropriate method in this case is naïve extrapolation. In
simple terms, this means to forecast no change. Of course there will be change, but with current knowledge
there is no more reason to expect warming than to expect cooling.
As we describe in our paper, we have been unable to find any forecast derived from evidence-based
(scientific) forecasting methods that supports the contention that the world faces dangerous manmade
global warming.
Appeals for urgent curtailment of human activity “just in case” are often couched in ways that imply that
industrial societies are inherently sinful, rather than that there might be a problem to be dealt with. Indeed,
interpretation of the precautionary principle is subjective and it is arguable that it is being misapplied to the
issue of climate change.
Firstly, even if forecasts of increasing temperatures turned out to be accurate, predicted temperatures and
other conditions are within the range of variations that have been experienced in the past. There is no
evidence that the natural environment “prefers” relatively cool to relatively warm average temperatures. In
fact, life in general prefers warmth.
Secondly, curtailing human activity would harm people’s health by making them poorer than they would
otherwise have been. This is likely to be the case even if curtailing human activity happened to reduce
global average temperatures. When the situation is framed in this way, the precautionary principle dictates
that it is policies to curtail economically efficient human activity that should themselves be curtailed.
The outlook for the climate over the 21st Century is highly uncertain. There is a word in the English
language to express high uncertainty. That word is “ignorance”. And ignorance is not a basis for
responsible government action. We should expect our politicians to have the courage to resist interest
groups’ calls for action in the face of ignorance.
The precautionary principle brings to mind the slogan on the Ministry of Truth building in George Orwell’s
1984: “Ignorance is Strength.” Instead of this political principle, we hope that politicians will turn to
scientific principles for making public policy.
Technical Report
Full-text available
My submission relates particularly to the following clause in the Terms of Reference: Identify the central/benchmark projections which are being used as the motivation for international agreements to combat climate change; and consider the uncertainties and risks surrounding these projections.
Technical Report
Full-text available
My submission relates particularly to the following clause in the Terms of Reference: Identify the central/benchmark projections which are being used as the motivation for international agreements to combat climate change; and consider the uncertainties and risks surrounding these projections.
Article
Full-text available
Policymakers need to know whether prediction is possible and, if so, whether any proposed forecasting method will provide forecasts that are substantially more accurate than those from the relevant benchmark method. An inspection of global temperature data suggests that temperature is subject to irregular variations on all relevant time scales, and that variations during the late 1900s were not unusual. In such a situation, a "no change" extrapolation is an appropriate benchmark forecasting method. We used the UK Met Office Hadley Centre's annual average thermometer data from 1850 through 2007 to examine the performance of the benchmark method. The accuracy of forecasts from the benchmark is such that even perfect forecasts would be unlikely to help policymakers. For example, mean absolute errors for the 20- and 50-year horizons were 0.18 Â oC and 0.24 Â oC respectively. We nevertheless demonstrate the use of benchmarking with the example of the Intergovernmental Panel on Climate Change's 1992 linear projection of long-term warming at a rate of 0.03 Â oC per year. The small sample of errors from ex ante projections at 0.03 Â oC per year for 1992 through 2008 was practically indistinguishable from the benchmark errors. Validation for long-term forecasting, however, requires a much longer horizon. Again using the IPCC warming rate for our demonstration, we projected the rate successively over a period analogous to that envisaged in their scenario of exponential CO2 growth--the years 1851 to 1975. The errors from the projections were more than seven times greater than the errors from the benchmark method. Relative errors were larger for longer forecast horizons. Our validation exercise illustrates the importance of determining whether it is possible to obtain forecasts that are more useful than those from a simple benchmark before making expensive policy decisions.
Article
Full-text available
Policymakers need to know whether prediction is possible and, if so, whether any proposed forecasting method will provide forecasts that are substantially more accurate than those from the relevant benchmark method. An inspection of global temperature data suggests that temperature is subject to irregular variations on all relevant time scales, and that variations during the late 1900s were not unusual. In such a situation, a "no change" extrapolation is an appropriate benchmark forecasting method. We used the UK Met Office Hadley Centre's annual average thermometer data from 1850 through 2007 to examine the performance of the benchmark method. The accuracy of forecasts from the benchmark is such that even perfect forecasts would be unlikely to help policymakers. For example, mean absolute errors for the 20- and 50-year horizons were 0.18 � oC and 0.24 � oC respectively. We nevertheless demonstrate the use of benchmarking with the example of the Intergovernmental Panel on Climate Change's 1992 linear projection of long-term warming at a rate of 0.03 � oC per year. The small sample of errors from ex ante projections at 0.03 � oC per year for 1992 through 2008 was practically indistinguishable from the benchmark errors. Validation for long-term forecasting, however, requires a much longer horizon. Again using the IPCC warming rate for our demonstration, we projected the rate successively over a period analogous to that envisaged in their scenario of exponential CO2 growth--the years 1851 to 1975. The errors from the projections were more than seven times greater than the errors from the benchmark method. Relative errors were larger for longer forecast horizons. Our validation exercise illustrates the importance of determining whether it is possible to obtain forecasts that are more useful than those from a simple benchmark before making expensive policy decisions.
Article
Full-text available
In 2007, the Intergovernmental Panel on Climate Change's Working Group One, a panel of experts established by the World Meteorological Organization and the United Nations Environment Programme, issued its Fourth Assessment Report. The Report included predictions of dramatic increases in average world temperatures over the next 92 years and serious harm resulting from the predicted temperature increases. Using forecasting principles as our guide we asked: Are these forecasts a good basis for developing public policy? Our answer is “no”. To provide forecasts of climate change that are useful for policy-making, one would need to forecast (1) global temperature, (2) the effects of any temperature changes, and (3) the effects of feasible alternative policies. Proper forecasts of all three are necessary for rational policy making. The IPCC WG1 Report was regarded as providing the most credible long-term forecasts of global average temperatures by 31 of the 51 scientists and others involved in forecasting climate change who responded to our survey. We found no references in the 1056-page Report to the primary sources of information on forecasting methods despite the fact these are conveniently available in books, articles, and websites. We audited the forecasting processes described in Chapter 8 of the IPCC's WG1 Report to assess the extent to which they complied with forecasting principles. We found enough information to make judgments on 89 out of a total of 140 forecasting principles. The forecasting procedures that were described violated 72 principles. Many of the violations were, by themselves, critical. The forecasts in the Report were not the outcome of scientific procedures. In effect, they were the opinions of scientists transformed by mathematics and obscured by complex writing. Research on forecasting has shown that experts' predictions are not useful in situations involving uncertainly and complexity. We have been unable to identify any scientific forecasts of global warming. Claims that the Earth will get warmer have no more credence than saying that it will get colder.