Content uploaded by Kesten Green
Author content
All content in this area was uploaded by Kesten Green on Jun 14, 2021
Content may be subject to copyright.
Forecasting for Public Policy
Kesten C. Green
University of South Australia Business School
& Ehrenberg-Bass Institute
Institute of Economic Affairs
11AM−12:30PM, Thursday 10th July 2014
Pyramid Room K4U.04, Kings Building
King’s College London
1
IEA July 2014 -
publicpolicyforecasting.com
Outline of the talk
1. Needs for forecasting
2. Evidence-based forecasting
3. Golden Rule of Forecasting
4. Commercial speech control
5. Forecasting for climate
policy
6. Q&A
2
IEA July 2014 -
publicpolicyforecasting.com
Objectives of the session
1. Introduce evidence-based or
scientific forecasting
2. Introduce the Golden Rule of
Forecasting Checklist
3. Describe 2 public policy
applications
3
IEA July 2014 -
publicpolicyforecasting.com
The fatal conceit
1. The knowledge problem
2. The chess pieces fallacy
4
IEA July 2014 -
publicpolicyforecasting.com
Chess pieces fallacy
5
Adam Smith commented on the theorist
who…
“…seems to imagine that he can arrange
the different members of a great
society with as much ease as the hand
arranges the different pieces upon a
chess-board.” *
*Adam Smith (1976). The theory of moral sentiments.
Liberty Classics: Indianapolis
Hayek on regulation
It was only when, because the economic system did not
accomplish all we wanted, we prevented it from doing what
it had been accomplishing, in an attempt to make it obey
us in an arbitrary way, that we realised that there was
anything to be understood. It was only incidentally, as a
by-product of the study of such isolated phenomena, that it
was gradually realised that many things which had been
taken for granted were, in fact, the product of a highly
complicated organism which we could only hope to
understand by the intense mental effort of systematic
inquiry. Indeed, it is probably no exaggeration to say that
economics developed mainly as the outcome of the
investigation and refutation of successive Utopian
proposals—if by ‘Utopian’ we mean proposals for the
improvement of undesirable effects of the existing system,
based upon a complete disregard of those forces which
actually enabled it to work.
6
Needs for forecasting*
1. Future without policy is uncertain
2. Effects of proposed policy is
uncertain**
3. Effects of alternatives are uncertain
4. Forecasts for all costs and benefits
over long term are needed
5. Independent auditing of forecasting is
needed
6. What is the alternative?
*For private sector, and for public when fatal conceit is
rejected.
** ”Unintended consequences.” 7
IEA July 2014 -
publicpolicyforecasting.com
The best and the brightest
•Group 1 consists of the best and
brightest experts in the world. They
know much accurate and relevant
information.
•Group 2 consists of intelligence
people. They have some relevant
knowledge and information.
•Which group would produce the most
accurate forecasts?
8
IEA July 2014 -
publicpolicyforecasting.com
Experts vs. novices for forecasting in complex
uncertain situations
Group 1 would do…
no better
…than Group 2.
9
IEA July 2014 -
publicpolicyforecasting.com
Evidence on accuracy of
unaided judgments by experts
Seer-sucker theory: “No matter how much
evidence exists that seers do not exist, suckers will
pay for the existence of seers” (Armstrong 1980;
based on a literature review).
Expert political judgment (Tetlock 2005)
284 experts made 82,361 forecasts of political,
social and economic developments over a 20-year
period.
They were little more accurate than novices.
10
IEA July 2014 -
publicpolicyforecasting.com
The Seer-Sucker Immunity
Everyone knows that they and their
favorite seers are immune from…
Seer-Suckeritis
…Just as everyone is above average.
11
IEA July 2014 -
publicpolicyforecasting.com
So what?
Surveying experts’ opinions about what
will happen in the distant future is
pointless…
…even when the implications of their
opinions are analyzed by computers.
What is the alternative?
12
IEA July 2014 -
publicpolicyforecasting.com
The alternative:
Evidence-based forecasting
•Use evidence-based methods
•(Analogous to evidence-based medicine)
•Fortunately, researchers have been
conducting experiments on how to forecast
under given conditions for nearly a century
13
IEA July 2014 -
publicpolicyforecasting.com
Forecasting principles project
•From 1995 to 2000
•Papers by 40 internationally recognized
forecasting experts
•Reviewed by another 123 leading
experts on forecasting
•Led to 139 “principles of forecasting”
published as a handbook in 2001
•Principles summarized by the
Forecasting Audit freeware
14
IEA July 2014 -
publicpolicyforecasting.com
A unifying theory for forecasting
•139 principles (now 140) is a lot
•Too many for polic y d e v e lop m e n t ?
•Is it possible to simplify?
•We (Armstrong, Green, & Graefe)
propose a unifying theory of forecasting…
•The Golden Rule of Forecasting
•…to make evidence-based forecasting
more accessible to forecasters &
reviewers
15
IEA July 2014 -
publicpolicyforecasting.com
Golden Rule of Forecasting is to
“
Be Conservative
”
*
Use cumulative knowledge on the situation
•Subject matter expertise moderated by
experimental evidence
•Behavior
•Long-run relationships
•Theory
•Levels
•Trend s
Use cumulative knowledge on evidence-based
forecasting methods
* Or
“
Forecast unto others as you would have them
forecast unto you.
”
16
IEA July 2014 -
publicpolicyforecasting.com
Are Golden Rule violations
easily spotted?
Yes, if the description of the forecasting
methods is adequate.
The Golden Rule of Forecasting Checklist
software can be used by novices following a
short preparation.
If the method description is not adequate, the
Golden Rule has been violated.
17
IEA July 2014 -
publicpolicyforecasting.com
When to be conservative
For all forecasting problems.
Especially important if situation is
•Complex
•Uncertain
–and
•Bias is likely…
A common conjunction in forecasting for
•government policies
•investment proposals
18
IEA July 2014 -
publicpolicyforecasting.com
Example of a complex uncertain
situation with biased forecasting
Demand forecasts for 24 large rail
transportation projects are
consistently optimistic, with a
median overestimate of 96 percent
for traffic (Flyvbjerg 2013).
IEA July 2014 -
publicpolicyforecasting.com 19
Example:
Conservative extrapolation guidance
Modify trends, if the…
a.series is variable or unstable
b.historical trend conflicts with causal
forces (contrary series)
c.forecast horizon is longer than the
historical series
d.short and long-term trend directions
are inconsistent
20
IEA July 2014 -
publicpolicyforecasting.com
Conservative causal model checklist
1. Use prior knowledge to select important
variables and estimate effects
2. Moderate effect estimates to reflect
uncertainty
3. Use all important variables
4. Combine models that use different
information, procedures.
21
IEA July 2014 -
publicpolicyforecasting.com
22
IEA July 2014 -
publicpolicyforecasting.com
Conservatism via combining
across methods & forecasters
23
Incorporates more prior knowledge
Reduces effects of:
–Data errors
–Computational errors
–Model selection errors
–Biased judgments
Error reductions, under ideal conditions,
exceed one-half (Graefe et al 2014)
IEA July 2014 -
publicpolicyforecasting.com
Experimental evidence on the
Golden Rule
Found 150 experimental
comparisons so far…
Number of comparisons supporting
vs rejecting the Golden Rule,
150:0.
Violating a guideline will typically
increase error by half again.
24
IEA July 2014 -
publicpolicyforecasting.com
Common causes of Golden Rule violations
A. Big data: Why so?
1. Encourages evasion of a priori analysis.
2. Includes irrelevant variables.
B. Complex statistical procedures: Why so?
Regression analysis of non-experimental data
on complex situations is invalid, no matter
what the sample size (Armstrong, “Illusions
in regression analysis”)
We were unable to find a single experimental
comparison showing that using complex methods
on big data improves forecast accuracy versus
reasonable alternatives.
25
IEA July 2014 -
publicpolicyforecasting.com
Barriers to adoption of the Golden Rule
26
Expensive: Need to use comprehensive prior
knowledge about relationships and
forecasting methods
Unimpressive:Methods are easy to understand
Boring: Forecasts not newsworthy (e.g.
PollyVote)
Clients pay for impressive, obscure, complex
methods, and stories based on
unaided expert opinions.
IEA July 2014 -
publicpolicyforecasting.com
GR checklist can be used by novices to
identify forecasts that violate the
Golden Rule
The simplicity of the Guidelines and the
use of checklists* gives clients the ability
to assess the worth of forecasts in a rapid
and inexpensive way.
If the method is too complex to
understand, give it a failing mark.
*Our paper summarizes the evidence on the
value of checklists.
27
IEA July 2014 -
publicpolicyforecasting.com
Possible action steps
A. Apply the Golden Rule Checklist to a
forecasting problem with which you are
familiar? How many of the 28 checklist items
were violated?
or
B. Consider a public policy issue, such as
“Will government mandated messages and
images on cigarette packs increase consumer
welfare?”
28
IEA July 2014 -
publicpolicyforecasting.com
If you have suggestions for improvements,
contact Andreas Graefe (a.graefe@lmu.de).
29
IEA July 2014 -
publicpolicyforecasting.com
Further information
For latest versions of…
1. Slides
2. Golden Rule of Forecasting working paper
3. Golden Rule of Forecasting Checklist
(available on paper, as Excel Sheet, and as an online tool)
5. Data on error increases from ignoring
guidelines
All at GoldenRuleofForecasting.com
30
IEA July 2014 -
publicpolicyforecasting.com
Conclusions
31
Golden Rule can be unpopular as a
constraint on dramatic forecasts
Applying complex statistical methods to
non-experimental data violates the Golden
Rule by ignoring knowledge about the
situation and forecasting methods.
Violating the Golden Rule of Forecasting
increases forecast errors enormously –
Violating a typical guideline increases error
by half again.
IEA July 2014 -
publicpolicyforecasting.com
Effects of government
mandated messages
(particularly disclaimers)
32
IEA July 2014 -
publicpolicyforecasting.com
Disclaimers fail to work as intended
Failure arises because disclaimers
often:
–relate to matters that are not vital for
the customer’s decision-making
–include negation (negative words)
–are based on opinion or tentative
knowledge
33IEA July 2014 -publicpolicyforecasting.com
Florida Statutory Disclaimer
NOTE: IMPLANT DENTISTRY IS NOT RECOGNIZED
AS A SPECIALTY AREA BY THE AMERICAN DENTAL
ASSOCIATION OR THE FLORIDA BOARD OF
DENTISTRY. THE AAID IS NOT RECOGNIZED AS A
BONA FIDE SPECIALTY ACCREDITING
ORGANIZATION BY THE AMERICAN DENTAL
ASSOCIATION OR THE FLORIDA BOARD OF
DENTISTRY.
There are two logical interpretations, one of which
seems unusual.
(1)It is a true statement if we substitute other
organizations (e.g. “Institute of Economic Affairs”) for
“AAID.”
(2)The AAID is not bona fide—the common
interpretation, and that constitutes false advertising.
34
IEA July 2014 -
publicpolicyforecasting.com
Why an experiment
rather than a survey?
Concern is effect of disclaimer on decision
making
•How much improvement in net welfare?
•Experiments are the only feasible way to assess
this
Surveys can show people are confused by
ads
•This has been well-known since the advent of
advertising surveys
•Particularly with disclaimers and corrective ads
•But surveys cannot assess the effects of a
disclaimer.
35IEA July 2014 -publicpolicyforecasting.com
Field work
Field work supervised by Gallup & Robinson
–They were not told the identity of the
client.
–They made suggestions on the wording
of the questions.
Experiment administered by CRG Global,
Florida (who also did not know client)
–In malls in Orlando, Daytona Beach, and
Fort Lauderdale.
–They trained the interviewers.
36IEA July 2014 -publicpolicyforecasting.com
Why face-to-face rather than
telephone interviews?
Realism
People read the ads and disclaimers
Best practice
When tangible item such as an ad needs to be
shown
»Jacoby and Handlin (1991)
37IEA July 2014 -publicpolicyforecasting.com
Is it proper to generalize from
experiments with convenience samples?
Yes, it is proper. All of the famous experiments (e.g., Milgram)
have been done with convenience samples. Same for medical
research.
In addition, it is consistent with research on this:
• Findings from experiments on students, yield valid results
•Are appropriate for experiments on decision making (DM)
processes
•Close match between lab and field*
•Accounting DM processes**
•Managerial DM***
*Locke 1986; **Liyanarachchi and Milne 2005; ***Remus
1996
38IEA July 2014 -publicpolicyforecasting.com
Has the experimental approach been
used as evidence in similar court cases?
•Berlex Labs vs Schering AG
•Use of disclaimers for Berlex drug (Berlex not
related the Schering AG)
•Convenience sample of physicians and
pharmacists for experiment by Berlex
•Schering AG commissioned a representative
sample experiment
•Findings supported the original conclusion
(From Jacoby and Szybillo 1994)
39IEA July 2014 -publicpolicyforecasting.com
Random assignment to
experimental treatments
Participants (subjects) shown two mock Yellow Pages ads
for implant dentists…
-Dr Alan Reed (AAID credentials shown)
-Dr Barry Smith (no AAID credentials shown)
…and asked (Q1) to recommend one of the dentists to a
friend in need of implant dental work and (Q11) which
one is better qualified to do implant dentistry?
Randomly assigned to receive the Dr Smith ad…
and a Dr Reed ad with one of the treatments:
• No disclaimer.
• Disclaimer required by Florida law
• A “modified” disclaimer
40
IEA July 2014 -
publicpolicyforecasting.com
41IEA July 2014 -publicpolicyforecasting.com
42IEA July 2014 -publicpolicyforecasting.com
IMPLANT DENTISTRY
Dr Alan Reed DDS
General Dentist
Fellow, American Academy of Implant Dentistry
Diplomate, American Board of Oral Implantology/Implant Dentistry
Implant dentistry is a technique by which artificial replacement teeth are
fastened to metal posts surgically implanted in a patient’s jaw bones.
Note: Implant dentistry is not recognized as a specialty area by the American
Dental Association or the Florida Board of Dentistry. The AAID is not
recognized as a bona fide specialty accrediting organization by the American
Dental Association or the Florida
Board of Dentistry.
CARD 2
43IEA July 2014 -publicpolicyforecasting.com
44IEA July 2014 -publicpolicyforecasting.com
IMPLANT DENTISTRY
Dr Barry Smith DDS
General Dentist
Implant dentistry is a technique by which artificial replacement teeth are
fastened to metal posts surgically implanted in a patient’s jaw bones.
CARD 4
45IEA July 2014 -publicpolicyforecasting.com
Definition of less-qualified
In the experiment, the dentist
who did not advertise
qualifications (Dr Smith) was
“less-qualified,” because ads
present best arguments.
46IEA July 2014 -publicpolicyforecasting.com
Why assume those with AAID
certificates are more skilled?
•Dentists with AAID credentials have
been subject to bona fide training and
assessment
•Our experiment gave the choice
between a dentist with training and
one without
•It is reasonable to assume that
someone who advertises training is
more skilled than one who does not
47IEA July 2014 -publicpolicyforecasting.com
What can you say about the
training provided by AAID?
The process
•Skill-based
•Learner responsibility
•Experiential learning with skilled practitioner
•Tes t sk il ls and knowledge
•Requires substantial study
Conforms to best practice for training
•“Learner Responsibility in Management Education, or
Ventures into Forbidden Research” Interfaces 1983.
•“Designing and Using Experiential Exercises,” in M. W.
DeLozier et al., Experiential Learning in Marketing
Education 1977.
48IEA July 2014 -publicpolicyforecasting.com
Recommend dentist to a friend
This is a survey about implant dentistry. That’s a
technique in which titanium metal posts are
permanently embedded in the patient’s jaw in a
series of surgical operations, and false teeth are
attached to the posts.
Please imagine that you have a friend who needs
implant dentistry. You find these two
advertisements in the Yellow Pages.
Please read these two Yellow Pages
advertisements carefully.
Q1 Please tell me, which of these dentists would
you recommend to your friend?
49
IEA July 2014 -publicpolicyforecasting.com
Selection of less-qualified dentist
When the FSD was used…
Inferior decisions were made by 1.6
times more people:
21% recommended the less-qualified Dr Smith
when FSD was used (n=100)
13% recommended the less-qualified Dr Smith
when FSD was not used (n=112)
Note: If there had been no improvement in decisions, there
would be no basis for the FSD. In practice, decision making
was harmed.
50IEA July 2014 -publicpolicyforecasting.com
Re-test of decision making (Q11)
A remarkably similar result for the
alternative way we posed the question:
Q11 Which of these dentists do you think is
better qualified to do implant dentistry?
Inferior judgments were made by 1.6 times
more people when the FSD was used:
19% erroneously judged Dr Smith was better-
qualified when the FSD was used
12% erroneously judged Dr Smith was better-
qualified when the FSD was not used
51IEA July 2014 -publicpolicyforecasting.com
Is an increase in poor decisions
from 13% to 21% important?
•It shows people make worse, not
better, decisions when exposed to the
FSD
•Effect is large:
Nearly 10% of those who would have
chosen a qualified dentist were
induced to chose an unqualified one
by the FSD
•FSD is not justified, adds to costs,
harms suppliers, harms customers.
52IEA July 2014 -publicpolicyforecasting.com
Retaining ads increased
inferior judgments (Q11)
When the FSD was retained…
Inferior judgments were made by 1.3 times
more people than when it wasn’t
retained.
22% erroneously judged Dr Smith was better-
qualified when FSD was retained (n=51)
16% erroneously judged Dr Smith was better-
qualified when FSD was returned (n=49)
The greater the exposure to the FSD the worse the
decisions.
53IEA July 2014 -publicpolicyforecasting.com
Effect of FSD on
less-educated people’s decisions
When exposed to FSD, people without a
college degree were about 1.5 times more
likely to make a poor decision than college-
educated people.
25% without college degree (n=48) made poor
decision
16% with college degree (n=50) did so
Without FSD, education had no effect on
decisions
54IEA July 2014 -publicpolicyforecasting.com
Effect of FSD on women’s decisions
Women exposed to the FSD were
1.8 times more likely to make a
poor decision than men
• 28% of women (n=46) made poor
decision
• 15% of men (n=54) did so
Without FSD, women made similar
decisions to men
55IEA July 2014 -publicpolicyforecasting.com
Majority of subjects did not understand
organizational arrangements after
exposure to FSD
Percent
•Recognized as specialty by ADA 68
•Recognized as specialty by FDB 67
•Recognized as accrediting body by ADA 64
•Recognized as accrediting body by FDB 60
•We measured only short-term effects. Another
experiment showed an effect opposite to the intention
of a disclaimer after 3 days*
*Skurnik, Ian, C. Yoon, D.C. Park & N. Schwarz (2005)
56IEA July 2014 -publicpolicyforecasting.com
FSD also imposes
costs on suppliers
FSD imposes costs on dentists who want
to inform prospective patients about their
credentials by
• reducing the effectiveness of their
ads
• increasing the cost of the ads
57IEA July 2014 -publicpolicyforecasting.com
Might a modified disclaimer work?
We tried to add clarity and to
remove the false statement, but
could not resolve two problems:
1)the organizational arrangements
are irrelevant to customers
2)“not” was unavoidable
58
IEA July 2014 -
publicpolicyforecasting.com
The modified disclaimer
“The American Academy of Implant
Dentistry (AAID) provides education,
training and testing in implant
dentistry. The AAID is the oldest U.S.
organization offering credentials in
the field. It is independent of the
American Dental Association, which
does not provide training or
certification in implant dentistry.”
59IEA July 2014 -publicpolicyforecasting.com
Modified disclaimer did not work
•Our attempt was not successful
15% who saw modified disclaimer made an inferior
decision
(vs 13% when no disclaimer)
•Prior research shows disclaimers are hard to
write
Listerine, same % misunderstood corrective message
whether written by the company or the FTC (Mazis
and Adkinson 1976)
•Is it possible to write a disclaimer that would
help consumers in this situation?
60IEA July 2014 -publicpolicyforecasting.com
“Commercial” speech restrictions:
Experimental evidence, and principle
•Discuss other evidence and principle…
•For details, see…
•Green, K. C. & Armstrong, J. S. (2012). Evidence on the
effects of mandatory disclaimers in advertising. Journal
of Public Policy and Marketing, 31, 293–304. [With
commentary, 305–324]
•Armstrong, J. S. & Green, K. C. (2012). Should we put
a price on free speech? Journal of Public Policy and
Marketing, 31, 325.
•Green, K. C. & Armstrong, J. S.(2012).Have the courts
protected free speech for business people? JPP&M
Internet Appendix.
61IEA July 2014 -publicpolicyforecasting.com
Global temperatures, long term:
Most important forecasting
problem of our time
62IEA July 2014 -publicpolicyforecasting.com
§Global warming described as the
“greatest moral problem of our time”
by former Australian PM Kevin Rudd
§Proposed policy responses very expensive
§Policy proposals based on explicit and
implicit forecasts
Conclusions from evidence-based
forecasting for global warming
•Policies rest on forecasts of (1) climate, (2)
climate effects, and (3) policy effects
•None are scientifically valid forecasts
–Violate relevant forecasting principles
–Violate the Golden Rule of Forecasting
•Validation studies show GW forecasts to be
inaccurate
•GW is an anti-scientific political movement…
similar to other manmade disaster alarms
63
IEA July 2014 -
publicpolicyforecasting.com
Global warming policies and the
requirement for forecasts
Rational policies require accurate forecasts from
validated evidence-based methods which
determine that:
1.Global warming will occur over the long term
2.Substantial welfare loss will result from
warming
3.Policies will reduce net welfare loss
All three are necessary
To date, there is n ot a single sci e n t i f ic forecast
supporting any of the three.
64
IEA July 2014 -
publicpolicyforecasting.com
No scientifically valid forecasts
for global warming
Based on our forecasting audit, the IPCC
forecasts violated 72 of the 89 relevant principles,
failing to:
1. Provide full disclosure of methods and
data
2. Assess reliability and validity of the data
3. Compare forecasts from different
methods
4. Be conservative,especially in situations
of high uncertainty and complexity.
See: Global Warming: Forecasts by Scientists
versus Scientific Forecasts
65
IEA July 2014 -
publicpolicyforecasting.com
No scientifically valid forecasts for
detrimental effects of global warming
•Are we currently at the optimum
temperature?
•Is warming more harmful than
cooling?
•Is warming more likely than
cooling?
66
IEA July 2014 -
publicpolicyforecasting.com
No scientifically valid forecasts for
cost-effective policies
•Two governme nt fore ca sting rep orts supported listing
polar bears as endangered
•Findings: Fewer than 14% of relevant forecasting
principles were properly applied
•Examples:
–Forecasts of up to 100 years were made based on an
analysis of data from five (consecutive) years
–Bias: “USGS Science Strategy to Support US Fish and
Wildlife Service Polar Bear Listing Decision”
–Refusal to provide the data used
•Source: Polar Bear Population Forecasts; Senate
Hearing
67
IEA July 2014 -
publicpolicyforecasting.com
Do the IPCC forecasts
follow the Golden Rule?
•Aware of the IPCC procedures, Green and
Armstrong and independently checked them
against the GR Guidelines “Checklist”
–Notice the parallel to medicine where checklists
have saved many lives
•You can do that yourself now that you know
where to find the checklist. It took us 10
minutes each.
–25 of the 27 guidelines judged to be relevant
–Of these, IPCC forecasts violated 25
68
IEA July 2014 -
publicpolicyforecasting.com
Why do forecasters violate the
Golden Rule?
•Use forecasts as a planning
document to motivate behavior
•Free to create any plan they like
•Forecasts restricted to determining
what will happen given the plan –
otherwise it is senseless
69
IEA July 2014 -
publicpolicyforecasting.com
People also violate the Golden
Rule because “Things are different
now”
How do we know they are
different?
Opinions of some experts
Be skeptical of this belief
As Eisenhower said, “Things are
more like they are now than they
ever have been.” 70
IEA July 2014 -
publicpolicyforecasting.com
Global temperature
forecasting
Please forecast the
missing years for the
series shown on the
two charts, each for
50 years
71
IEA July 2014 -
publicpolicyforecasting.com
Results from the professional
forecasters
•The graphs are nearly identical, so
how can they claim that the situation
is unique?
•Half of them predicted increased
temperatures for both.
•Series A temperatures actually went
down.
•So far, series B temperatures are
down.
72
IEA July 2014 -
publicpolicyforecasting.com
(Source of concept: W. Meyer 2009)
73IEA July 2014 -publicpolicyforecasting.com
Forecasting trends
A trend is a trend is a trend.
But the question is, will it bend?
Will it alter its course through some unforeseen
force and come to a premature end?
Cairncross
Golden Rule Guidelines:
Lacking clear causal support, damp the
trend.
If the causal forces are contrary with the
trend, forecast no trend. (The Simon-Ehrlich
bet was the most famous application of this
guideline.)
74
IEA July 2014 -
publicpolicyforecasting.com
Validation studies: Armstrong-
Gore “Bet”
•Armstrong proposed bet to Al Gore to
demonstrate the need for proper validation
studies. (Motivation: Julian Simon and his bet
with Ehrlich and Holdren).
•Terms:
–10 years for Global Mean temperature
•Positions:
–Gore: “tipping point” or even “too late”
–Armstrong: 10-years is too short
•At the start, Gore would have a ⅓chance to
win based on
random variation
•Updated monthly at theclimatebet.com
75
IEA July 2014 -
publicpolicyforecasting.com
76
IEA July 2014 -
publicpolicyforecasting.com
Long-term validation study on
global warming
To f orec ast und er hig h co mple xit y and uncertainty,
use the no-change “benchmark model.”
•We tested that against the forecast used by the
IPCC (0.03°C-per-year), using UK Met Office Hadley
Centre’s annual average temperature data, 1850-
2007.
•For long-term forecasts (1 to 100 years ahead), the
no-change model’s forecast errors were 1/7of the
IPCC model’s (7,550 predictions).
•For horizons 91 to 100, the no-change model’s
errors were 1/12 of the IPCC model’s.
Source: Green, Armstrong & Soon (2009)
77
IEA July 2014 -
publicpolicyforecasting.com
Possible improvements in global
mean temperature forecasting
1. Improve “nowcasting” (starting year
contains measurement error)
2. Adjust for known biases in the Hadley data
3. Use alternative data sets
4. Consider damped trends for short-term
5. Combine evidence-based forecasts
However, the no-change model provides
forecasts sufficiently accurate for policy
makers…
e.g. MAPE of 50-year-ahead forecasts =
0.24oC.
78
IEA July 2014 -
publicpolicyforecasting.com
Recent progress in
global mean temperature forecasting*
79
IEA July 2014 -
publicpolicyforecasting.com
*Green & Armstrong in “Climate change: The facts 2014”, IPA.
Available from http://www.kestencgreen.com/G&A-Skyfall.pdf
Evidence-based climate forecasts
for the 21st Century
•Each year’s global average temperature for the 100
years to 2113 will be the same, more or less, as
the 2013 figure
•Monitor against the University of Alabama at
Huntsville’s (UAH) lower troposphere temperature
–satellite-based better assessment of the global
average than HadCRUT3
–fully and openly documented
–therefore, less likely to be biased
•Our chapter for the IPA Facts book provides good
news: There is neither need to worry about climate
change, nor reason to take action.
80
IEA July 2014 -
publicpolicyforecasting.com
Short-term validation of our
polar bear population forecast
Described as a “tipping point” in the January 2008
U.S. Senate endangerment hearing… rapid decrease in
population forecast by polar bear scientists.
Armstrong, Green, and Soon report for Alaska
government and subsequent paper forecast that polar
bear population would increase in the short-run, then
level off.
Evidence since the hearings suggests that the polar
bear population has increased. (Mitch Taylor,
forthcoming).
81
IEA July 2014 -
publicpolicyforecasting.com
Conclusion: Global warming
movement is based on an anti-
scientific forecast of manmade
calamity
•We then forecast the outcome of this
movement by using “structured analogies”
•Julian Simon had originally used analogies
to show that the global warming movement
was a common social phenomenon (also
Madness of Crowds, etc.)
82
IEA July 2014 -
publicpolicyforecasting.com
Use of analogies in forecasting
•Often used to sell a forecast, but no
value for accuracy.
•Structured analogies seeks analogies
and then a clerk makes forecasts
based on what happened.
83
IEA July 2014 -
publicpolicyforecasting.com
Structured analogies procedure
1. Generate analogies:
Asked experts to describe analogies to the current
dangerous manmade global warming alarm; reviewed
the literature
2. Assess similarity:
Consulted published sources and assessed proposed
analogies for similarity to the target AGW situation;
selected those that met predetermined criteria
3. Thumbnail sketches:
Prepared brief descriptions of the analogies
4. Assess outcomes:
Obtained evidence analogies’ outcomes from published
sources
5. Forecast for target:
Derived forecasts about Global Warming Alarm from
modal analogy outcomes
8484
IEA July 2014 -
publicpolicyforecasting.com
Evidence on structured analogies
Conducted a validation study using 8 conflict situations
Example:Artists Protest (Chance: 17%; UJ: 10%; SA2+: 50%)
Findings from 97 structured analogies forecasts and 106 unaided
expert forecasts :
Method % accurate
Guessing 28
Expert unaided judgment 32
SA with two or more analogies 56
Source: Green and Armstrong (2007)
85
IEA July 2014 -
publicpolicyforecasting.com
Analogies to Global Warming
•71 analogies proposed (literature;
experts)
•26 met criteria
–Catastrophe
–Manmade
–Harm physical environment
•None were based on scientific evidence
•Government actions recommended in
all
86
IEA July 2014 -
publicpolicyforecasting.com
Examples of analogies to
global warming
1. Uncontrolled reproduction and
degeneration (eugenics) –1883
2. Soil erosion threat to agricultural
production –1934
3. DDT and cancer –1962
4. Population growth and famine –1968
5. Global cooling –1975
6. Electrical wiring and cancer –1979
7. Mercury in fish –2004
87
IEA July 2014 -
publicpolicyforecasting.com
Outcomes of the analogies
• Governments took action in 23 of the 26 analogies
• None of the predicted outcomes came true
• Movements gradually faded
• Government expenditures remained
GW is part of common social phenomenon and, like the
others, will slowly fade, but the harmful costs to
society will continue.
Source: Green and Armstrong,
“Effects of the Global Warming Alarm” (2011)
88
IEA July 2014 -
publicpolicyforecasting.com
What Simon said in 1992
Julian Simon claimed that
environmental alarmists are
motivated by political rather than
scientific objectives.
He concluded that doomsayers’
forecasts were always wrong and
that the environment keeps getting
better.
89
IEA July 2014 -
publicpolicyforecasting.com