Kesten Green

University of South Australia | UniSA · UniSA Business

Findings from tests of the predictive validity of causal models estimated using ordinary-least-squares (OLS) multiple regression were presented. Out-of-sample forecasts from 99 published models were compared with those from models estimated using nine simple and conservative alternative methods and one naive model. Forecast errors from models estim...

Slides used for a talk to the Climate Science and Economics Group on May 8 2024. The talk described findings from research to assess the predictive validities and reliabilities of solar and anthropogenic models of NH surface temperatures relative to a benchmark model of no change. The relationship between models’ statistical fits and their predicti...

Preprint of: Green, K. C. (2024). J. Scott Armstrong (1937 - 2023): Iconoclast and Champion of Science for Practical Purposes. Foresight: The International Journal of Applied Forecasting, 72 (1), 5-7. ...Much of Scott’s career had been devoted to making useful scientific findings on forecasting accessible to practitioners as well as to researcher...

Lott and Varney (2022) estimated a model of the effect of immigrant population numbers, as a proportion of the total population, on the homicide victim rate for 31 European countries for which data were available for some or all years between 2010 to 2020 amounting to 315 observations in total. 1 Their ordinary least squares (OLS) multiple regressi...

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly fail...

This chapter addresses how stakeholders of scientific research can assess whether research is compliant with the scientific method and then promote useful scientific research to improve products, services, processes, methods, and decision-making. We address stakeholders in sections for universities, scientific journals, governments, regulators and...

Objectivity underlies all of the criteria for complying with the scientific method. Only an objective description of an objective study and its findings can be considered scientific. Yet objectivity does not come naturally, and so scientists must struggle to overcome a tendency to advocate for subjectively preferred hypotheses when they practice sc...

The invitation for those nominating candidates for the Nobel Prize in economics, the “Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel,” described the award of the prize as being “based solely on scientific merit.” No criteria for judging scientific merit were provided, but nominators were directed to “consider origin and gend...

Gerd Gigerenzer, Director of the Max Planck Institute for Human Development, commented on reading papers in the Journal of Experimental Psychology from the 1920s and 1930s. He observed that “This was professionally a most depressing experience, but not because these articles were methodologically mediocre. On the contrary, many of them make today’s...

We believe that the primary role of a scientist is to make discoveries that can help to improve peoples’ lives, whether directly such as through the discovery of a vaccine against a disease or indirectly such as through the invention of a procedure that can improve efficiency or lead to better decisions. In this chapter, we provide practical advice...

Researchers who have been applying the scientific method to important problems for over 20 centuries are responsible for saving lives and improving our quality of life. Their efforts have provided us with the comforts and the myriad of opportunities that we have to live fulfilling lives that could barely be imagined in earlier times.

In this chapter, we raise making a useful contribution as a motive for university researchers, discuss some of the recent history of government involvement in science, and examine the evidence on whether or not government funding and regulation of research is beneficial.

List of works referred to in Armstrong & Green (2022) The Scientific Method

The scientific method is largely responsible for improving life expectancies and the quality of life over the past 2000 years. Individual scientists, in their efforts to discover how things work and how to make them better have used the method on their own or in collaboration with others to make the world a better place.

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly fail...

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly fail...

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly fail...

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly fail...

We asked 52 university students to make predictions about conflict situations. We gave each student four conflicts from a set of eight. We asked half our participants to assume there were 100 situations similar to a target conflict. We then asked them “…in how many of these situations would…” each one of a list of possible outcomes occur? The full...

Rules for choosing a single-decision forecast from a set of up to five analogies that have been rated for similarity to a target conflict

From “Bad Environmental and Resource Scares”, Chapter 18 of Julian L Simon’s 1996 “ The Ultimate Resource 2”, Princeton University Press, pp 258-273, and “Healing the Planet” afternote on p 274. Reproduced with the kind permission of the Simon family.

The document provides a list of references to articles, books, and movies relevant to the study of regulation.

The document contains quotations from notable economists, politicians, and other commentators that are in favour of government regulations

The document contains quotations by notable economists, politicians, and other commentators

Proposed remedies for harmful regulation

A list of institutions conducting research on regulations, with links

Katsikopoulos et al. (2021) found that the simple and easily understood recency heuristic-which uses a single historical observation to forecast week-ahead percentage of doctor visits associated with influenza symptoms-reduced forecast errors by nearly one-half compared to Google Flu Trends' (GFT's) complex and opaque machine learning model-which u...

The Global Warming Challenge, aka The Climate Bet (theclimatebet.com), is a record of an experiment to test the relative accuracy of Al Gore's 2007 dangerous man made global warming "tipping point" alarming projection relative to the no-change (no-trend) forecasts of Green, Armstrong, and Soon (2009) proposed by Scott Armstrong in his challenge to...

Flyer for Armstrong & Green presentation at ICCC Ten Washington, D.C., Including the Golden Rule of Forecasting Checklist and abstracts of papers with evidence on "Long-range climate and public policy forecasting"

Ten checklists for Armstrong & Green's "The Scientific Method" book in the form of pdf forms. For more information on "The Scientific Method: A Guide to Finding Useful Knowledge" (Cambridge, 2022), see thescientificmethod.info.

Relationship between relative forecast accuracy and R-squares for 99 models

"Do we face dangerous global warming?" Scott Armstrong gave a talk at Lehigh University on June 7, 2019 about research with Kesten Green on scientific climate forecasting. The talk was invited by the graduating Class of 1959, of which he is a member, for their 60th Reunion.

Purpose: Commentary on M4-Competition and findings to assess the contribution of data models--such as from machine learning methods--to improving forecast accuracy. Methods: (1) Use prior knowledge on the relative accuracy of forecasts from validated forecasting methods to assess the M4 findings. (2) Use prior knowledge on forecasting principles...

Purpose: Commentary on M4-Competition and findings to assess the contribution of data models—such as from machine learning methods—to improving forecast accuracy. Methods: (1) Use prior knowledge on the relative accuracy of forecasts from validated forecasting methods to assess the M4 findings. (2) Use prior knowledge on forecasting principles and...

In the mid-1900s, there were two streams of thought about forecasting methods. One stream-led by econometricians-was concerned with developing causal models by using prior knowledge and evidence from experiments. The other was led by statisticians, who were concerned with identifying idealized "data generating processes" and with developing models...

Problem Do conservative econometric models that comply with the Golden Rule of Forecasting provide more accurate forecasts? Methods To test the effects of forecast accuracy, we applied three evidence-based guidelines to 19 published regression models used for forecasting 154 elections in Australia, Canada, Italy, Japan, Netherlands, Portugal, Spai...

Relative absolute error (RAE) of forecasts from damping compared to forecasts from the original regression models. (DOCX)

Relative absolute error (RAE) of forecasts from equalizing compared to forecasts from the original regression models. (DOCX)

Commentary of the findings of the M4-Competition presented by Scott Armstrong at the M4 Conference on Monday, December 10, 2018 in New York City.

The contribution of regression analysis (econometrics) to advertising and media decision-making is questioned and found wanting. Econometrics cannot be expected to estimate valid and reliable forecasting models unless it is based on extensive experimental data on important variables, across varied conditions. This article canvasses alternative, evi...

Problem: Multiple regression analysis (MRA) is commonly used to develop forecasting models that inform policy and decision making, but the technique does not appear to have been validated for that purpose. Methods: The predictive validity of published least squares MRA models is tested against naive benchmarks, alternative methods that are either p...

We examined two papers with long-range forecasts of global mean temperatures (IPCC 2007, and Green, Armstrong, & Soon 2009) that included forecasts of global mean temperatures, and one paper that used the IPCC projections to forecast a dramatic decline in the population of polar bears for compliance with the scientific method. Ratings for complianc...

Problem: The scientific method is unrivaled for generating useful knowledge, yet papers published in scientific journals frequently violate the scientific method. Methods: A definition of the scientific method was developed from the writings of pioneers of the scientific method including Aristotle, Newton, and Franklin. The definition was used as...

ironlawofregulation.com redirects here. More material relevant to regulation is listed below the following slideshow. _________________________________________ What most in this room believe… The general welfare is greatest when the “simple and obvious system of natural liberty” that Adam Smith described prevails. Who can really know better…what is...

Problem How to help practitioners, academics, and decision makers use experimental research findings to substantially reduce forecast errors for all types of forecasting problems. Methods Findings from our review of forecasting experiments were used to identify methods and principles that lead to accurate forecasts. Cited authors were contacted to...

Decision making is improved by avoiding unaided expert judgment, and using structured judgmental procedures instead. The Red Team-Blue Team approach is one such structured approach. It recognizes that it is difficult for people-including scientists and public officials-to remain objective about the consequences of public policies and regulations. T...

Problem: How to help practitioners, academics, and decision makers use experimental research findings to substantially reduce forecast errors for all types of forecasting problems. Methods: Findings from our review of forecasting experiments were used to identify methods and principles that lead to accurate forecasts. Cited authors were contacted...

Discusses the scientific evidence of the effects of regulation, and describes four checklists for assessing whether a regulation is likely to provide substantive net benefit.

Earlier versions of this Tree appear in various publications and presentations. For more information on the methods and how to implement them, see "Forecasting Methods and Principles: Evidence-based Checklists."

See also Green (2018) "The harm of regulation: Evidence and solutions" a presentation at the 6th Annual Friedman Conference. DOI: 10.13140/RG.2.2.29921.02407

Notes for an interview with Steve Bannon on Breitbart London Radio

Purpose: To respond to issues posed in the four commentaries on Armstrong, Du, Green and Graefe (this issue) regarding the immediate usefulness of that paper’s test of advertisements’ compliance with persuasion principles, and regarding the need for further research. Approach: Address commentators’ concerns using logic, prior research findings, an...

Purpose – This paper aims to test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine. Design/methodology/approach – Scores were calculated from the ratings o...

Purpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine. Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96...

This article proposes a unifying theory, or the Golden Rule, of forecasting. The Golden Rule of Forecasting is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek out and use all knowledge relevant to the problem, including knowledge of methods...

The Golden Rule of Forecasting is a general rule that applies to all forecasting problems. The Rule was developed using logic and was tested against evidence from previously published comparison studies. The evidence suggests that a single violation of the Golden Rule is likely to increase forecast error by 44%. Some commentators argue that the Rul...

The Golden Rule of Forecasting is a general rule that applies to all forecasting problems. The Rule was developed using logic and was tested against evidence from previously published comparison studies. The evidence suggests that a single violation of the Golden Rule is likely to increase forecast error by 44%. Some commentators argue that the Rul...

This article proposes a unifying theory, or Golden Rule, of forecasting. The Golden Rule of Forecasting is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek out and use all knowledge relevant to the problem, including knowledge of methods val...

A checklist of requirements for following The Golden Rule of Forecasting. From Armstrong, Green, and Graefe (2015).

The Golden Rule of Forecasting requires forecasters to be conservative by forecasting in a way that is consistent with cumulative knowledge about the situation and about forecasting. Violating the average guideline increases forecast errors by more than 40%. We examined whether the United Nations Intergovernmental Panel on Climate Change’s (IPCC’s)...

With the objective of improving the accuracy of election forecasts, we examined three evidence-based forecasting guidelines that are relevant to forecasting with causal models. The guidelines suggest: (1) modifying estimates of the strength of variable effects to account for uncertainty, (2) combining forecasts from diverse models, and (3) utilizin...

The United Nations Intergovernmental Panel on Climate Change’s (IPCC’s) forecast of dangerous manmade global warming is an important influence on public policy decisions in many countries. Rational policies require that policy makers obtain forecasts from evidence-based methods. A review of comparative studies found that sophisticatedly simple meth...

The Golden Rule of Forecasting Checklist was used to evaluate IPCC “business as usual” global warming scenario and the Green, Armstrong & Soon no-change model forecasts. Consensus ratings by Armstrong and Green indicated that of 20 relevant Golden Rule Checklist guidelines: • the IPCC scenarios followed none • the no-change model followed 95%

This article introduces this JBR Special Issue on simple versus complex methods in forecasting. Simplicity in forecasting requires that (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by dec...

This article introduces the Special Issue on simple versus complex methods in forecasting. Simplicity in forecasting requires that (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models,forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by decision-...

Purpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine.Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96...

Purpose: To respond to issues posed in the four commentaries on Armstrong, Du, Green and Graefe (this issue) regarding the immediate usefulness of that paper’s test of advertisements’ compliance with persuasion principles, and regarding the need for further research. Approach: Address commentators’ concerns using logic, prior research findings, and...

Earlier versions of this Tree appear in various publications and presentations. Note that the presence of a method in the Tree does not signal its predictive validity. For evidence on which methods are valid, see "Forecasting Methods and Principles: Evidence-based Checklists."

The Golden Rule of Forecasting counsels forecasters to be conservative when making forecasts. We tested the value of three of the four Golden Rule guidelines that apply to causal models: modify effect estimates to reflect uncerainty; use all important variables; and combine forecasts from diverse models. These guidelines were tested using out-of-sa...

A talk to Institute of Economic Affairs summer interns at King's College London. [publicpolicyforecasting.com redirects here. Related research is listed below the following slideshow.]

A letter to Mr Wanamaker about the findings of Armstrong, Du, Green, & Graefe (2015).

The Golden Rule of Forecasting requires that forecasters be conservative by making proper use of cumulative knowledge and by not going beyond that knowledge. The procedures that have been used to forecast dangerous manmade global warming violate the Golden Rule. Following the scientific method, we investigated competing hypotheses on climate change...

Test your forecasting skills: Print this page and draw in your forecasts The charts show monthly global mean temperatures over two half centuries (both during the industrial era). The task is to draw in your forecasts for the next 25 years for both charts.

This article reviews experimental evidence on the effects of policies intended to promote behavior by firms that is more socially responsible and less socially irresponsible. Corporate social responsibility (CSR) can provide firms with opportunities for profit, but changes are likely to increase total welfare only if firms adopt them freely and wit...

The field of forecasting is concerned with making statements about matters that are currently unknown. The terms "forecast," "prediction," "projections," and "prognosis" are interchangeable as commonly used. Forecasting is also concerned with the effective presentation and use of forecasts.

Following Green, Armstrong and Soon’s (IJF 2009) (GAS) naïve extrapolation, Fildes and Kourentzes (IJF 2011) (F&K) found that each of six more-sophisticated, but inexpensive, extrapolation models provided forecasts of global mean temperature for the 20 years to 2007 that were more accurate than the “business as usual” projections provided by the co...

This article reviews experimental evidence on the effects of policies intended to promote behavior by firms that is more socially responsible and less socially irresponsible. Corporate social responsibility (CSR) can provide firms with opportunities for profit, but changes are likely to increase total welfare only if firms adopt them freely and wit...

Should public policy guarantee First Amendment rights to all citizens at all times? Some of our commentators think not. We argue that there is good reason to hold to rights. Consistent with economic theory and behavioral research, experimental findings show that mandatory disclaimers harm producers and consumers. They are also expensive to develop...

Chart of Hadley "global" average temperature anomalies in degrees Celcius (Centigrade) to ~2012, showing the half-century periods used in the associated Mystery Temperature Chart forecasting quiz, plus 25-year LSQ trend line for first forecast period

Quiz answer scoring regime, and results from attendees at the Green, Soon, and Armstrong seminar at the International Symposium on Forecasting in Boston, 2012.

We found no evidence that consumers benefit from government-mandated disclaimers in advertising. Experiments and common experience show that admonishments to change or avoid behaviors often have effects opposite to those intended. We found 18 experimental studies that provided evidence relevant to mandatory disclaimers. Mandated messages increased...

An embryonic research project with the objective of developing evidence-based forecasting models of the effect of policy settings on long-term per-capita growth rates. growthofnations.com directs here.

Our review of five experiments on the effects of mandatory disclaimers led us to conclude that (1) the information they provide is not important to consumers and (2) it is poorly understood by them. We conducted an experiment to assess the effects on decision making of including a mandatory disclaimer in a print advertisement for a dentist offering...

Online Supplement to: Green, K. C., & Armstrong, J. Scott. “Evidence on the Effects of Mandatory Disclaimers in Advertising” in Journal of Public Policy & Marketing, Autumn 2012

The validity of the manmade global warming alarm requires the support of scientific forecasts of (1) a substantive long-term rise in global mean temperatures in the absence of regulations, (2) serious net harmful effects due to global warming, and (3) cost-effective regulations that would produce net beneficial effects versus alternatives policies,...

Forecasting procedures are needed only if there is uncertainty about the future. Forecasts are unnecessary when one can control events. For example, predicting the temperature in your home does not require the use of forecasting procedures because you can control it. A forecast that the sun will rise tomorrow is of little value. Many decisions, how...

Forecasting is concerned with making statements about the as yet unknown. There are many ways that people go about deriving forecasts. This entry is concerned primarily with procedures that have performed well in empirical studies that contrast the accuracy of alternative methods.Evidence about forecasting procedures has been codified as condition-...

When forecasting decisions in conflict situations, experts are often advised to figuratively stand in the other person's shoes. We refer to this as "role thinking", because, in practice, the advice is to think about how other protagonists will view the situation in order to predict their decisions. We tested the effect of role thinking on forecast...

When the beach patrol raises the alarm that a shark has been sighted we know what to do, but how should we respond to an alarm that is based on predictions of what will happen 100 years from now and the person raising the alarm tells us we must make great sacrifices now to avoid the predicted catastrophe? To answer this question, we forecast effect...

The validity of the manmade global warming alarm requires the support of scientific forecasts of (1) a substantive long-term rise in global mean temperatures in the absence of regulations, (2) serious net harmful effects due to global warming, and (3) cost-effective regulations that would produce net beneficial effects versus alternatives policies,...