J. Scott Armstrong

J. Scott Armstrong
University of Pennsylvania | UP · The Wharton School

PnD. MIT Management

About

555
Publications
347,789
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
35,717
Citations
Additional affiliations
July 2011 - August 2011
University of South Australia
Position
  • Visiting Pofessor
March 1997 - April 1997
University of Auckland
Position
  • Visiting Professor of Marketing
June 1968 - present
University of Pennsylvania
Position
  • Professor (Full)
Education
September 1965 - May 1968

Publications

Publications (555)
Data
The following books and articles were discussed in the second edition of Long-Range Forecasting, published in 1985. In the few cases where unpublished sources are cited, instructions are provided on how to obtain a copy of the book or article. An attempt was also made to choose the most readable source when multiple sources were available. The list...
Data
We asked 52 university students to make predictions about conflict situations. We gave each student four conflicts from a set of eight. We asked half our participants to assume there were 100 situations similar to a target conflict. We then asked them “…in how many of these situations would…” each one of a list of possible outcomes occur? The full...
Data
Rules for choosing a single-decision forecast from a set of up to five analogies that have been rated for similarity to a target conflict
Data
From “Bad Environmental and Resource Scares”, Chapter 18 of Julian L Simon’s 1996 “ The Ultimate Resource 2”, Princeton University Press, pp 258-273, and “Healing the Planet” afternote on p 274. Reproduced with the kind permission of the Simon family.
Data
Electronic (Excel) version of the Checklist for Writing Management Reports
Research
Full-text available
Proposed remedies for harmful regulation
Experiment Findings
Full-text available
The Global Warming Challenge, aka The Climate Bet (theclimatebet.com), is a record of an experiment to test the relative accuracy of Al Gore's 2007 dangerous man made global warming "tipping point" alarming projection relative to the no-change (no-trend) forecasts of Green, Armstrong, and Soon (2009) proposed by Scott Armstrong in his challenge to...
Data
Flyer for Armstrong & Green presentation at ICCC Ten Washington, D.C., Including the Golden Rule of Forecasting Checklist and abstracts of papers with evidence on "Long-range climate and public policy forecasting"
Method
Full-text available
Ten checklists for Armstrong & Green's "The Scientific Method" book in the form of pdf forms. For more information on "The Scientific Method", see thescientificmethod.info.
Presentation
Full-text available
"Do we face dangerous global warming?" Scott Armstrong gave a talk at Lehigh University on June 7, 2019 about research with Kesten Green on scientific climate forecasting. The talk was invited by the graduating Class of 1959, of which he is a member, for their 60th Reunion.
Preprint
Full-text available
Purpose: Commentary on M4-Competition and findings to assess the contribution of data models--such as from machine learning methods--to improving forecast accuracy. Methods: (1) Use prior knowledge on the relative accuracy of forecasts from validated forecasting methods to assess the M4 findings. (2) Use prior knowledge on forecasting principles...
Article
Purpose: Commentary on M4-Competition and findings to assess the contribution of data models—such as from machine learning methods—to improving forecast accuracy. Methods: (1) Use prior knowledge on the relative accuracy of forecasts from validated forecasting methods to assess the M4 findings. (2) Use prior knowledge on forecasting principles and...
Preprint
Full-text available
In the mid-1900s, there were two streams of thought about forecasting methods. One stream-led by econometricians-was concerned with developing causal models by using prior knowledge and evidence from experiments. The other was led by statisticians, who were concerned with identifying idealized "data generating processes" and with developing models...
Article
Full-text available
Problem Do conservative econometric models that comply with the Golden Rule of Forecasting pro- vide more accurate forecasts? Methods To test the effects of forecast accuracy, we applied three evidence-based guidelines to 19 published regression models used for forecasting 154 elections in Australia, Canada, Italy, Japan, Netherlands, Portugal, Sp...
Data
Relative absolute error (RAE) of forecasts from damping compared to forecasts from the original regression models. (DOCX)
Data
Relative absolute error (RAE) of forecasts from equalizing compared to forecasts from the original regression models. (DOCX)
Conference Paper
Full-text available
Commentary of the findings of the M4-Competition presented by Scott Armstrong at the M4 Conference on Monday, December 10, 2018 in New York City.
Conference Paper
Full-text available
Problem: Multiple regression analysis (MRA) is commonly used to develop forecasting models that inform policy and decision making, but the technique does not appear to have been validated for that purpose. Methods: The predictive validity of published least squares MRA models is tested against naive benchmarks, alternative methods that are either p...
Conference Paper
Full-text available
We examined two papers with long-range forecasts of global mean temperatures (IPCC 2007, and Green, Armstrong, & Soon 2009) that included forecasts of global mean temperatures, and one paper that used the IPCC projections to forecast a dramatic decline in the population of polar bears for compliance with the scientific method. Ratings for complianc...
Article
Full-text available
Problem: The scientific method is unrivaled for generating useful knowledge, yet papers published in scientific journals frequently violate the scientific method. Methods: A definition of the scientific method was developed from the writings of pioneers of the scientific method including Aristotle, Newton, and Franklin. The definition was used as...
Article
Full-text available
Problem How to help practitioners, academics, and decision makers use experimental research findings to substantially reduce forecast errors for all types of forecasting problems. Methods Findings from our review of forecasting experiments were used to identify methods and principles that lead to accurate forecasts. Cited authors were contacted to...
Research
Full-text available
Decision making is improved by avoiding unaided expert judgment, and using structured judgmental procedures instead. The Red Team-Blue Team approach is one such structured approach. It recognizes that it is difficult for people-including scientists and public officials-to remain objective about the consequences of public policies and regulations. T...
Preprint
Full-text available
Super Bowl advertisers ignore many persuasion principles, to their cost.
Working Paper
Full-text available
Problem: How to help practitioners, academics, and decision makers use experimental research findings to substantially reduce forecast errors for all types of forecasting problems. Methods: Findings from our review of forecasting experiments were used to identify methods and principles that lead to accurate forecasts. Cited authors were contacted...
Chapter
Full-text available
The PollyVote uses evidence-based techniques for forecasting the popular vote in presidential elections. The forecasts are derived by averaging existing forecasts generated by six different forecasting methods. In 2016, the PollyVote correctly predicted that Hillary Clinton would win the popular vote. The 1.9 percentage-point error across the last...
Method
Full-text available
Earlier versions of this Tree appear in various publications and presentations. For more information on the methods and how to implement them, see "Forecasting Methods and Principles: Evidence-based Checklists."
Article
Full-text available
A Recap of the 2016 Election Forecasts - Volume 50 Issue 2 - James E. Campbell, Helmut Norpoth, Alan I. Abramowitz, Michael S. Lewis-Beck, Charles Tien, James E. Campbell, Robert S. Erikson, Christopher Wlezien, Brad Lockerbie, Thomas M. Holbrook, Bruno Jerôme, Véronique Jerôme-Speziari, Andreas Graefe, J. Scott Armstrong, Randall J. Jones, Alfred...
Article
Full-text available
The PollyVote Forecast for the 2016 American Presidential Election - Volume 49 Issue 4 - Andreas Graefe, Randall J. Jones, J. Scott Armstrong, Alfred G. Cuzán
Code
Checklist for multiple raters to rate a proposal for an advertisement against persuasion principles.
Article
Full-text available
Purpose: To respond to issues posed in the four commentaries on Armstrong, Du, Green and Graefe (this issue) regarding the immediate usefulness of that paper’s test of advertisements’ compliance with persuasion principles, and regarding the need for further research. Approach: Address commentators’ concerns using logic, prior research findings, an...
Article
Purpose – This paper aims to test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine. Design/methodology/approach – Scores were calculated from the ratings o...
Code
Excel spreadsheet checklists for comparing written and oral reports to evidence-based persuasion principles supported by Armstrong's Persuasive Advertising book, or logic.
Article
Full-text available
Purpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine. Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96...
Article
Full-text available
This article proposes a unifying theory, or the Golden Rule, of forecasting. The Golden Rule of Forecasting is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek out and use all knowledge relevant to the problem, including knowledge of methods...
Article
Full-text available
The Golden Rule of Forecasting is a general rule that applies to all forecasting problems. The Rule was developed using logic and was tested against evidence from previously published comparison studies. The evidence suggests that a single violation of the Golden Rule is likely to increase forecast error by 44%. Some commentators argue that the Rul...
Article
Full-text available
The Golden Rule of Forecasting is a general rule that applies to all forecasting problems. The Rule was developed using logic and was tested against evidence from previously published comparison studies. The evidence suggests that a single violation of the Golden Rule is likely to increase forecast error by 44%. Some commentators argue that the Rul...
Article
Full-text available
This article proposes a unifying theory, or Golden Rule, of forecasting. The Golden Rule of Forecasting is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek out and use all knowledge relevant to the problem, including knowledge of methods val...
Method
Full-text available
A checklist of requirements for following The Golden Rule of Forecasting. From Armstrong, Green, and Graefe (2015).
Presentation
Full-text available
Conference Paper
Full-text available
The Golden Rule of Forecasting requires forecasters to be conservative by forecasting in a way that is consistent with cumulative knowledge about the situation and about forecasting. Violating the average guideline increases forecast errors by more than 40%. We examined whether the United Nations Intergovernmental Panel on Climate Change’s (IPCC’s)...
Conference Paper
Full-text available
With the objective of improving the accuracy of election forecasts, we examined three evidence-based forecasting guidelines that are relevant to forecasting with causal models. The guidelines suggest: (1) modifying estimates of the strength of variable effects to account for uncertainty, (2) combining forecasts from diverse models, and (3) utilizin...
Conference Paper
Full-text available
The United Nations Intergovernmental Panel on Climate Change’s (IPCC’s) forecast of dangerous manmade global warming is an important influence on public policy decisions in many countries. Rational policies require that policy makers obtain forecasts from evidence-based methods. A review of comparative studies found that sophisticatedly simple meth...
Presentation
Full-text available
The Golden Rule of Forecasting Checklist was used to evaluate IPCC “business as usual” global warming scenario and the Green, Armstrong & Soon no-change model forecasts. Consensus ratings by Armstrong and Green indicated that of 20 relevant Golden Rule Checklist guidelines: • the IPCC scenarios followed none • the no-change model followed 95%
Article
Full-text available
This article introduces this JBR Special Issue on simple versus complex methods in forecasting. Simplicity in forecasting requires that (1) method, (2) representation of cumulative knowledge, (3) relationships in models, and (4) relationships among models, forecasts, and decisions are all sufficiently uncomplicated as to be easily understood by dec...
Article
Full-text available
This article examines whether decomposing time series data into two parts - level and change - produces forecasts that are more accurate than those from forecasting the aggregate directly. Prior research found that, in general, decomposition reduced forecasting errors by 35%. An earlier study on decomposition into level and change found a forecast...
Code
Spreadsheet checklist of persuasion principles for creating advertisements - The Persuasion Principles Audit software. Supporting document: Persuasion Principles Audit Report Sample Format. Supporting document: Persuasion Principles Checklist Instructions
Presentation
Full-text available
Article
Full-text available
Purpose: To test whether a structured application of persuasion principles might help improve advertising decisions. Evidence-based principles are currently used to improve decisions in other complex situations, such as those faced in engineering and medicine.Approach: Scores were calculated from the ratings of 17 self-trained novices who rated 96...
Article
Full-text available
Purpose: To respond to issues posed in the four commentaries on Armstrong, Du, Green and Graefe (this issue) regarding the immediate usefulness of that paper’s test of advertisements’ compliance with persuasion principles, and regarding the need for further research. Approach: Address commentators’ concerns using logic, prior research findings, and...
Method
Full-text available
Checklist for developing and using a time line for planning
Method
Full-text available
Code
The interactive training module for the Advertising Principles Audit
Method
Full-text available
Persuasive Advertising (Armstrong, 2010) focuses on how to develop persuasive ads, but not on how to deliver those ads to customers. Here are five methods that can be used to decide a media budget as well as how to allocate that spending across various media. I start with the weakest method and work towards the strongest. Ideally, one should use al...
Research
Full-text available
The evaluation of advertising agencies and proposals rests heavily of judgments. Fortunately, much is known about how to improve judgments.
Presentation
Full-text available
Earlier versions of this Tree appear in various publications and presentations. Note that the presence of a method in the Tree does not signal its predictive validity. For evidence on which methods are valid, see "Forecasting Methods and Principles: Evidence-based Checklists."
Presentation
Full-text available
Conference Paper
Full-text available
The Golden Rule of Forecasting counsels forecasters to be conservative when making forecasts. We tested the value of three of the four Golden Rule guidelines that apply to causal models: modify effect estimates to reflect uncerainty; use all important variables; and combine forecasts from diverse models. These guidelines were tested using out-of-sa...
Article
Full-text available
We review the performance of the PollyVote, which combined forecasts from polls, prediction markets, experts’ judgment, political economy models, and index models to forecast the two-party popular vote in the 2012 U.S. Presidential Election. Throughout the election year the PollyVote provided highly accurate forecasts, outperforming each of its com...
Article
Full-text available
We summarize the literature on the effectiveness of combining forecasts by assessing the conditions under which combining is most valuable. Using data on the six US presidential elections from 1992 to 2012, we report the reductions in error obtained by averaging forecasts within and across four election forecasting methods: poll projections, expert...
Data
A letter to Mr Wanamaker about the findings of Armstrong, Du, Green, & Graefe (2015).
Article
Full-text available
The Big-Issue Model predicts election outcomes based on voters’ perceptions of candidates’ ability to handle the most important issue. The model provided accurate forecasts of the 2012 U.S. presidential election. The results demonstrate the usefulness of the model in situations where one issue clearly dominates the campaign, such as the state of th...
Article
Full-text available
The Golden Rule of Forecasting requires that forecasters be conservative by making proper use of cumulative knowledge and by not going beyond that knowledge. The procedures that have been used to forecast dangerous manmade global warming violate the Golden Rule. Following the scientific method, we investigated competing hypotheses on climate change...
Article
Full-text available
Our review of the evidence found that the notion that higher pay leads to the selection of better executives is undermined by the prevalence of poor recruiting methods. Moreover, higher pay fails to promote better performance. Instead, it undermines the intrinsic motivation of executives, inhibits their learning, leads them to ignore other stakehol...
Data
We summarize the literature on the effectiveness of combining forecasts by assessing the conditions under which combining is most valuable. Using data on the six US presidential elections from 1992 to 2012, we report the reductions in error obtained by averaging forecasts within and across four election forecasting methods: poll projections, expert...
Data
The Big-Issue Model predicts election outcomes based on voters’ perceptions of candidates’ ability to handle the most important issue. The model provided accurate forecasts of the 2012 U.S. presidential election. The results demonstrate the usefulness of the model in situations where one issue clearly dominates the campaign, such as the state of th...
Data
Test your forecasting skills: Print this page and draw in your forecasts The charts show monthly global mean temperatures over two half centuries (both during the industrial era). The task is to draw in your forecasts for the next 25 years for both charts.
Article
Full-text available
This article reviews experimental evidence on the effects of policies intended to promote behavior by firms that is more socially responsible and less socially irresponsible. Corporate social responsibility (CSR) can provide firms with opportunities for profit, but changes are likely to increase total welfare only if firms adopt them freely and wit...
Chapter
Full-text available
The field of forecasting is concerned with making statements about matters that are currently unknown. The terms "forecast," "prediction," "projections," and "prognosis" are interchangeable as commonly used. Forecasting is also concerned with the effective presentation and use of forecasts.
Article
In this brief commentary on the paper Designing Research with In-Built Differentiated Replication, we expand on concerns about a lack of replication research raised by the authors by focusing on three key questions of continuous importance: Why should more replication research be conducted? Why do we find so few replications studies? What can be do...
Article
Full-text available
When deciding for whom to vote, voters should select the candidate they expect to best handle issues, all other things equal. A simple heuristic predicted that the candidate who is rated more favorably on a larger number of issues would win the popular vote. This was correct for nine out of ten U.S. presidential elections from 1972 to 2008. We then...
Article
Full-text available
We conducted laboratory experiments for analyzing the accuracy of three structured approaches (nominal groups, Delphi, and prediction markets) relative to traditional face-to-face meetings (FTF). We recruited 227 participants (11 groups per method) who were required to solve a quantitative judgment task that did not involve distributed knowledge. T...
Method
Full-text available
A checklist of evidence-based standards for forecasting. The checklist is a version of one that appeared in Armstrong's (2001) "Principles of Forecasting: A Handbook for Researchers and Practitioners"
Article
Full-text available
Philippe Jacquart, EMLYON Business School a (jacquart@em-lyon.com) February 2013 (v19) The commentators take divergent positions on our conclusion that current executive pay practices harm businesses' performance. Commentators Deci and Ryan (2013), as well as Hogarth and Kolev (2013), are largely sympathetic to our conclusions, and Jones (2013) pro...
Article
Full-text available
Following Green, Armstrong and Soon’s (IJF 2009) (GAS) naïve extrapolation, Fildes and Kourentzes (IJF 2011) (F&K) found that each of six more-sophisticated, but inexpensive, extrapolation models provided forecasts of global mean temperature for the 20 years to 2007 that were more accurate than the “business as usual” projections provided by the co...
Article
Full-text available
The present study reviews the accuracy of four methods for forecasting the 2013 German election: polls, prediction markets, expert judgment, and quantitative models. On average, across the two months prior to the election, polls were most accurate, with a mean absolute error of 1.4 percentage points, followed by quantitative models (1.6), expert ju...
Article
Full-text available
The experimental evidence in this collection of papers is sufficient for organizations to take action—at least with respect to investigating or testing alternative pay schemes. Some organizations have already implemented a number of these procedures. The failure of an organization’s directors to follow evidence-based procedures for executive pay mi...

Questions

Question (1)
Question
In our current paper, the "Golden Rule of Forecasting" we are searching for evidence on whether, over the past half-century, the practice of forecasting in any area of the social or management sciences has been shown to produce more accurate forecasts or less accurate forecasts or if there has been no measurable change. Please provide citations to relevant sources.

Projects

Projects (12)
Project
To compile evidence on the predictive validity of The Golden Rule of Forecasting. The Golden Rule of Forecasting is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek out and use all knowledge relevant to the problem, including knowledge of methods validated for the situation. Twenty-eight guidelines are logically deduced from the Golden Rule. A review of evidence identified 105 papers with experimental comparisons; 102 support the guidelines. Ignoring a single guideline increased forecast error by more than two-fifths on average. Ignoring the Golden Rule is likely to harm accuracy most when the situation is uncertain and complex, and when bias is likely. Non-experts who use the Golden Rule can identify dubious forecasts quickly and inexpensively. To date, ignorance of research findings, bias, sophisticated statistical procedures, and the proliferation of big data, have led forecasters to violate the Golden Rule. As a result, despite major advances in evidence-based forecasting methods, forecasting practice in many fields has failed to improve over the past half-century.
Project
To assess the predictive validity and usefulness of prediction markets under different conditions.
Project
To develop and compare alternative causal models for forecasting the long-term economic growth rates of nations. After some initial research, this project is currently on hold due to other commitments.