No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
References
Abstract
List of works referred to in Armstrong & Green (2022) The Scientific Method
ResearchGate has not been able to resolve any citations for this publication.
This is a book about forecasting methods. The emphasis is on methods for long-range forecasting. The methods are applicable to all areas of the social, behavioral, and management sciences. Much is known about forecasting methods, but little is used. Why? Because what is known in one field is unknown in another. Because what is known frequently contradicts our common sense. Because what is known challenges our beliefs and our behavior. Long-Range Forecasting is a book for "doers," the people who have done or who are doing forecasting. These doers may be in business, in government, in academia, or in consulting. Doers may also be students working on forecasting projects in courses such as finance, marketing, economics, or sociology, or in those increasingly popular courses dealing with the future. Some of the doers have little expertise, and some have much. If you have a lot of expertise, you will find many things to disagree with in this book. That is how it should be; if there is nothing in a book that challenges your beliefs, then there is no opportunity to learn. Of course, you will also find things to make you feel good about your current beliefs, but you will not learn much from these things. The way to learn is to find ideas that you disagree with and then to suspend judgment and experiment with these ideas. Long-Range Forecasting is a guide to forecasting methods. Although it is designed to be read, it can also be used as a reference book.
The precautionary principle is a political principle, not a scientific one. The principle is used to urge the cessation or avoidance of a human activity in situations of uncertainty, just in case that activity might cause harm to human health or the natural environment. In practice, the precautionary principle is invoked when an interest group identifies an issue that can help it to achieve its objectives. If the interest group is successful in its efforts to raise fears about the issue, the application of the scientific method is rejected and a new orthodoxy is imposed.
Background:
The ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines are widely endorsed but compliance is limited. We sought to determine whether journal-requested completion of an ARRIVE checklist improves full compliance with the guidelines.
Methods:
In a randomised controlled trial, manuscripts reporting in vivo animal research submitted to PLOS ONE (March-June 2015) were randomly allocated to either requested completion of an ARRIVE checklist or current standard practice. Authors, academic editors, and peer reviewers were blinded to group allocation. Trained reviewers performed outcome adjudication in duplicate by assessing manuscripts against an operationalised version of the ARRIVE guidelines that consists 108 items. Our primary outcome was the between-group differences in the proportion of manuscripts meeting all ARRIVE guideline checklist subitems.
Results:
We randomised 1689 manuscripts (control: n = 844, intervention: n = 845), of which 1269 were sent for peer review and 762 (control: n = 340; intervention: n = 332) accepted for publication. No manuscript in either group achieved full compliance with the ARRIVE checklist. Details of animal husbandry (ARRIVE subitem 9b) was the only subitem to show improvements in reporting, with the proportion of compliant manuscripts rising from 52.1 to 74.1% (X2 = 34.0, df = 1, p = 2.1 × 10-7) in the control and intervention groups, respectively.
Conclusions:
These results suggest that altering the editorial process to include requests for a completed ARRIVE checklist is not enough to improve compliance with the ARRIVE guidelines. Other approaches, such as more stringent editorial policies or a targeted approach on key quality items, may promote improvements in reporting.
Background
The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades. Moreover, recent studies indicate that these metrics have become targets and follow Goodhart’s Law, according to which, “when a measure becomes a target, it ceases to be a good measure.”
Results
In this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening. In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists. Measures such as a journal’s impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors. Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department.
Conclusions
Academic publishing has changed considerably; now we need to reconsider how we measure success.
One of the most universal trends in science and technology today is the growth of large teams in all areas, as solitary researchers and small teams diminish in prevalence1–3. Increases in team size have been attributed to the specialization of scientific activities³, improvements in communication technology4,5, or the complexity of modern problems that require interdisciplinary solutions6–8. This shift in team size raises the question of whether and how the character of the science and technology produced by large teams differs from that of small teams. Here we analyse more than 65 million papers, patents and software products that span the period 1954–2014, and demonstrate that across this period smaller teams have tended to disrupt science and technology with new ideas and opportunities, whereas larger teams have tended to develop existing ones. Work from larger teams builds on more-recent and popular developments, and attention to their work comes immediately. By contrast, contributions by smaller teams search more deeply into the past, are viewed as disruptive to science and technology and succeed further into the future—if at all. Observed differences between small and large teams are magnified for higher-impact work, with small teams known for disruptive work and large teams for developing work. Differences in topic and research design account for a small part of the relationship between team size and disruption; most of the effect occurs at the level of the individual, as people move between smaller and larger teams. These results demonstrate that both small and large teams are essential to a flourishing ecology of science and technology, and suggest that, to achieve this, science policies should aim to support a diversity of team sizes.
***** OPEN ACCESS at: https://doi.org/10.1016/j.edurev.2018.09.003
****************
With the increasing dominance of digital reading over paper reading, gaining understanding of the effects of the medium on reading comprehension has become critical. However, results from research comparing learning outcomes across printed and digital media are mixed, making conclusions difficult to reach. In the current meta-analysis, we examined research in recent years (2000–2017), comparing the reading of comparable texts on paper and on digital devices. We included studies with between-participants (n = 38) and within-participants designs (n = 16) involving 171,055 participants. Both designs yielded the same advantage of paper over digital reading (Hedge's g = −0.21; dc = −0.21). Analyses revealed three significant moderators: (1) time frame: the paper-based reading advantage increased in time-constrained reading compared to self-paced reading; (2) text genre: the paper-based reading advantage was consistent across studies using informational texts, or a mix of informational and narrative texts, but not on those using only narrative texts; (3) publication year: the advantage of paper-based reading increased over the years. Theoretical and educational implications are discussed.
In the mid-1900s, there were two streams of thought about forecasting methods. One stream-led by econometricians-was concerned with developing causal models by using prior knowledge and evidence from experiments. The other was led by statisticians, who were concerned with identifying idealized "data generating processes" and with developing models from statistical relationships in data, both in the expectation that the resulting models would provide accurate forecasts. At that time, regression analysis was a costly process. In more recent times, regression analysis and related techniques have become simple and inexpensive to use. That development led to automated procedures such as stepwise regression, which selects "predictor variables" on the basis of statistical significance. An early response to the development was titled, "Alchemy in the behavioral sciences" (Einhorn, 1972). We refer to the product of data-driven approaches to forecasting as "data models." The M4-Competition (Makridakis, Spiliotis, Assimakopoulos, 2018) has provided extensive tests of whether data models-which they refer to as "ML methods"-can provide accurate extrapolation forecasts of time series. The Competition findings revealed that data models failed to beat naïve models, and established simple methods, with sufficient reliability to be of any practical interest to forecasters. In particular, the authors concluded from their analysis, "The six pure ML methods that were submitted in the M4 all performed poorly, with none of them being more accurate than Comb and only one being more accurate than Naïve2" (p. 803.) Over the past half-century, much has been learned about how to improve forecasting by conducting experiments to compare the performance of reasonable alternative methods. On the other hand, despite billions of dollars of expenditure, the various data modeling methods have not contributed to improving forecast accuracy. Nor can they do so, as we explain below.
We introduce several new variants of the dice experiment by Fischbacher and Föllmi-Heusi (Journal of the European Economic Association 11(3):525–547, 2013) to investigate measures to reduce lying. Hypotheses on the relative performance of these treatments are derived from a straightforward theoretical model. In line with previous research, we find that groups of two subjects lied at least to the same extent as individuals—even in a novel treatment where we assigned to one subject the role of being the other’s monitor. However, we find that our participants hardly lied if they do not benefit and only others do, even if they were in a reciprocal relationship. Thus, we conclude that collaboration on lying mostly happens for personal gain. To mitigate selfish lying, an honesty oath which aims to increase moral awareness turned out to be effective.
The goal of this paper is to provide a long overdue clarification and upgrade to what has been called the intrinsic-extrinsic dichotomy in the realm of motivation. We argue that the concept of intrinsic motivation should be limited to refer to the pleasure gained from an activity, divorced from any further elements. It means liking the doing. The term has been confounded with a different type of motivation, which is properly labeled achievement motivation and which refers to competition against some standard of excellence (subconscious or conscious). Achievement motivation means wanting to do well. One can like doing something and not care about how well
Problem
How to help practitioners, academics, and decision makers use experimental research findings to substantially reduce forecast errors for all types of forecasting problems.
Methods
Findings from our review of forecasting experiments were used to identify methods and principles that lead to accurate forecasts. Cited authors were contacted to verify that summaries of their research were correct. Checklists to help forecasters and their clients undertake and commission studies that adhere to principles and use valid methods were developed. Leading researchers were asked to identify errors of omission or commission in the analyses and summaries of research findings.
Findings
Forecast accuracy can be improved by using one of 15 relatively simple evidence-based forecasting methods. One of those methods, knowledge models, provides substantial improvements in accuracy when causal knowledge is good. On the other hand, data models – developed using multiple regression, data mining, neural nets, and “big data analytics” – are unsuited for forecasting.
Originality
Three new checklists for choosing validated methods, developing knowledge models, and assessing uncertainty are presented. A fourth checklist, based on the Golden Rule of Forecasting, was improved.
Usefulness
Combining forecasts within individual methods and across different methods can reduce forecast errors by as much as 50%. Forecasts errors from currently used methods can be reduced by increasing their compliance with the principles of conservatism (Golden Rule of Forecasting) and simplicity (Occam’s Razor). Clients and other interested parties can use the checklists to determine whether forecasts were derived using evidence-based procedures and can, therefore, be trusted for making decisions. Scientists can use the checklists to devise tests of the predictive validity of their findings.
This article explores an important credibility problem in the research literature beyond the issue of questionable data analysis methods: the problem of omission of relevant previous research in published research articles. This article focuses on this problem in 2 areas: (a) studies purporting to demonstrate the effects of people’s experiences on their later life outcomes while failing to discuss or mention the probable causal role of genetic inheritance in producing these effects, despite the strong evidence for this connection from behavior genetics research; and (b) studies of specific aptitudes (specific abilities) such as verbal, spatial, or reasoning that fail to acknowledge or mention that such aptitudes are indicator variables for general mental ability (GMA; or intelligence) and that after proper control for GMA the residuals in these aptitudes make essentially no contribution to prediction of real world academic, occupational, or job performance. It is only the GMA component in such aptitudes that produces the ability to predict. As is well known today, the issue of the credibility of research conclusions is prominent (Ioannidis, 2005). In both the areas examined in this article, these deficiencies create serious and unnecessary credibility problems, and the doubts they inspire about credibility could unfortunately be generalized to other research areas in which these problems do not exist.
Clarity and accuracy of reporting are fundamental to the scientific process. Readability formulas can estimate how difficult a text is to read. Here, in a corpus consisting of 709,577 abstracts published between 1881 and 2015 from 123 scientific journals, we show that the readability of science is steadily decreasing. Our analyses show that this trend is indicative of a growing use of general scientific jargon. These results are concerning for scientists and for the wider public, as they impact both the reproducibility and accessibility of research findings.
In the wake of large-scale retraction scandals, we urge scientific publishers to be more proactive in stamping out fake peer-reviewing practices. They should work with editors, authors and research institutes to implement an effective system of precautions and penalties.
Fraudulent peer review can arise when editors rely on authors' recommended reviewers. These names are often genuine but have a false e-mail address that enables the authors to write a favourable review of their own paper.
Springer Nature, also the publisher of Nature, this year retracted 107 papers from one of its journals on the basis of fake peer review (see T. Stigbrand Tumor Biol. doi.org/b7gg; 2017). Two years ago, it retracted 64 articles in 10 of its journals on similar grounds (see Nature doi.org/b7gh; 2015).
In our view, retracting such papers is not enough. Editors need to double-check the authenticity of potential reviewers and insist that authors provide academic identities for their suggested reviewers, including institutional e-mail addresses, ORCID identifiers and Scopus Author IDs.
Journals should confidentially share each other's databases of falsified reviewers' details and of offending authors. Publishers could then reject new submissions from those authors for a set period. National academic committees and research institutes might consider revoking research funding for such authors and demoting them.
We study the mental health of graduate students at eight top-ranked economics PhD programs in the United States using clinically validated surveys. We find that 24.8 percent experience moderate or severe symptoms of depression or anxiety—more than two times the population average. Though our response rate was 45.1 percent and sample selection concerns exist, conservative lower bounds nonetheless suggest higher prevalence rates of such symptoms than in the general population. Mental health issues are especially prevalent at the end of the PhD program: 36.7 percent of students in years 6+ of their program experience moderate or severe symptoms of depression or anxiety, versus 21.2 percent of first-year students. Of economics students with these symptoms, 25.2 percent are in treatment, compared to 41.4 percent of graduate students in other programs. A similar percentage of economics students (40–50 percent) say they cannot honestly discuss mental health with advisers as say they cannot easily discuss nonacademic career options with them. Only 26 percent find their work to be useful always or most of the time, compared to 70 percent of economics faculty and 63 percent of the working age population. We provide recommendations for students, faculty, and administrators on ways to improve graduate student mental health. (JEL A23, I12, I18, I23)
Are articles by authors with institutional connections or personal ties to the publishing journal's editor(s)/coeditors of lower quality than those authored without such connections? Examination of articles published in six core economics journals in 1990 found that articles authored by those with such connections, especially service on the publishing journal's editorial board, are statistically and numerically of higher quality than articles by those without such connections. In addition, this quality difference does not decrease over time.
Described by the philosopher A.J. Ayer as a work of ‘great originality and power’, this book revolutionized contemporary thinking on science and knowledge. Ideas such as the now legendary doctrine of ‘falsificationism’ electrified the scientific community, influencing even working scientists, as well as post-war philosophy. This astonishing work ranks alongside The Open Society and Its Enemies as one of Popper’s most enduring books and contains insights and arguments that demand to be read to this day. © 1959, 1968, 1972, 1980 Karl Popper and 1999, 2002 The Estate of Karl Popper. All rights reserved.
A leading economist discusses the potential of happiness research (the quantification of well-being) to answer important questions that standard economics methods are unable to analyze.
Revolutionary developments in economics are rare. The conservative bias of the field and its enshrined knowledge make it difficult to introduce new ideas not in line with received theory. Happiness research, however, has the potential to change economics substantially in the future. Its findings, which are gradually being taken into account in standard economics, can be considered revolutionary in three respects: the measurement of experienced utility using psychologists' tools for measuring subjective well-being; new insights into how human beings value goods and services and social conditions that include consideration of such non-material values as autonomy and social relations; and policy consequences of these new insights that suggest different ways for government to affect individual well-being. In Happiness, emphasizing empirical evidence rather than theoretical conjectures, Bruno Frey substantiates these three revolutionary claims for happiness research. After tracing the major developments of happiness research in economics and demonstrating that we have gained important new insights into how income, unemployment, inflation, and income demonstration affect well-being, Frey examines such wide-ranging topics as democracy and federalism, self-employment and volunteer work, marriage, terrorism, and watching television from the new perspective of happiness research. Turning to policy implications, Frey describes how government can provide the conditions for people to achieve well-being, arguing that a crucial role is played by adequate political institutions and decentralized decision making. Happiness demonstrates the achievements of the economic happiness revolution and points the way to future research.
https://plato.stanford.edu/archives/win2018/entries/scientific-reproducibility
Private information is at the heart of many economic activities. For decades, economists have assumed that individuals are willing to misreport private information if this maximizes their material payoff. We combine data from 90 experimental studies in economics, psychology, and sociology, and show that, in fact, people lie surprisingly little. We then formalize a wide range of potential explanations for the observed behavior, identify testable predictions that can distinguish between the models, and conduct new experiments to do so. Our empirical evidence suggests that a preference for being seen as honest and a preference for being honest are the main motivations for truth‐telling.
How do the findings of road safety research affect the practice by which the road infrastructure is built and operated? The question is seldom asked. I discuss the complexities of the research-practice symbiosis in the light of two historical anecdotes. These allow me to point out several issues of concern.
My general conclusion is that the relationship, as it evolved over time, is unpremeditated and occasionally dysfunctional. Issues of concern are the lightness with which decisions affecting road-user safety can be based on opinion that is unsupported by evidence, that such opinions can trump inconvenient evidence, that research findings can be willfully distorted or disregarded, that questionable results can be given a ring of consensual truth, and that the questions which research asks and what findings get published are at times influenced by external interest. In sum, the concern is that practice is not sufficiently evidence-based. Road users have a right to expect that decisions substantially affecting their safety take into account fact-based expectation of safety consequences. It is therefore time to endow the research-practice relationship with a premeditated and purposeful structure.
Background
Given the increasing popularity of reading from screens, it is not surprising that numerous studies have been conducted comparing reading from paper and electronic sources. The purpose of this systematic review and meta‐analysis is to consolidate the findings on reading performance, reading times and calibration of performance (metacognition) between reading text from paper compared to screens.
Methods
A systematic literature search of reports of studies comparing reading from paper and screens was conducted in seven databases. Additional studies were identified by contacting researchers who have published on the topic, by a backwards search of the references of found reports and by a snowball search of reports citing what was initially found. Only studies that were experiments with random assignment and with participants who had fundamental reading skills and disseminated between 2008 and 2018 were included. Twenty‐nine reports with 33 identified studies met inclusion criteria experimentally comparing reading performance ( k = 33; n = 2,799), reading time ( k = 14; n = 1,233) and/or calibration ( k = 11; n = 698) from paper and screens.
Results
Based on random effects models, reading from screens had a negative effect on reading performance relative to paper ( g = −.25). Based on moderator analyses, this may have been limited to expository texts ( g = −.32) as there was no difference with narrative texts ( g = −.04). The findings were similar when analysing literal and inferential reading performance separately ( g = −.33 and g = −.26, respectively). No reliable differences were found for reading time ( g = .08). Readers had better calibrated (more accurate) judgement of their performance from paper compared to screens ( g = .20).
Conclusions
Readers may be more efficient and aware of their performance when reading from paper compared to screens.
The “replication crisis” has been attributed to misguided external incentives gamed by researchers (the strategic-game hypothesis). Here, I want to draw attention to a complementary internal factor, namely, researchers’ widespread faith in a statistical ritual and associated delusions (the statistical-ritual hypothesis). The “null ritual,” unknown in statistics proper, eliminates judgment precisely at points where statistical theories demand it. The crucial delusion is that the p value specifies the probability of a successful replication (i.e., 1 – p), which makes replication studies appear to be superfluous. A review of studies with 839 academic psychologists and 991 students shows that the replication delusion existed among 20% of the faculty teaching statistics in psychology, 39% of the professors and lecturers, and 66% of the students. Two further beliefs, the illusion of certainty (e.g., that statistical significance proves that an effect exists) and Bayesian wishful thinking (e.g., that the probability of the alternative hypothesis being true is 1 – p), also make successful replication appear to be certain or almost certain, respectively. In every study reviewed, the majority of researchers (56%–97%) exhibited one or more of these delusions. Psychology departments need to begin teaching statistical thinking, not rituals, and journal editors should no longer accept manuscripts that report results as “significant” or “not significant.”
Loss aversion, the principle that losses loom larger than gains, is among the most widely accepted ideas in the social sciences. The first part of this article introduces and discusses the construct of loss aversion. The second part of this article reviews evidence in support of loss aversion. The upshot of this review is that current evidence does not support that losses, on balance, tend to be any more impactful than gains. The third part of this article aims to address the question of why acceptance of loss aversion as a general principle remains pervasive and persistent among social scientists, including consumer psychologists, despite evidence to the contrary. This analysis aims to connect the persistence of a belief in loss aversion to more general ideas about belief acceptance and persistence in science. The final part of the article discusses how a more contextualized perspective of the relative impact of losses versus gains can open new areas of inquiry that are squarely in the domain of consumer psychology.
This article is protected by copyright. All rights reserved.
Higgins and Liberman (2018) and Simonson and Kivetz (2018) offer scholarly and stimulating perspectives on loss aversion and the implications for the sociology of science of its acceptance as a virtual law of nature. In our view, Higgins and Liberman (2018) largely complement our conclusion that the empirical evidence does not support loss aversion. Moreover, in alignment with our call for a contextualized perspective, they provide an excellent discourse on how a more nuanced view of reference points and consumers’ regulatory focus enriches our understanding of the psychological impact of losses and gains. Simonson and Kivetz (2018) approached our perspective with skepticism, and, while they retain some skepticism, they express agreement on the larger point that loss aversion has been accepted too uncritically. Both commentaries point to a need for a critical reevaluation of prevailing paradigms. Here, we build on these perspectives, as well as our experience working on the topic of loss aversion, to call for structural changes to facilitate scholarly debate on science's status quo.
This article is protected by copyright. All rights reserved.
We agree with Gal and Rucker (2018 – this issue) that loss aversion is not as firmly established as typically assumed. We affirm, however, the more general principle put forward within Prospect Theory (Kahneman & Tversky, 1979), which is that reference points increase people's sensitivity to objective changes in value. We show how the literatures on counterfactual thought, social comparison and goal pursuit are consistent with the notion that reference points increase sensitivity to change in value, while not being consistent with loss aversion. We then examine, within the framework of Regulatory Focus theory (Higgins, 1997, 1998), how different reference points combine with characteristics of the actor and the situation to give rise to loss aversion (more sensitivity to negative outcomes than to positive outcomes) as well as to the reverse pattern (more sensitivity to positive outcomes than to negative outcomes). Our review suggests that the status quo, even when used as a reference point, is not necessarily neutral. It also suggests that anchor points other than the status quo may serve as reference points and that people may use more than one reference point simultaneously. More generally, we call for a critical examination of the “bad is stronger than good” principle.
This article is protected by copyright. All rights reserved.
Although we disagree with some of Gal and Rucker's (2018 – this issue) specific evidence and with their overstated conclusion regarding loss aversion, their overarching message makes a worthwhile contribution. In particular, loss aversion is less robust and universal than has been assumed while its most prominent empirical support – the endowment effect and the status quo bias – is susceptible to multiple alternative explanations. Instead of accepting loss aversion as true unless proven otherwise, we should treat it like other decision properties and psychological accounts that are contingent on various moderators and call for an analysis of psychological mechanisms. In this commentary, we suggest that gatekeepers, such as reviewers, tend to favor loss aversion and other widely accepted tendencies, while demanding a much higher support‐threshold for alternative or newer accounts. Although building on prior theories and concepts is of course important, the bias in favor of incumbent assumptions can impede scientific progress, bar new ideas from the literature, and reinforce well‐established but contingent notions that may apply under some conditions but not others.
This article is protected by copyright. All rights reserved.
This book focuses on what makes people happy. The author explains methods for measuring subjective life satisfaction and well-being by discussing economic and sociodemographic factors, as well as the psychological, cultural and political dimensions of personal happiness.
Does higher income increase happiness? Are people in rich countries, such as the United States, the United Kingdom and Scandinavian countries, happier than those living elsewhere? Does losing one’s job make one unhappy? What is the role of genetic endowments inherited from our parents? How important are physical and emotional health to subjective life satisfaction? Do older people tend to be happier, or younger people? Are close social relationships necessary for happiness? Do political conditions, such as respect for human rights, democracy and autonomy, play a part? How can governments contribute to the population’s happiness? This book answers these questions on the basis of extensive interdisciplinary research reflecting the current state of knowledge.
The book will appeal to anyone interested in learning more about the various dimensions of personal well-being beyond the happiness-prosperity connection, as well as to policymakers looking for guidance on how to improve happiness in societies.
Principles of Forecasting: A Handbook for Researchers and Practitioners summarizes knowledge from experts and from empirical studies. It provides guidelines that can be applied in fields such as economics, sociology, and psychology. It applies to problems such as those in finance (How much is this company worth?), marketing (Will a new product be successful?), personnel (How can we identify the best job candidates?), and production (What level of inventories should be kept?).
The book is edited by Professor J. Scott Armstrong of the Wharton School, University of Pennsylvania. Contributions were written by 40 leading experts in forecasting, and the 30 chapters cover all types of forecasting methods. There are judgmental methods such as Delphi, role-playing, and intentions studies. Quantitative methods include econometric methods, expert systems, and extrapolation. Some methods, such as conjoint analysis, analogies, and rule-based forecasting, integrate quantitative and judgmental procedures. In each area, the authors identify what is known in the form of `if-then principles', and they summarize evidence on these principles.
The project, developed over a four-year period, represents the first book to summarize all that is known about forecasting and to present it so that it can be used by researchers and practitioners. To ensure that the principles are correct, the authors reviewed one another's papers. In addition, external reviews were provided by more than 120 experts, some of whom reviewed many of the papers. The book includes the first comprehensive forecasting dictionary.
In light of recent concerns about reproducibility and replicability, the ASA issued a Statement on Statistical Significance and p-values aimed at those who are not primarily statisticians. While the ASA Statement notes that statistical significance and p-values are “commonly misused and misinterpreted,” it does not discuss and document broader implications of these errors for the interpretation of evidence. In this article, we review research on how applied researchers who are not primarily statisticians misuse and misinterpret p-values in practice and how this can lead to errors in the interpretation of evidence. We also present new data showing, perhaps surprisingly, that researchers who are primarily statisticians are also prone to misuse and misinterpret p-values thus resulting in similar errors. In particular, we show that statisticians tend to interpret evidence dichotomously based on whether or not a p-value crosses the conventional 0.05 threshold for statistical significance. We discuss implications and offer recommendations.
Version 6 of the UAH MSU/AMSU global satellite temperature dataset represents an extensive revision of the procedures employed in previous versions of the UAH datasets. The two most significant results from an end-user perspective are (1) a decrease in the global-average lower tropospheric temperature (LT) trend from +0.14°C decade⁻¹ to +0.11°C decade⁻¹ (Jan. 1979 through Dec. 2015); and (2) the geographic distribution of the LT trends, including higher spatial resolution, owing to a new method for computing LT. We describe the major changes in processing strategy, including a new method for monthly gridpoint averaging which uses all of the footprint data yet eliminates the need for limb correction; a new multi-channel (rather than multi-angle) method for computing the lower tropospheric (LT) temperature product which requires an additional tropopause (TP) channel to be used; and a new empirical method for diurnal drift correction. We show results for LT, the midtroposphere (MT, from MSU2/AMSU5), and lower stratosphere (LS, from MSU4/AMSU9). A 0.03°C decade⁻¹ reduction in the global LT trend from the Version 5.6 product is partly due to lesser sensitivity of the new LT to land surface skin temperature (est. 0.01°C decade⁻¹), with the remainder of the reduction (0.02°C decade⁻¹) due to the new diurnal drift adjustment, the more robust method of LT calculation, and other changes in processing procedures.
ResearchGate is increasingly used by scholars to upload the full-text of their articles and make them freely available for everyone. This study aims to investigate the extent to which ResearchGate members as authors of journal articles comply with publishers' copyright policies when they self-archive full-text of their articles on ResearchGate. A random sample of 500 English journal articles available as full-text on ResearchGate were investigated. 108 articles (21.6%) were open access (OA) published in OA journals or hybrid journals. Of the remaining 392 articles, 61 (15.6%) were preprint, 24 (6.1%) were post-print and 307 (78.3%) were published (publisher) PDF. The key finding was that 201 (51.3%) out of 392 non-OA articles infringed the copyright and were non-compliant with publishers' policy. While 88.3% of journals allowed some form of self-archiving (SHERPA/RoMEO green, blue or yellow journals), the majority of non-compliant cases (97.5%) occurred when authors self-archived publishers' PDF files (final published version). This indicates that authors infringe copyright most of the time not because they are not allowed to self-archive, but because they use the wrong version, which might imply their lack of understanding of copyright policies and/or complexity and diversity of policies.
Natural experiments or quasi-natural experiments in economics are serendipitous situations in which persons are assigned randomly to a treatment (or multiple treatments) and a control group, and outcomes are analysed for the purposes of putting a hypothesis to a severe test; they are also serendipitous situations where assignment to treatment ‘approximates’ randomized design or a well-controlled experiment.