No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
A Guide to Delphi
Abstract
When we rely on the judgment of experts to help produce our forecasts, the key issues are how to get the appropriate information from our consultants and how to get a forecast if we are using multiple experts. Gene Rowe describes the Delphi method, tells what it offers to the forecaster, and explains what the pitfalls are in its implementation. Copyright International Institute of Forecasters, 2007
... Expert judgment approaches essentially divide into those that utilise many experts, and those that utilise a relatively smaller number of experts (e.g. workshops with up to 20 participants) (Armstrong, 2001;Rowe, 2007). The former type generally involves the use of questionnaires, and is typified by the Delphi approach, in which a survey is reiterated over a number of 'rounds', during which feedback from prior rounds is provided and at which experts are encouraged to rethink the problem and revise their estimates if they deem this appropriate. ...
... The latter type of approaches generally involve the use of highly structured small group methods, which require experts to be present at one place/time, and which may take place (in the first instance) over a couple of days. 'Structure' is important, because unstructured group meetings are well-known to suffer from a wide variety of problems, such as over-dominance by dogmatic individuals, political game playing, premature closure of discussions, the tendency of individuals in groups to discuss common knowledge at the expense of unusual or uniquely held ideas, and so on (Armstrong, 2001;Rowe, 2007). Most of these methods involve: the use of experienced facilitators who are adept at pre-empting some of the typical group process problems; brainstorming-like approaches, to ensure that problems are not toonarrowly framed; open or secret voting, to ensure that all participants have a say; use of materials (e.g. ...
... For example, a method using expert judgment may also supply relevant information from databases for the experts to consider, and many foresight/forecasting approaches may be multi-stage, and involve a number of different processes over the course of an entire project (e.g. Delphi and scenarios, with additional literature and database searches) (Armstrong, 2001;Rowe, 2007). ...
... regional zu analysieren. Armstrong, 2006a;Häder, 2009;Powell, 2003;Rowe, 2007;Rowe & Wright, 1999, 2001 Weitere als wirksam betrachtete Programme beinhalten Maßnahmen, die auf das Eingreifen von Zeugen/Zuschauern von Straftaten ("Gewalt -Sehen -Helfen"; "Wer nichts tut, macht mit"), Alkoholmissbrauch von Minderjährigen ("Keine Kurzen für Kurze"), Verkehrssicherheit ("Guardian Angels"-Projekte), Sicherheit im öffentlichen Nahverkehr ("Bus-Coach-Projekt"), Fairplay im Sport ("Fair ist mehr") oder Medienkompetenzen ("Mit Sicherheit durchs Netz") abzielen. ...
... Im Rahmen der nationalen Institutionen-und Expertenbefragung beschrieben die befragten Expertinnen Armstrong, 2006a;Häder, 2009;Powell, 2003;Rowe, 2007;Rowe & Wright, 1999;2001 ...
From the project website (youprev.eu): In the years 2011 and 2012, the study „Youth deviance and youth violence: A European multi-agency perspective on best practices in prevention and control“ (YouPrev) has been conducted in six European countries (Belgium, Germany, Hungary, Portugal, Slovenia and Spain). The main goal of the project was to deepen knowledge on prevention and control of juvenile delinquent behaviour and to make this knowledge available to relevant professional groups and institutions. Multiple methodological approaches were taken, including reviews of existing preventive approaches via expert surveys and analyses of available materials, local studies of juvenile delinquency and its prevention in urban and rural areas based upon surveys among adolescents in school contexts and on interviews with multi-professional samples of practitioners, Delphi surveys, and national and international multi-professional workshops. Manuals for continuing education of police and social work professionals were set up in each country; brochures summarizing key results and pointing out policy implications were published, and findings were presented and discussed at an international conference.
YouPrev results emphasize that juvenile delinquency is on the one hand a universal developmental phenomenon. Breaking rules is part of the process of growing up, and most adolescents commit some offences during their adolescence. At the same time, juvenile delinquency is socially embedded. Family and peers are of utmost importance. Growing up under conditions of low or unstable parental supervision, spending one’s time with delinquent friends are both highly significant risk factors for delinquent behaviour. Social disorganization of the neighbourhood is a risk factor for juvenile violence. The wider social and economic context is also a determining factor. While youth crime will in many aspects remain stable in the near and intermediate future, it will also be affected by large societal trends, especially demographic change (population ageing), technological change, and economic trends and developments.
High-quality prevention measures share common features such as being targeted at risk factors and protective factors, being directed towards at-risk populations and including early intervention, connecting multiple institutions and professions (especially police, school and social work) and including evaluative measures. YouPrev findings support the importance of tailoring specific preventive measures to the minority of repeat and persistent offenders. Given the multitude of strains and risk factors which characterize this group, the need for cooperation between social work services and the police / criminal justice system becomes evident. YouPrev findings clearly show that young people have concepts of factors influencing their behaviour and hence of delinquency prevention. Analyses of young persons’ concepts of successful crime prevention provide hints as to the accessibility of youngsters through preventive measures and agents. They stress the importance of considering interpersonal networks and informal social control as key factors for success or failure of preventive measures.
... Expert judgment approaches essentially divide into those that utilise many experts, and those that utilise a relatively smaller number of experts (e.g. workshops with up to 20 participants) (Armstrong, 2001;Rowe, 2007). The former generally involves the use of questionnaires, and is typified by the Delphi approach, in which a survey is reiterated over a number of "rounds", during which feedback from prior rounds is provided and at which experts are encouraged to rethink the problem and revise their estimates if they deem this appropriate. ...
... The latter approache generally involve the use of highly structured small group methods, which require experts to be present at one place/time, and which may take place (in the first instance) over a couple of days. "Structure" is important, because unstructured group meetings are well-known to suffer from a variety of problems, such as over-dominance by dogmatic individuals, political game playing, premature closure of discussions, the tendency of individuals in groups to discuss common knowledge at the expense of unusual or uniquely held ideas and so on (Armstrong, 2001;Rowe, 2007). ...
... Ashton and Ashton (1985) recommend combining independent forecasts from a group of experts, whereby the level of expertise does not have to be extremely high, and weight each expert's forecast equally. Rowe (2007) presents evidence that the accuracy of expert forecasts can be improved through the use of the Delphi technique. This is an iterative survey, where experts are asked to provide reasoning for their forecasts, to respond and give feedback to the forecasts done by other anonymous experts (Green et al., 2007). ...
... This feedback and information is then used to revise the forecasts. If done properly, this method enhances unbiased estimates and forecasts and takes full account of the knowledge and judgment of all experts (Rowe, 2007). ...
... The practical implementation of long-terms methods regarding the problem of forecasting the prospects of oil and gas offshore projects can be realized with the help of experts. Expert methods also require background information, which serves as a starting point for the formation of expert assessments [23][24][25]. Such reference data are necessary to understand the dynamics of the predicted phenomenon, the nature of the relationship and the interdependence between the various indicators and events characterizing it. ...
The problem of the development of offshore oil and gas Arctic projects has risen recently due to the depletion of traditional overland oil and gas fields, as well as the discovery of significant hydrocarbon reserves in the Arctic seas and the strategic importance of the development of that zone for the Russian Federation. The decision-making process associated with the development of these projects takes place in high level of uncertainty of external environment. Consequently, existing tools and methods of forecasting the prospects of the projects are hardly applicable due to its specificity. This article presents a special approach for assessing the prospects of oil and gas offshore Arctic projects. It consists of a theoretical base, which is the newly developed TESCIMP-methodology, practical tools for implementing it, as well as a complex approach for the assessment. The research is segmented into four stages: the analysis of the factors influencing project’s prospects as well as the key pros and cons for developing it, a two-dimensional classification of specific indicators which allocates essential and stimulating ones and forecasting the prospects of the project, identification of projects’ levels which implies the selection of appropriate TESCIMP-factors. The indicators within those factors underlie practical tools of checklist and expert surveys that help evaluate the projects’ prospects. The last stage proposes a complex approach for a further assessment, which consists of three steps and based on the combination of expert surveys and statistical methods.
... 7 The Delphi method was developed at the RAND Corporation in the 1950's for predicting bombing requirements in a cold-war conflict with the USSR [12]. Surveys of the Delphi method are provided by [13], [36] and [37], while [29] review the judgmental forecasting approach. ...
Preliminary evidence on the long--run trajectory of the accelerator industry suggests that it may be close to the maturity phase of its cycle. If this is the case, how can we measure the benefits of an uncertain breakthrough in acceleration technology? Who are the main stakeholders interested by such a breakthrough? We identify these subjects and sketch some avenues for answering these questions. We thus present a model for the social Cost-Benefit Analysis (CBA) of research infrastructures and illustrate the results of its implementation for assessing the benefits of accelerators in basic science and hadrontherapy. Lastly, we move from the social CBA of single research infrastructures to modeling a major change in the accelerator technology and hence in the industry. A research agenda on the potential impacts of a technological breakthrough is presented.
... To overcome biases and group pressure, it is helpful to introduce anonymity, diversity, and opportunity for dissent into the forecast process. The long-established Delphi method introduces such principles (for an overview of the Delphi method, see Guide to Delphi, Rowe, 2007). So, too, does the Key Points ■ To overcome biases and group pressure, it is helpful to introduce anonymity, diversity, and opportunity for dissent into the forecast process. ...
Despite great acclaim for the potential and use of prediction markets as an efficient
tool for aggregating individual judgments, corporate adoption has been limited. To
find out why, Thomas Wolfram examined existing research and interviewed several dozen
key business executives. He reports that problems with overall acceptance of the corporate
prediction market (CPM) often stem from a lack of trust by management as well as a greater
business focus on big data and social-media content. However, hope for the CPM persists, if
certain obstacles can be finessed.
... Expert surveys are main source of information to predicting future throughput with their rich experience as technologist, operators, planners and others. Delphi method [33] is one of the popular approach by welldefining questionnaires to specific parties that yields a consensus of factors and opinions on future container throughput. ...
Planning of Container Terminal equipment has always been uncertain due to seasonal and fluctuating throughput demand, along with factors of delay in operation, breakdown and maintenance. Many time-series models have been developed to forecast the unforeseen future of container throughput to project the needed amount of port equipments for optimum operation. Conventionally, a "ratio" method developed by port consultants at early port design stage is adopted for equipment planning, giving no consideration to the dynamic growth of the port in terms of improved layout and technological advancement in equipments. This study seeks first to enhance the empirical approach of the equipment planning at the end of planning time horizon by including assumed coefficient of port capacity parameters. The second is to compare the size of equipment purchase by receiving different terminal's future throughput demand from two univariate forecasting models at planning time horizon. The empirical method of equipment planning will be tested against the conventional yard equipment per quay crane ratio after deriving the throughput demand from forecasting models of Holt-Winter's exponential smoothing and seasonal ARIMA (autoregression integrated moving average) model. Results in the form of graphs and tables indicate similar forecasting pattern by two models and equipment estimation proofs to avail more redundancy for optimum operation. Suggestions for better estimation of equipments are also made for future models.
... Expert surveys are main source of information to predicting future throughput with their rich experience as technologist, operators, planners and others. Delphi method [33] is one of the popular approach by welldefining questionnaires to specific parties that yields a consensus of factors and opinions on future container throughput. ...
Planning of Container Terminal equipment has always been uncertain due to seasonal and fluctuating throughput demand, along with factors of delay in operation, breakdown and maintenance. Many timeseries models have been developed to forecast the unforeseen future of container throughput to project the needed amount of port equipments for optimum operation. Conventionally, a "ratio" method developed by port consultants at early port design stage is adopted for equipment planning, giving no consideration to the dynamic growth of the port in terms of improved layout and technological advancement in equipments. This study seeks first to enhance the empirical approach of the equipment planning at the end of planning time horizon by including assumed coefficient of port capacity parameters. The second is to compare the size of equipment purchase by receiving different terminal's future throughput demand from two univariate forecasting models at planning time horizon. The empirical method of equipment planning will be tested against the conventional yard equipment per quay crane ratio after deriving the throughput demand from forecasting models of Holt-Winter's exponential smoothing and seasonal ARIMA (autoregression integrated moving average) model. Results in the form of graphs and tables indicate similar forecasting pattern by two models and equipment estimation proofs to avail more redundancy for optimum operation. Suggestions for better estimation of equipments are also made for future models.
The introduction of MASSs, facilitated by the advancement of autonomous navigation technologies, is anticipated to lead to the emergence of new technologies, novel vessel types, and innovative job positions like remote operators at remote operation centers. The MASS Code is currently being discussed by the Maritime Safety Committee of the International Maritime Organization. This Code is expected to be adopted in the form of non-mandatory guidelines until 2025, with the goal of establishing mandatory requirements by 2028. Additionally, revisions to the International Convention on Standards of Training, Certification and Watchkeeping for Seafarers related to crew training and qualifications are planned for adoption by 2027. These revisions will include requirements for MASS operators. This paper aims to examine the sustainable key competencies required for safe MASS operation by remote operators using the Analytic Hierarchy Process method, considering the emergence of the new profession of remote operator. Building upon the 66 knowledge, understanding, and proficiencies outlined for marine officers in the STCW Convention, the paper aims to identify the additional competencies required for remote operators and contribute to the development of a training model for the future.
The COVID-19 pandemic has had a devastating effect on many industries around the world including tourism and policy makers are interested in mapping out what the recovery path will look like. We propose a novel statistical methodology for generating scenario-based probabilistic forecasts based on a large survey of 443 tourism experts and stakeholders. The scenarios map out pessimistic, most-likely and optimistic paths to recovery. Taking advantage of the natural aggregation structure of tourism data due to geographic locations and purposes of travel, we propose combining forecast reconciliation and forecast combinations implemented to historical data to generate robust COVID-free counterfactual forecasts, to contrast against. Our empirical application focuses on Australia, analyzing international arrivals and domestic flows. Both sectors have been severely affected by travel restrictions in the form of international and interstate border closures and regional lockdowns. The two sets of forecasts, allow policy makers to map out the road to recovery and also estimate the expected effect of the pandemic.
Making efficient and effective decisions in the chaotic environment of humanitarian relief distribution (HRD) is challenging. Decision-makers need to concentrate on numerous decision factors categorized into decision objectives, variables, and constraints. Recent HRD literature focuses on optimizing procedures while neglecting the quantification of essential requirements (decision factors) for information systems to provide decision-making support. In this article, we address this gap by accumulating affecting decision factors from both literature and practice. We investigated the practical implications of these factors in HRD decision-making by measuring the preferences of a Delphi panel consisting of 23 humanitarian experts. The results from our study emphasize the importance of the decision factors in the proposed process model for HRD in a large-scale sudden onset. Our work provides researchers not only with a comprehensive set of practically feasible decision factors in HRD but also with an understanding of their influences and correlations.
El presente trabajo pretende analizar los factores determinantes que condicionan las ventas anuales en una empresa comercializadora, centrándose en el estudio de las principales variables y su incidencia en los resultados. El trabajo se ha dividido en cuatro bloques principales. En primer lugar se ha realizado una revisión literaria de las estrategias y métodos de previsión existentes, empleados en el ámbito económico de empresas. En segundo lugar se ha efectuado un análisis de las variables influyentes, externas e internas a considerar para analizar la situación y evolución de resultados de ventas a lo largo de sucesivos periodos. En tercer lugar, se presenta una propuesta de procedimiento y horizonte predictivo. Finalmente, en el último bloque se realiza la previsión basada en la Metodología ARIMA con los datos reales conocidos en una empresa tipo.
Traditional groups meetings are an inefficient and ineffective method for making forecasts and decisions. We compare two structured alternatives to traditional meetings: the Delphi technique and prediction markets. Delphi is relatively simple and cheap to implement and has been adopted for diverse applications in business and government since its origins in the 1950s. It can be used for nearly any forecasting, estimation, or decision making problem not barred by complexity or ignorance. While prediction markets were used more than a century ago, their popularity waned until more recent times. Prediction markets can be run continuously, and they motivate participation and participants to reveal their true beliefs. On the other hand, they need many participants and clear outcomes in order to determine pay-offs. Moreover, translating knowledge into a price is not intuitive to everyone and constructing contracts that will provide a useful forecast may not be possible for some problems. It is difficult to maintain confidentiality with markets and they are vulnerable to manipulation. Delphi is designed to reveal panelists’ knowledge and opinions via their forecasts and the reasoning they provide. This format allows testing of knowledge and learning by panelists as they refine their forecasts but may also lead to conformity due to group pressure. The reasoning provided as an output of the Delphi process is likely to be reassuring to forecast users who are uncomfortable with the “black box” nature of prediction markets. We consider that, half a century after its original development, Delphi is under-utilized.
ResearchGate has not been able to resolve any references for this publication.