Article

RT Delphi: An efficient, “round-less” almost real time Delphi method

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The authors have recently developed a new approach to performing a Delphi study that does not involve the use of sequential “rounds” and as a result, greatly improves the efficiency of the process and shortens the time to perform such studies. This paper describes this process, RT Delphi, and illustrates its use in a decision-making application drawn from the Millennium Project of the American Council for the United Nations University. The illustrative application involves setting priorities among strategies for dealing with anticipated terrorist activities that might be initiated by a single deranged individual.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... A "Real-Time Delphi" process may offer benefits in the development of COS. The Real-Time Delphi method was developed by Gordon and Pease [6] to improve the speed and efficiency of gathering opinions of experts and making decisions in situations of urgency. As software has developed since then, so too have approaches to using the Real-Time Delphi method. ...
... This provides feedback to participants more quickly than at the end of a Multi-Round Delphi survey round, which often last up to 3 weeks in COS [4]. The Real-Time Delphi method comprises a round-less Delphi approach [6], where participants are encouraged to re-visit the survey and re-rate items throughout the period in which the Delphi survey is live. Essentially, it removes the time taken in the Multi-Round Delphi for the survey administrator to evaluate the results and provide feedback to participants. ...
... Despite claims that a Real-Time Delphi will improve efficiency of the survey process by removing the strict rounds and the time it takes for participants to complete the survey [6,8], there is no strong evidence on how it compares to the Multi-Round Delphi. ...
Article
Full-text available
Background The Delphi method is used in a wide variety of settings as a method of building consensus on important issues. Traditionally, the Delphi method uses multiple rounds of a survey to allow for feedback of other participants’ survey responses in between rounds. By informing participants about how others answer a question or prioritise specific topics, it allows for diverse opinions to inform the consensus process. For this reason, the Delphi method is popular as a consensus building approach in developing core outcome sets (COS), i.e. the minimum agreed set of standardised outcomes that should be measured and reported in studies on a specific health condition. In a COS setting, participants prioritise the importance of outcomes for inclusion in a COS. This usually involves participating in multiple rounds of a survey that can span several weeks or months. Challenges with participant retention have been highlighted in previous COS. We will compare a three-round with a Real-Time Delphi approach on prioritised outcomes. This trial is embedded within the COHESION study which is developing a COS for interventions treating neonatal encephalopathy. Methods One hundred and eighty stakeholders (parents/caregivers of infants diagnosed and treated with neonatal encephalopathy, healthcare providers and researchers) will be randomised using stratified randomisation to take part in either the Multi-Round or Real-Time Delphi. Stakeholders will rate the importance of the same set of outcomes in both arms. We will compare the prioritised outcomes at the end of both surveys as well as other parameters such as feedback, initial condition and iteration effects. Discussion This trial will provide evidence to inform decisions on the use of Multi-Round compared to Real-Time Delphi survey methods. Trial registration NCT04471103. Registered on 14 July 2020.
... The name Delphi method was proposed by the philosopher Abraham Kaplan, who was part of the American research center The Rand Corporation, where it was developed in the late 1950s and 1960s as an effective means of collecting and synthesizing expert opinions. Since the first RAND study was published in 1964, the technique has been used very frequently in a wide range of subjects (Celiktas & Kocar, 2010a;Banuls & Salmeron, 2008;Yuan, Chih-Hung Hsieh & Wang, 2006;Gordon & Pease, 2006;Builes & Manrique, 2000). It became popular when applied to a large-scale national technology forecast in the 1960s in the UU. ...
... As for Andersen et al (2003), they carried out a technology foresight study on sensor technology to improve the strategic perspective of sensor technology, (Tichy, 2004) discusses whether foresight exercises should be based on assessments by the best experts or a broader base of less specialized experts and whether self-assessment by experts is an acceptable method. Also (Gordon & Pease, 2006) they propose the development of a new approach to conducting a Delphi study that does not involve the use of sequential rounds called Real-Time Delphi. ...
... Comparing the traditional methods of Popper (2008) and (Smith & Saritas, 2011), new methods for prospective studies were found which are shown in the following table: (Santos, Araujo & Correia, 2017;Nazarko et al.,2014;Kobayashi et al., 2006;Gordon & Pease, 2006;Andersen et al., 2003) Interviews ...
Article
Full-text available
The purpose of this article is to identify new techniques and tools related to the application of the Delphi method, as well as to show which of these techniques and tools have been applied in future studies for emerging technologies, the methodology used was carried out through the in-depth review of 50 articles directly related and obtained from the Scopus database, which had as criteria the focus on emerging technologies, sector of application, time horizon and methods used, an analysis on two axes of discussion was carried out techniques and methods used in future studies compared with the proposals by authors such as Popper and Smith and Saritas, and applications in emerging technologies, at the end 26 new methods and 153 emerging technologies were found prioritized.
... Two clear limitations of the classic Delphi technique were prolonged study durations and high panel member attrition [25]. Aiming to overcome these issues, Gordon and Pease [26] developed the concept of an information technology-enabled contemporaneous extension called real-time Delphi, to improve speed of the data collection process and syntheses of opinions. Conducting a real-time Delphi relies on specially designed software to administer the survey; the functionality or capabilities of which can negatively impact on the success of a study. ...
... According to Best [53], accuracy can be improved if experts are provided with both quantitative and qualitative arguments. In a real-time Delphi experts are able to immediately react on each other's responses, increasing the degree of information experts can interact with, which may aid in recapturing their own point of view [26]. ...
Article
Full-text available
The modified Delphi technique is widely used to develop consensus on group opinion within health services research. However, digital platforms are offering researchers the capacity to undertake a real-time Delphi, which provides novel opportunities to enhance the process. The aim of this case study is to discuss and reflect on the use of a real-time Delphi method for researchers in emergency nursing and cognate areas of practice. A real-time Delphi method was used to develop a national survey examining knowledge, perceptions and factors influencing pain assessment and management practices among Australian emergency nurses. While designing and completing this real-time Delphi study, a number of areas, emerged that demanded careful consideration and provide guidance to future researchers.
... 36 As part of the review, we foresee discussing how to assess the potential risk of bias (which can lead to misuse and patients' harm), and if current tools such as 'PROBAST' are appropriate. 37 Stage 3: real-time Delphi exercise We will conduct a real-time Delphi survey 38 to review and refine the items of the PRECOG reporting checklist. Participants will be identified initially through the professional network of the core working group and of the external advisors, and further via literature search (including but not limited to the existing scoping review and the planned systematic review), social media screening and snowballing by the active participants. ...
... The participant selection will be based on area expertise grouping (computer science, biostatistics, biomedical informatics, statistics, epidemiology, standards, causal inference, ethics), used to determine the sample size (discussed below). We choose a computer-based, realtime Delphi, 38 since it offers some operational advantages with respect to conventional multi-round Delphi Open access techniques, for example, responder's attrition. 39 In brief, real-time Delphi is a 'roundless' exercise based on an online survey platform. ...
Article
Full-text available
Introduction While there are guidelines for reporting on observational studies (eg, Strengthening the Reporting of Observational Studies in Epidemiology, Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement), estimation of causal effects from both observational data and randomised experiments (eg, A Guideline for Reporting Mediation Analyses of Randomised Trials and Observational Studies, Consolidated Standards of Reporting Trials, PATH) and on prediction modelling (eg, Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis), none is purposely made for deriving and validating models from observational data to predict counterfactuals for individuals on one or more possible interventions, on the basis of given (or inferred) causal structures. This paper describes methods and processes that will be used to develop a Reporting Guideline for Causal and Counterfactual Prediction Models (PRECOG). Methods and analysis PRECOG will be developed following published guidance from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network and will comprise five stages. Stage 1 will be meetings of a working group every other week with rotating external advisors (active until stage 5). Stage 2 will comprise a systematic review of literature on counterfactual prediction modelling for biomedical sciences (registered in Prospective Register of Systematic Reviews). In stage 3, a computer-based, real-time Delphi survey will be performed to consolidate the PRECOG checklist, involving experts in causal inference, epidemiology, statistics, machine learning, informatics and protocols/standards. Stage 4 will involve the write-up of the PRECOG guideline based on the results from the prior stages. Stage 5 will seek the peer-reviewed publication of the guideline, the scoping/systematic review and dissemination. Ethics and dissemination The study will follow the principles of the Declaration of Helsinki. The study has been registered in EQUATOR and approved by the University of Florida’s Institutional Review Board (#202200495). Informed consent will be obtained from the working groups and the Delphi survey participants. The dissemination of PRECOG and its products will be done through journal publications, conferences, websites and social media.
... Moreover, after several months the consensus may already be outdated again. Real-Time Delphi (RTD) is a more efficient computer-aided and roundless way to conduct Delphi studies, which reduces several disadvantages of a classical Delphi study such as the time needed to conduct it by allowing the experts' answers to be statistically analyzed and processed in "real-time" [3][4][5]. ...
... If the participant was not the first and is above the threshold defined by the creator, then, after answering all theses of a topic area, they see the aggregated answers in the form of a boxplot and, for the textual data, a listing (3). They then have the opportunity to change their answers again (4). After answering the theses, the participants can also view the results of the study and the possibly changed aggregated values after any period of time by simply using the previously used link again. ...
Conference Paper
Full-text available
This paper shows the design of a prototype for conducting Real-Time-Delphi studies at time-critical events in research projects or workshops where experts are only available once. Real-Time-Delphi studies are used to get a consensus from experts about future developments of a specific topic. Application of the prototype in these contexts mitigates issues from classical Delphi Studies, as it allows them to be carried out in significantly shorter periods of time. The development of the tool went through several iterations with first applications and evaluations from a researcher's and a lecturer's perspective.
... Despite the success of such studies, they were expensive and time consuming. A variation known as Real-Time Delphi was introduced by Gordon and Pease [34]. This method preserved the key principles of conventional Delphi surveys, which are: (1) anonymity, (2) controlled feedback of responses to all group members, (3) iteration, and (4) statistical aggregation of individuals' responses) [35]. ...
... We customized a web-program developed as part of prior research [34,38] for our project's Real-Time Delphi survey. It comprised five main pages, one per substance. ...
Article
Full-text available
Although HIV and substance use disorders (SUDs) constitute a health syndemic, no research to date has examined the perceived negative impacts of different SUDs for people with HIV (PWH). In May 2019, 643 stakeholders in the U.S., representing clients of AIDS service organizations (ASOs), ASO staff, and HIV/AIDS Planning Council members, participated in an innovative Stakeholder-Engaged Real-Time Delphi (SE-RTD) survey focused on the prevalence and individual-level negative impact of five SUDs for PWH. The SE-RTD method has advantages over conventional survey methods by efficiently sharing information, thereby reducing the likelihood that between-group differences are simply due to lack of information, knowledge, and/or understanding. The population-level negative impacts were calculated by weighting each SUD’s individual-level negative impact on indicators of the HIV Care Continuum and other important areas of life by the perceived prevalence of each SUD. Overall, we found these SUDs to have the greatest population-level negative impact scores (possible range 0–24): alcohol use disorder (population-level negative impact = 6.9; perceived prevalence = 41.9%), methamphetamine use disorder (population-level negative impact = 6.5; perceived prevalence = 3.2%), and opioid use disorder (population-level negative impact = 6.4; perceived prevalence = 34.6%). Beyond further demonstration of the need to better integrate SUD services within HIV settings, our findings may help inform how finite funding is allocated for addressing the HIV-SUD syndemic within the U.S. Based on our findings, such future efforts should prioritize the integration of evidence-based treatments that help address use disorders for alcohol, methamphetamine, and opioids.
... The choice of experts is an essential part of a Delphi study (Gordon and Pease, 2006). Forming a panel of experts does not follow the principle of statistical representativeness (de Loë et al., 2016). ...
... Projections with an EP between 30% and 40% showed a high degree of non-occurrence (Keller and von der Gracht, 2014). In order to determine consensus among experts, we used the IQR as a frequently used indicator (Gordon and Pease, 2006;Keller and von der Gracht, 2014;Warth et al., 2013). The experts reached consensus when a projection achieved "an IQR of ≤ 25%" (Keller and von der Gracht, 2014, p. 86). ...
Article
Blockchain technology is gaining awareness and drawing attention in corporate practice and academia. Both fields expect a fundamental impact of blockchain on business and society. However, since blockchain research within the business model context is still in a nascent stage, more in-depth insights is required of blockchain's impact on firms’ value creation and value capture. This study builds on a Delphi approach and aims to identify the future value creation potential of blockchain for organizations by 2030. Based on expert interviews, workshop insights, and prior literature, we developed a meaningful set of 36 projections of blockchain implications for business models. Our findings, based on the elements of the PEST framework, predict massive efficiency gains through technological progress and promise complementary offerings through various novel combination possibilities, novel forms of collaboration and business model opportunities, and a dissipation of the significance of blockchain types. The combined use of blockchain solutions with other technologies is likely to serve as the basis for ecosystem developments. Our projected finding is that the internet of value will replace the internet of information by 2030. Thereby, our research contributes to technological forecasting and strategic planning by providing managers clear indications of blockchain developments and action recommendations.
... 38 The RT-Delphi approach was developed by Gordon and Pease to improve efficiency of the process and shorten the time of performance. 39 Since then, several online tools have been developed to facilitate the RT-Delphi design 40 and literature describing the use of RT-Delphi and comparison with the traditional multiround Delphi approach is growing. 23 41-44 In contrast to the traditional Delphi, the real-time approach is round-less and offers a constant iteration by providing immediate (real-time) individual and aggregated feedback. ...
... Participants are encouraged to revisit and engage in the survey several times during the study period. 39 40 42 44 In comparison with the traditional approach, the real-time approach encompasses all key Delphi features 43 and is similar from all key perspectives. 23 41 43 44 Furthermore, the real-time approach is particularly suitable for managing larger groups, decreases moderators' workload, simplifies inclusion of people from different geographic locations and can be leaner in costs. ...
... 38 The RT-Delphi approach was developed by Gordon and Pease to improve efficiency of the process and shorten the time of performance. 39 Since then, several online tools have been developed to facilitate the RT-Delphi design 40 and literature describing the use of RT-Delphi and comparison with the traditional multiround Delphi approach is growing. 23 41-44 In contrast to the traditional Delphi, the real-time approach is round-less and offers a constant iteration by providing immediate (real-time) individual and aggregated feedback. ...
... Participants are encouraged to revisit and engage in the survey several times during the study period. 39 40 42 44 In comparison with the traditional approach, the real-time approach encompasses all key Delphi features 43 and is similar from all key perspectives. 23 41 43 44 Furthermore, the real-time approach is particularly suitable for managing larger groups, decreases moderators' workload, simplifies inclusion of people from different geographic locations and can be leaner in costs. ...
Article
Full-text available
Introduction: An online interactive repository of available medication adherence technologies may facilitate their selection and adoption by different stakeholders. Developing a repository is among the main objectives of the European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE) COST Action (CA19132). However, meeting the needs of diverse stakeholders requires careful consideration of the repository structure. Methods and analysis: A real-time online Delphi study by stakeholders from 39 countries with research, practice, policy, patient representation and technology development backgrounds will be conducted. Eleven ENABLE members from 9 European countries formed an interdisciplinary steering committee to develop the repository structure, prepare study protocol and perform it. Definitions of medication adherence technologies and their attributes were developed iteratively through literature review, discussions within the steering committee and ENABLE Action members, following ontology development recommendations. Three domains (product and provider information (D1), medication adherence descriptors (D2) and evaluation and implementation (D3)) branching in 13 attribute groups are proposed: product and provider information, target use scenarios, target health conditions, medication regimen, medication adherence management components, monitoring/measurement methods and targets, intervention modes of delivery, target behaviour determinants, behaviour change techniques, intervention providers, intervention settings, quality indicators and implementation indicators. Stakeholders will evaluate the proposed definition and attributes' relevance, clarity and completeness and have multiple opportunities to reconsider their evaluations based on aggregated feedback in real-time. Data collection will stop when the predetermined response rate will be achieved. We will quantify agreement and perform analyses of process indicators on the whole sample and per stakeholder group. Ethics and dissemination: Ethical approval for the COST ENABLE activities was granted by the Malaga Regional Research Ethics Committee. The Delphi protocol was considered compliant regarding data protection and security by the Data Protection Officer from University of Basel. Findings from the Delphi study will form the basis for the ENABLE repository structure and related activities.
... Regarding this study's research, a web-based Delphi survey design was chosen. This Delphi format follows the general principles of the Delphi procedure involving anonymity, iteration, guided feedback, and the statistical aggregation of group estimates, but additionally includes a real-time calculation of the group's opinion (Gordon and Pease 2006). The approach provides results that are comparable to conventional Delphi surveys (Gnatzy et al., 2011) while increasing process efficiency, facilitating greater flexibility in participation, and being more convenient for participants (Aengenheyster et al., 2017;Gnatzy et al., 2011;Gordon and Pease 2006). ...
... This Delphi format follows the general principles of the Delphi procedure involving anonymity, iteration, guided feedback, and the statistical aggregation of group estimates, but additionally includes a real-time calculation of the group's opinion (Gordon and Pease 2006). The approach provides results that are comparable to conventional Delphi surveys (Gnatzy et al., 2011) while increasing process efficiency, facilitating greater flexibility in participation, and being more convenient for participants (Aengenheyster et al., 2017;Gnatzy et al., 2011;Gordon and Pease 2006). ...
Article
Bringing innovative products to markets is essential for companies, especially in competitive environments. One of the driving forces for successful new product development (NPD) is the inclusion of the functions that form part of the process such as purchasing and supply management (PSM). While extensive research has been conducted on the role of procurement in promoting innovation, insufficient research on the future of the PSM function in NPD has been conducted. This study addresses this gap, considering increasing digitalization and its effect on the future PSM function managing purchasing and innovation. To explore the future of PSM in NPD, a Delphi survey with 91 experts from industry, academia, and politics or associations was performed. The results reveal that digitalization will influence the PSM function in NPD across different time horizons and forms depending on the subject area. While in many areas, an interplay between digital technologies and human PSM professionals is expected, in certain domains the PSM function is likely to lose added value compared to digital technologies. Furthermore, digitalization is expected to be a potential enabler of a stronger key innovation role for PSM in NPD but to also introduce new requirements such as data science capabilities.
... In particular, the Delphi method, whose term derives from the Greek oracle Delphos, is a systematic and interactive method of prediction, which is based on a panel of experts and is carried out through a series of iterations, called rounds. Many academic works have adopted this method since its development [14,[39][40][41][42][43][44]. As the core of the Delphi approach, experts are required to evaluate projections (representations of possible futures) and assess their societal impact and the likelihood that they will occur within a specific time horizon. ...
... The way the questionnaire was structured was exactly as the e-Delphi website suggests, but for practical reasons we edited the survey using Google Form. Other standards, such as the real-time Delphi solution proposed by several studies [14,42,43,49] could have led to a better comparison among experts, but would have likely caused more withdraws to the survey. ...
Article
Full-text available
In this paper, we use a Delphi approach to investigate whether, and to what extent, blockchain-based applications might affect firms’ organizations, innovations, and strategies by 2030, and, consequently, which societal areas may be mainly affected. We provide a deep understanding of how the adoption of this technology could lead to changes in Europe over multiple dimensions, ranging from business to culture and society, policy and regulation, economy, and technology. From the projections that reached a significant consensus and were given a high probability of occurrence by the experts, we derive four scenarios built around two main dimensions: the digitization of assets and the change in business models.
... "Dicho método también mejora la eficiencia del proceso y optimiza el tiempo para la realización de este tipo de estudios" (Gordon y Pease, 2006), beneficios que tienen un efecto positivo a corto plazo, pues mejorar la eficiencia del proceso también abre el surgimiento de aportes que complementan los resultados obtenidos. De tal manera, los investigadores tendrán formas de acelerar el proceso y darle mayor profundidad debido al surgimiento de nuevas interrogantes, lo cual los hace más competentes. ...
Chapter
Full-text available
En el trabajo se describen experiencias de innovación educativa y organizacional llevadas a cabo en la Universidad de Siena (especialmente por parte del "Teaching & Learning Center" del Santa Chiara Lab) que parecen particularmente significativas para describir los procesos de cambio que tienen lugar y los desafíos que las universidades deben enfrentar.
... One Delphi variant, Real-time (RT) Delphi (Gordon & Pease, 2006), is not only asynchronous within rounds, but entirely "roundless": the iterative process (providing individual responses, viewing information from other participants, and amending responses) is not synchronized by a facilitator even at the end of a round; rather, each participant can iterate immediately, even if other participants have not yet done so. This is even more flexible in the timing of participants' contributions, and can potentially speed up the Delphi process. ...
Article
Full-text available
In many complex, real‐world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high‐quality e‐courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web‐based software platform and Delphi‐style social processes. The result is an end‐to‐end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN‐building and collaboration combine beneficially and cumulatively.
... "Dicho método también mejora la eficiencia del proceso y optimiza el tiempo para la realización de este tipo de estudios" (Gordon y Pease, 2006), beneficios que tienen un efecto positivo a corto plazo, pues mejorar la eficiencia del proceso también abre el surgimiento de aportes que complementan los resultados obtenidos. De tal manera, los investigadores tendrán formas de acelerar el proceso y darle mayor profundidad debido al surgimiento de nuevas interrogantes, lo cual los hace más competentes. ...
Chapter
Full-text available
The article analyzes the issue of higher education students’ employability, taking on two constructs as a point of view: Transversal skills as requested by workplaces, and active learning methods, focusing on the value of incorporating project management as a teaching device by allowing students to prepare themselves for the labor market. The general question driving the article is: How can project management, precisely the PRINCE2 method, foster students’ employability by enabling their acquisition of transversal skills?
... The Delphi method has several advantages, given in the present study. These advantages include the following: participants can be found anywhere in the world, a quick consensus of those discussed is obtained, it covers a wide range of expertise, avoids long interpretations and discussions, the predicted questions are specific and have predetermined answers, and you get a final report agreed by all experts during the study (Linstone and Turoff et al., 1976;Gordon and Pease, 2006;Botterill and Platenkamp, 2014;Chalmers and Armour, 2019;Humphrey-Murto et al., 2020). Find involved Group 4 Questions regarding process innovation in the fields of activity A-S Examples: Do you appreciate the level of innovation in the field? ...
Article
Full-text available
Involvement in sustainable development is a voluntary activity. Organizations apply the principles of sustainable development only when they identify several benefits. These benefits are identified, especially with the financial ones. The involvement of organizations in sustainable organizations has different intensity levels. These intensity levels are influenced by psychosocial factors (PF), attitudes toward organizational risks, and organizational and urban policies. The present paper identifies the key psychological factors involved in applying organizational sustainability principles within organizations. For this research, five groups were created for in-depth interviews with key people from Romania’s innovative companies. To identify the importance of the framework analysis factors, the Delphi method was used, in which 20 experts from different fields of activity were involved. Following the rounds involved in the Delphi method, the ranking of PF on four levels of importance was accepted, based on planned behavior and reasoned action theory. These levels were correlated with the intensity levels of involvement in sustainable development. The entire market study was conducted during COVID-19, which significantly impacted specific directions. As a result, it could be observed that motivation, learning attitude, behavior, and intention to take precedence are essential in the organizational sustainability approach.
... Delphi modifications such as the number and timing of Delphi rounds: a series of rounds may be held over weeks or months or they may occur over a single day (ie a 'Delphi conference') ). There are also computer-based approaches that do without rounds altogether and enable varying modes of participation (Gordon & Pease, 2006;). One modification we made was to allow participants to respond either by online survey or via interviewer-guided survey by phone, Skype or face-to-face. ...
... It will, for example, be important to study participant experience of taking part in these exercises (e.g. the degree of participation intensity, motivation, engagement, sense of ownership and empowerment [131,132]), and to assess how far it is possible to reach agreements that stakeholders understand and accept. Evaluation might also examine the potential of using real-time Delphi to avoid having sequential rounds [133][134][135]. This might improve efficiency, reduce dropout, and minimise participant time, potentially without comprising user experience, inclusivity and robustness of consensus results [136] though the implications of such an approach for the ability of different groups, such as patients and carers, to participate would require careful evaluation. ...
Article
Full-text available
Background Practical methods for facilitating process improvement are needed to support high quality, safe care. How best to specify (identify and define) process improvements – the changes that need to be made in a healthcare process – remains a key question. Methods for doing so collaboratively, rapidly and remotely offer much potential, but are under-developed. We propose an approach for engaging diverse stakeholders remotely in a consensus-building exercise to help specify improvements in a healthcare process, and we illustrate the approach in a case study. Methods Organised in a five-step framework, our proposed approach is informed by a participatory ethos, crowdsourcing and consensus-building methods: (1) define scope and objective of the process improvement; (2) produce a draft or prototype of the proposed process improvement specification; (3) identify participant recruitment strategy; (4) design and conduct a remote consensus-building exercise; (5) produce a final specification of the process improvement in light of learning from the exercise. We tested the approach in a case study that sought to specify process improvements for the management of obstetric emergencies during the COVID-19 pandemic. We used a brief video showing a process for managing a post-partum haemorrhage in women with COVID-19 to elicit recommendations on how the process could be improved. Two Delphi rounds were then conducted to reach consensus. Results We gathered views from 105 participants, with a background in maternity care ( n = 36), infection prevention and control ( n = 17), or human factors ( n = 52). The participants initially generated 818 recommendations for how to improve the process illustrated in the video, which we synthesised into a set of 22 recommendations. The consensus-building exercise yielded a final set of 16 recommendations. These were used to inform the specification of process improvements for managing the obstetric emergency and develop supporting resources, including an updated video. Conclusions The proposed methodological approach enabled the expertise and ingenuity of diverse stakeholders to be captured and mobilised to specify process improvements in an area of pressing service need. This approach has the potential to address current challenges in process improvement, but will require further evaluation.
... obtained after different sequential rounds of feedback, in this case it is intended to obtain the agreement of the group in a given period of time, in which participants have access the application as frequently as is necessary, and as many times as desired either; as well as changing their minds as often as considered required (T. J. Gordon, 2009;T. Gordon & Pease, 2006;T. Gordon, 2008). ...
Conference Paper
Full-text available
In this paper, we present the research progress in the conception, development and validation of a Group-Spatial Decision Support System application (G-SDSS) named Geospatial System of Collective Intelligence (SIGIC for its acronym in Catalan and Spanish), aimed to support the decision-making process in geographic complex scenarios, mainly regarding the planning, organisation and/or use of resources in the territory, through modelling of spatio-temporal locations of goods, services and/or events derived from an interdisciplinary consensus of a group of experts. The conceptualisation of this system has allowed us to suggest the paradigm of Collective Spatial Analysis.
... Para el desarrollo de la presente investigación se ha optado por emplear la técnica Deplhi. Esta técnica permite recoger y decantar el conocimiento de un grupo de expertos sobre el tema complejo, obteniendo el máximo grado de consenso (Gordon & Pease, 2006). En el campo de la investigación educativa, encontramos un notable número de evidencias que muestran la funcionalidad y variedad en la aplicación del método Delphi 978-84-1377-360-5 (Green, 2014;García-Ruiz & Lena-Acebo, 2018). ...
... Over the history of the Delphi method variations have been extensive (Linstone & Turoff, 1975. With technological advancements, variations to the structure of Delphi approaches includes development of a Real Time Delphi that allows for multiple participants without the need to rely on a series of rounds (Gordon & Pease, 2006). Project BOHEMIA extends the Real Time Delphi by using an argumentative Delphi that is in real time and uses the review of the literature to allow participants to choose their reason for their quantitative score (as well the opportunity to add their own reason) (Gheorghiu et al., 2017). ...
Article
(Open Access https://doi.org/10.1016/j.futures.2021.102818 ). In this paper we present five scenarios that describe preferred futures for the design of compulsory schooling. The work addresses the counterfactual research question: What if compulsory schooling was a 21 st century invention? and is informed by the research in anticipatory systems thinking. The expert advice evolved over multiple rounds of input using a modified Delphi approach. The scenario development round followed a consensus and dissensus process in the early stages. The creation of a series of preferred futures for compulsory schooling can be used by education leaders in states, provinces, and countries to create momentum for strategic change in the static structure of schooling that generally maintains industrial models of schooling. The combining of a modified Delphi process with scenario development can be used to present possible futures which can be contextualised, divergent, and used as a catalyst to create other preferred scenarios for a future based on a success-for-all paradigm.
... ? To derive an objective, credible answer to this question, we develop Delphi-based scenarios (Nowack et al., 2011). We use a Delphi survey for data collection because this method allows the collection of both qualitative and quantitative input from a heterogeneous expert panel (Grisham, 2009). The survey itself was organized in a real-time format T. Gordon and Pease, 2006). The methodology has proven to enhance the validity, acceptance, plausibility and consistency of future-oriented studies by allowing experts to effectively discuss complex matters in a structured and anonymous group communication process (Belton et al., 2019;Linstone and Turoff, 2011). ...
Article
COVID-19 has had a tremendous impact on the world's society and economy. This also applies to European football; the continent's largest professional sports ecosystem has millions of employees in football-related jobs and even more people emotionally attached to the sport. To account for stakeholders' unprecedented level of uncertainty regarding regulatory, economic, social, and technological implications for their business, we systematically examine how COVID-19 will impact the future of the European football ecosystem in the short, medium and long term. We present a Delphi-based scenario analysis with 110 subject matter experts who assessed 15 future projections both on a quantitative and qualitative basis. We find that, for example, a salary cap for players would have the highest impact on the ecosystem, but it is unlikely to be implemented, while an increased awareness for social responsibility would be the most desirable effect of the crisis. To refine the results, we account for surface-level and deep-level characteristics of participants and find significant effects in both cases. We identify three different clusters of projections and discuss the potential threats and opportunities of COVID-19 for the European football ecosystem to contribute to the scientific discussion and to provide guidance for policy- and decision-makers.
... Analyzing a Word document, transferring the results into table format for analysis, and preparing the second round personalized for more than 40 panelists can be very time-consuming and prone to errors. As an alternative questionnaire format, we recommend considering the real-time Delphi (RTD) format [18] (using a platform) and/or sending an url. The RTD also has other advantages, such as immediate feedback. ...
Article
Full-text available
The Delphi technique is a suitable methodology for structuring group communication to answer current and prospective research questions within several rounds. The method is used in many disciplines and characterized by anonymity, iteration, controlled feedback, and statistical “group response” (Rowe & Wright, 2001). This technical paper presents practical details and lessons learned from a two-round Delphi-based scenario study in which projections (Delphi statements, questions or hypotheses) were developed with findings from expert interviews and an expert workshop. This Delphi study provides answers to future-related questions for which other research methods are inappropriate. This is depicted as a five-step process, making it easy to follow and replicable, for example to help first-time Delphi-method researchers. In doing so, the authors aim to provide the community with valuable technical insights and guidance for studies applying the Delphi technique both to prospective questions and in other research settings. •Conducting a Delphi study can be a slow process with respect to receiving feedback from the panelists. Planning an appropriate period for distributing the questionnaire may produce a higher return rate. A sufficient time buffer should be incorporated into project planning. •Projections that create dissent among the panelists may provide valuable results. •Data analytics, software programs and online social networks can support researchers, save time and resources, and provide further insights in the process of conducting a Delphi study.
... Different methods to assess and appraise risk have been developed (Covello and Merkhofer 1993;Horvath and Zuckerman 1993), especially related to technology (Lefley 1997; . Methods, such as RT Delphi (Gordon and Pease 2006) or costeffective models (Hubbard 2014), have been applied to the prioritization of technologies. However, these methods fall short when assessing the uncertainties of new technological developments. ...
Conference Paper
Full-text available
Energy Communities are finding their way into the local energy systems as new regulations surge. However, they often lack resources due to their limited size, and depend heavily on subsidies for providing competitive offerings. In parallel, new technologies support the development of smart services for the energy market and provide chances for increasing the competitiveness of energy communities. This paper utilizes the multi-criteria mapping (MCM) method to discuss with stakeholders from energy communities in Germany the relevance and priorities for realizing specific smart services. A general ranking, as well as four perspective-based rankings, are analyzed by discussing contrasts and uncertainties. The results provide relevant insights on potentials from each service and a basis for the design of new information systems and architectures for energy communities.
... The Delphi method is especially useful for long-range forecasting (20-30 years), as expert opinions are the only source of information available. During the last ten years, the Delphi method was used more often especially for national science and technology foresight (Gordon, T. & Pease, A., 2006). Since the Delphi method relies on expert opinion provided by panels, it is obviously important to select the right experts. ...
... Deplhi. Esta técnica permite recoger y decantar el conocimiento de un grupo de expertos sobre el tema complejo, obteniendo el máximo grado de consenso (Gordon & Pease, 2006). En el campo de la investigación educativa, encontramos un notable número de evidencias que muestran la funcionalidad y variedad en la aplicación del método Delphi 978-84-1377-360-5 (Green, 2014;García-Ruiz & Lena-Acebo, 2018). ...
... The Delphi Analysis summarizes the opinion on emerging concepts, and on the development studies that lack empirical data and intends to objectively obtain experts' responses (Gordon & Pease, 2006). This method usually considers several and consecutive response rounds (Rodrí guez- Mañas et al., 2013). ...
Article
Full-text available
The lack of cooperation between stakeholders and coordination of departments, the absence of systems interoperability, and the resistance to change by policymakers represent some of the challenges that make the implementation of the Smart City a complicated task. Nevertheless, the literature is lacking a broader comprehension of the Smart City barriers to enable policymakers to design and execute their strategies. Therefore, this paper aims to find the (critical) barriers that have been associated with having a significant influence on the success of a Smart City’s implementation and that are within the control of policymakers, based on the empirical knowledge and experience of experts. A study of the temporal aspect in which they can be overcome is also performed to allow better planning. Moreover, 50 barriers were aggregated and organized in eight distinct areas from a thematic analysis of an initial list of 114 barriers collected from the literature. Nine experts participated in the Delphi Analysis, which demonstrated that although Smart Cities are usually associated with technological and financial constraints, the 15 most critical barriers are mainly from the areas of Governance, Project, and Organization. The method combined a two-round survey with a focus group, using quantitative and qualitative approaches.
... Again, the questionnaires were personalized for each participant. Studies have shown that this method improves the decision quality of the Delphi method (Gordon and Pease, 2006). This second round lasted 8 days (from March 22 to 30, 2021), with a response rate of 100%. ...
Article
Given that the Chief Digital Office (CDO) is a new business position, it is not yet clear what the key functions they must develop to successfully undertake the company digital transformation process. Although previous literature has tentatively covered this issue, there are still gaps in knowledge about which functions are truly relevant. The aim of this paper is to fill this gap by (1) clarifying the key CDO functions through a Delphi study, validated by a digital dynamic capabilities (DDC) framework (Warner and Wäger 2019), and (2) presenting a measurement instrument useful for assessing the degree of the key CDO functions performance. The findings highlight that key CDO functions are those mainly related to the ability to scan the external environment for unexpected digital trends. In contrast, there is an interesting agreement not to consider CDO responsible for maintaining and advancing the current IT infrastructure and architecture. Our study contributes to understanding the role of a new emergent Csuite position that strategically integrates business and technological knowledge in successfully managing the digital transformation process.
... For this study, we used a real-time Delphi format, which calculates the group opinion instantly and displays it to the experts, which gives them the chance to reassess their answers immediately (Gnatzy et al., 2011;T. Gordon & Pease, 2006). Real-time formats are increasingly used as they deliver results similar to conventional Delphi studies while being more efficient (Aengenheyster et al., 2017;Di Zio et al., 2017;Geist, 2010). To enter the platform, experts received an individualized invitation link, with which they were able to reaccess the platform at any time, view t ...
Article
Full-text available
Organizations increasingly adopt technologies and logistics measures to improve their social sustainability, reduce transportation‐related carbon emissions, or even design their supply chains carbon negative. However, the speed at which this adoption is progressing is not fast enough to address the sustainability challenges of the world today. To gain deeper knowledge about the adoption process of sustainability‐related measures in road freight transportation, this paper examines barriers currently hindering a broader market penetration and derives expected timeframes when mass adoption of 14 relevant technologies will occur. Furthermore, the technologies' impacts on the economic, environmental, and social dimensions of sustainability are discussed providing recommendations, on which technologies the respective stakeholders might focus on in the future. Since we want to examines a long‐term technology perspective of the next 25 years, we rely on systematic foresight methodology. More specifically, we collect empirical data by use of a real‐time Delphi survey, which particularly suits complex and uncertain environments. Our global panel includes 116 experts from 25 different countries. The assessments are grouped into short‐term, midterm, midterm to long‐term, and long‐term developments. The categories shed light on potential drivers and barriers to the implementation of the surveyed technologies. The findings emphasize the need to systematically select suitable measures and promote those that are expected to deliver short‐term sustainability improvements to address current deficits in a timely manner. The article further outlines the crucial role of politics in developing necessary regulative frameworks to drive sustainability in road freight transportation.
... Anonymity during the survey minimizes the effect of being influenced by other participants, something which commonly happens in a group brainstorming session [65]. Nonetheless, as mentioned by Gordon and Pease, the objective of Delphi is not necessarily to obtain a consensus [66]. In addition to the convergence of opinions, the process of collecting responses can contribute insightful ideas to the main objective. ...
Article
Full-text available
Background The availability of underexploited agricultural residues in Thailand opens up the opportunity to supply second-generation bioethanol production. The national implementation of residues-to-biofuel can potentially boost the bioeconomy and greenhouse gas mitigation but requires the involvement of multiple stakeholders in the development of effective policy recommendations. This study aims to optimize the implementation of the national strategy through the use of a multi-criteria approach that involves participatory prioritization by current stakeholders in order to evaluate certain aspects and important indicators for second-generation bioethanol development. Methods The Delphi-AHP technique was used to analyze the degree of importance of different criteria. The evaluation process was conducted with various stakeholders and used a pairwise comparison of 4 dimensions (main criteria) and 12 indicators (sub-criteria). Participants were asked to rate factors related to technical feasibility, environmental impacts, economic feasibility and social impacts in terms of importance. Results Bioethanol stakeholders in Thailand from five different sectors (industry/business, NPO/NGOs, the governmental sector, academic/research institutes and financial institutions/banks) participated in the Delphi survey. The 20 experts’ evaluation of the four dimensions ranked economic feasibility (32.7%) highest in terms of level of importance, followed by environmental impacts (25.1%), technical feasibility (24.9%) and social impacts (17.3%). When assessing the sub-criteria, the participants selected ‘final price per liter’, ‘added value of input materials’ and ‘net energy balance’ as the top three most important indicators among the 12 sub-criteria. In terms of a link between the preferred criteria and the participants’ expertise, the results encouraged taking different backgrounds and affiliations into account in the policy planning phase. Conclusions The stakeholder survey indicated the importance of economic aspects, highlighting the need to take governmental driven policy into consideration. However, implementation scenarios have to be embedded in a broader range of aspects because all the dimensions were rated as being highly impactful. For future sustainable bioenergy, the inclusion of stakeholders’ opinions can result in multifaceted scenarios that can be linked to social acceptance and benefits for all relevant players when developing policy recommendations for advanced bioenergy.
... "Dicho método también mejora la eficiencia del proceso y optimiza el tiempo para la realización de este tipo de estudios" (Gordon y Pease, 2006), beneficios que tienen un efecto positivo a corto plazo, pues mejorar la eficiencia del proceso también abre el surgimiento de aportes que complementan los resultados obtenidos. De tal manera, los investigadores tendrán formas de acelerar el proceso y darle mayor profundidad debido al surgimiento de nuevas interrogantes, lo cual los hace más competentes. ...
Chapter
Full-text available
En la actualidad, los cambios en los ámbitos laborales a escala mundial requieren que los egresados universitarios sean competentes para atender las demandas actuales del trabajo. Instituciones educativas como la Universidad Autónoma del Noreste (Uane) implementan métodos de aprendizaje activo acorde con el contexto contemporáneo para fortalecer las competencias de empleabilidad. Esto implica orientar procesos educativos hacia aprendizajes que fortalezcan la empleabilidad, con un fuerte referente de conocimientos para aplicarlos a situaciones reales del trabajo y competencias suaves para la interacción y logro de resultados. En este sentido, es de gran valor para las empresas y organizaciones que los can¬didatos cuenten con competencias suaves, también conocidas como habilidades blandas, habilidades sociales, de comportamiento o habilidades interpersonales; también, ponen atención a la capacidad de liderar, de comunicarse de manera efectiva, de trabajar en equipo, de ser responsable, persuasivo y adaptable, de tener iniciativa y empatía, de gestionar información, y de desenvolverse y actuar acorde con las situaciones en contextos de complejidad e incertidumbre. Como parte de los procesos educativos curriculares, se encuentran las prácticas profesionales para fortalecer y desarrollar en el estudiante competencias para la empleabilidad. En este sentido, las preguntas que detonan la presente investigación son: ¿cómo se caracterizan las estrategias curriculares que promueven la empleabilidad en la Uane?, ¿cuál es el estatus de las estrategias curriculares 4td y de las prácticas profesionales sobre los resultados en los periodos de julio-diciembre 2019 y enero-junio 2020 con la pandemia de Covid-19? Método: El estudio es comparativo y descriptivo. La población de estudiantes en las cuatro técnicas didácticas (4td) en julio-diciembre 2019 fue de 969 y en enero-junio 2020 fue de 1420. La población de estudiantes en prácticas profesionales liberadas en julio-diciembre 2019 fue de 195 y en enero-junio 2020 fue de 153 estudiantes. La técnica para recolectar la información fue la encuesta. El desarrollo en el 2019 fue mediante el análisis de contenido en preguntas abiertas, frecuencias y estadística descriptiva en preguntas cerradas. En el 2020, el análisis de contenido ofreció la oportunidad para rediseñar el instrumento con preguntas con opciones de respuesta cerradas para analizarlas mediante la estadística descriptiva. Resultados: Los resultados de las competencias que desarrollan los estudiantes durante su trayectoria escolar mediante la estrategia curricular 4td y las prácticas profesionales son consistentes en ambos periodos 2019 y 2020. Conclusiones: La planeación, implementación y evaluación de 4td como estrategia curricular de empleabilidad, así como la integración de prácticas profesionales al currículo, representan oportunidades de habilitar en el estudiante competencias que le servirán como aprendizaje para la vida y la empleabilidad. Sin duda son cimientos para una mejor sociedad, por ello la importancia de la voluntad institucional en este tipo de acciones en beneficio de los estudiantes. Desde las competencias curriculares que promueven la empleabilidad en la Uane y dada la situación laboral a escala mundial que estamos viviendo con la pandemia de Covid-19, es necesario considerar el emprendedurismo como alternativa de trabajo y formar a los alumnos en métodos que les orienten para crear, implementar y distribuir productos o servicios, considerando herramientas digitales, y elementos de análisis financieros y de mercado. Otra consideración es incorporar la satisfacción de los empleadores como estudio longitudinal y que retroalimente las estrategias curriculares de la universidad. Palabras clave: Estrategias curriculares de empleabilidad, competencias suaves, métodos de aprendizaje activo, prácticas profesionales
... La literatura revisada indica que la técnica Delphi (Baena, 2004;Glenn, 2009;Bakule, et alt, 2016;Miklos y Tello, 2007;Gordon. y Pease, 2006) es la precursora en el desarrollo de pronósticos basados en técnicas cualitativas; fue desarrollada en 1951 por la Corporación RAND, para las fuerzas armadas de EEUU; en 1962 la información fue desclasificada y publicada por sus creadores Dalkey y Helmer bajo el título: "Una aplicación experimental del Método Delphi para uso de los expe ...
Conference Paper
Full-text available
La presente investigación tuvo como objetivo determinar el nivel de percepción de Malestares Musculoesqueléticos aspirantes a soldados del Ejército pertenecientes a la selección fútbol. La fundamentación teórica se sustenta en (Pezzotta, Querques, Pecorelli, Nani y Sironi, 2017), (Kilic, Goedhart, Hägglund y col, 2017), Toro, Martínez y Díaz (2016), Camargo y Herrera (2016), (Takata, Nakase, Inaki, Mochizuki y col, 2016), (Carrasquero, 2015), (Mendinueta y Herazo, 2005) Metodológicamente la investigación orientó bajo un enfoque empirista-positivista, aplicada y cuantitativa, con un diseño no experimental, transeccional–descriptivo de campo. La población fue de 44 jugadores de fútbol, con un muestreo censal. La recopilación de la información se realizó mediante un cuestionario de percepción de malestares musculoesqueléticos Cornell versión española (Carrasquero, 2015), con 75 ítems con escala de actitudes lickert y 1 dicotómica; La confiabilidad fue calculada a través del alpha cronbach, r=0,86. Los resultados para la frecuencia del malestar demuestran que las secciones inferiores corporales tienen un nivel moderadamente alto para: muslos, rodillas, pantorrillas y pies. La severidad presento que el plano más afectado fue el inferior, con un nivel de incomodidad moderadamente bajo. El indicador productividad reporta que no presentan molestias que afectaran el entrenamiento. En términos generales se concluye que el nivel de percepción de los Malestares Musculoesqueléticos, se localiza en el plano sagital inferior y en los segmentos proximales y distales. Coincidiendo con los reportados por los antecedentes y reportes científicos previos a esta investigación.
... The DELPHI technique is a qualitative sociological consensus method, belonging to the in-depth interview group, allowing the collection of subjective information based on the criteria and opinions of a group of experts. It is aimed at understanding informants' perspectives on a research topic, without the need to generate direct interaction between its components within a physical space [45,46,47,48,49,50]. ...
Article
Full-text available
Among the factors that influence the quality of life of people with Autism Spectrum Disorder (ASD), physical activity and sport are key dimensions of physical well-being. Few studies take into account the perspectives of people with ASD in order to understand the extent of physical well-being and their subjective perception of it. The development of a system of quality of life indicators related to physical activity and sport for people with ASD and their analysis is the aim of this study, providing guidelines for improvement. A study was carried out with a sample of 276 people, professionals (n=143), family members (n=73) and people with ASD (n=60). The people with ASD in the sample belonged to three age categories: children aged 8-12 years, adolescents aged 12-18 years and adults over 18 years. The research team used a qualitative methodology in the collection of information. They adapted three questionnaires, with validated quality of life scales, to guide the interviews and also applied them in the design of the focus group protocols. Subsequently, the research team analysed the information collected in the focus groups with families and professionals using a DELPHI method. A system for coding the responses and qualitative analysis of the responses was also prepared for the analysis of the data by the research team. Finally, the information obtained was compared with a group of representative experts. The results concluded with the elaboration of a system of quality of life indicators related to the dimension of physical well-being, as well as guidelines and proposals that bring together the perspectives of people with ASD in relation to the practice of sport and physical activity. There is a need to increase the practice of sport among people with ASD in order to promote their health, social participation and personal satisfaction. It is concluded that it is not possible to obtain a broad picture of the quality of life of people with ASD and their families due to lack of information. However, the method and the results obtained represent a first approach at national level to increase knowledge about the quality of life of people with ASD.
Article
Background: Given substance use disorders (SUDs) among people with HIV are highly prevalent, integrating SUD services within HIV service settings is needed to help end the HIV epidemic. In this study, we assessed the setting-intervention fit (SIF) of 9 evidence-based SUD interventions: acamprosate, disulfiram, oral naltrexone, injectable naltrexone, oral buprenorphine, injectable buprenorphine, contingency management, motivational interviewing, and cognitive behavioral therapy (CBT). Setting: Clinical and nonclinical HIV service organizations (HSOs) in the United States. Methods: In May 2020, a stakeholder-engaged real-time Delphi was completed with 202 HSOs. HSO respondents rated the extent to which each SUD intervention was fundable, implementable, retainable, sustainable, scalable, and timely for their HSO, and these 6 items were summed into an SIF score (possible range of 0-18). Results: Motivational interviewing had the highest average SIF score (11.42), with SIF scores above the midpoint (9.5) for clinical (11.51) and nonclinical HSOs (11.36). For nonclinical HSOs, none of the other interventions were above the midpoint. For clinical HSOs, the average SIF scores were above the midpoint for CBT (10.97) and oral buprenorphine (9.51). Multivariate regression analyses, which controlled for characteristics of the HSO respondent, revealed geographic region of the United States and whether the HSO currently offered any substance use services as 2 of the best predictors of SIF scores. Conclusions: Notwithstanding the need to improve the SIF for the other evidence-based SUD interventions, motivational interviewing, CBT, and oral buprenorphine are currently the evidence-based SUD interventions with greatest perceived fit for integration within HSOs in the United States.
Chapter
Additive Manufacturing (AM) is presented as a set of alternative technologies to conventional manufacture. AM can cause profound changes in the industry, the economy and the society of the future. These changes will affect in different de-grees, all aspects that make up the current manufacturing landscape. The Delphi method is a technique that allows collecting the opinion of a group of individuals (experts) using repeated inquiries. This method is used, among other cases, when attempting to have a vision about future events. The main objective of this work is to analyze the responses to a Delphi inquiry addressed to Spanish experts from the professional and university field committed to the manufacturing and engi-neering sector. The consensual answers to the issues this consultation raises can provide a horizon indicating what changes the Additive Manufacturing will bring to the manufacture scene. The issues that arise revolve around groups of technol-ogies to be established in the future, to the different business models that will be used, and the standards and regulations that will be required.
Article
Full-text available
Developing collaborative and cooperative research across academic disciplines and university administrative boundaries can be a challenge. In an attempt to understand and propose solutions to this challenge, the authors of this paper set out to: test an innovative combination of methods to generate and evaluate ideas and strategies; and to write about the findings using collaborative online methods. During this process universities in the UK moved to online working and so the authors completed this paper through entirely online means. The authors - a team of academic researchers from the University of AAA - came together in sessions designed as a hybrid of World Café and Delphi technique approaches to discuss challenges and solutions. The findings were written up drawing on insights from the use of massively authored papers (also known as ‘massively open online papers’, MOOPs), and online tools to enable remote collaboration. Expert consensus was sought in this project within a group of participants (N = 7) in one university setting to create a MOOP. This paper presents details of the process, the findings, and reflections on this collaborative and cooperative exercise. That this paper was written using the methods discussed within it, highlights the value and success of the approach. In light of the current Coronavirus pandemic and the increased need to work remotely, this paper offers academics useful strategies for meaningful and productive online collaboration
Article
Strategy tools and frameworks are crucial for managers to navigate their business environment and formulate strategies. Extant research has focused on the characteristics, dimensions, applications, and impact of traditional tools. However, there are questions regarding the suitability of these tools to the increasingly dynamic environments faced by strategy practitioners characterized by blurring industry boundaries, uncertainty, and ambiguity. Using an expert-panel approach, we address this research gap by investigating how strategy experts from practice and academia assess established strategy tools in dynamic environment. We identify the characteristics of strategy tools that experts value in such contexts and which can inform future development of context-specific strategy tools. Additionally, we also investigate why experts select and apply specific tools and how they combine these tools. Our findings further allow us to explore the difference in perspectives of strategy scholars and practitioners, which is necessary to reconcile the gap between strategy theory and practice. Finally, we discuss implications of the study for strategy and management research, education, and practice.
Article
Full-text available
RESUMEN El envejecimiento de las personas con discapacidad intelectual es una cuestión que en los últimos años está afectando de manera preocupante a estas personas, deteriorando su calidad de vida y, en el caso de aquellos que trabajan, presentando nuevos retos a los centros laborales, que ven la necesidad de tomar medidas para atender a esta nueva situación desde diferentes ámbitos. A partir de estas premisas, se diseña una investigación, promo-vida por la Universidad de Burgos, Plena Inclusión Castilla y León y la Gerencia de Servicios Sociales de Castilla y León, con el objetivo de estudiar la relación que existe entre el proceso de envejecimiento en los trabajadores con discapacidad intelectual y su calidad de vida, diseñando un modelo de intervención, a partir de un "Protocolo de medidas e indicadores", que ayuden a concretar y definir los apoyos que han de prestar los profesionales y las organizaciones en esta etapa vital. Se utiliza una metodología cualitativa de tipo descriptiva interpretativa a partir de la técnica Delphi, con la participación de 22 expertos provenientes de diferentes ámbitos del empleo y la discapacidad intelectual. La investigación se complementa con un grupo focal formado por 8 personas vincu-ladas al ámbito de la gestión, con el fin de proporcionar una visión estratégica global y valorar sus posibilidades
Thesis
Warning in advance of potential threats allows decision-makers to analyse potential impacts, design policies and practices for better preparedness and act strategically to maximise benefits or minimise costs. Early warning systems are of increasing importance in ecological processes since human interference has altered the pace of environmental change. This is particularly important in forestry where production periods are long. The aim of this study was to find novel threats of future concern to European forestry that need careful observation in order to enable timely action. Long before its actual occurrence a potential crisis sends off a repeated and persistent trail of early warning signals. Monitoring these weak signals is considered an appropriate effort to possible future issues with potential high impact, that are unlikely to occur. For this study, a tailored version of the foresight method Delphi was selected. The two round Delphi method was applied as a scanning tool using a diverse international group of 23 European forestry experts. 26 threats of future concern to European forestry were identified. Six of these were considered as novel threats, for usage in an early warning system. The identified weak signals connected to these threats were mostly signals of the past that cannot be used to monitor early warning signals, here further research and training is recommended. Decision-makers can still use the issues identified by this research to further improve the decision-making process.
Article
The Delphi approach is a central method in technological forecasting. It has been used in thousands of scholarly works and extensively outside academia, often in relatively narrowly scoped case studies. With the notable exception of Japan and a few other countries, broad Delphi studies run in the context of national-level policy planning are seldom seen. This study presents results on implementing the long-running Japanese Science and Technology Prediction Delphi survey developed by the National Institute of Science and Technology Policy (NISTEP) in Finland. This study reports the implementation of the 702 statement Delphi and on the reflections of the expert opinion study against quantitative data on Finnish research capabilities. The differences in expert opinion are discussed between Japanese and Finnish experts and the importance is highlighted that contextual factors play in responses received from the experts even on extremely global issues, such as the development of science and technology. It is proposed that quantitative data can serve as an important contextualization tool for the expert opinion results.
Article
This paper presents a research study that is a valuable first project in setting the stage for bringing a foresight perspective to the impacts of severe flooding on health, poverty, and equity in Houston, out to the year 2050. Drawing on qualitative and quantitative research gathered from methodologies such as the Delphi and Cross-Impact Analysis combined with insights gained from interviews with subject-matter experts, it explores four alternative, plausible scenarios for Houston’s long-term future. Exploring scenarios that feature the continuation of present-day trends, a system collapse, a system balanced by a new equilibrium, and a system transformation provides an opportunity for stakeholders and policymakers to work together to create a more inclusive future for all city residents.
Article
Financial innovation and technological advances are growing at a pace unrivalled by any other period in history. However, as more stakeholders enter these markets, criminals are exploiting their inadvertent security deficiencies to launder illicit funds or finance terrorism. This three-round policy Delphi study involved consultations with 52 experts from different industries and countries to understand future risk-prone technological developments, possible prevention measures and relevant stakeholders. Results highlight a range of money laundering and terrorist financing risks being enabled by advances in distributed ledger technologies (predominantly through cryptocurrencies), new payment methods and financial technology (FinTech). These threats include privacy-enhanced cryptoassets, transaction laundering, e-currencies and digital-only financial services. Findings also suggest that detection-based countermeasures (currently the primary preventative approach) can be coupled with more diverse countermeasures to increase effectiveness. However, the unique circumstances and constraints specific to different stakeholders will affect the nature, utility, and extent to which they can implement certain countermeasures. As such, a ‘one-size-fits-all’ approach to prevention is undesirable. Drawing on expert insight from the study, we propose a framework and a 3-point standard of implementation to motivate cost-effective, user-friendly, and innovation-friendly measures to improve suspicious activity detection and futureproof technologies before their criminal exploitation becomes mainstream.
Article
Objective: In recent years, there are several systematic reviews published on animal experiments of Traditional Chinese medicine (TCM). PRISMA (preferred reporting items for systematic reviews and meta-analysis) guidelines provide a guarantee for significantly improving the reporting quality of systematic reviews (SRs) and meta-analysis (MAs) to a certain extent; however, there are still certain defects found in the quality of SRs/MAs of animal experiments of TCM. It has been found that especially, the descriptions of the rationale and animal characteristics of TCM interventions are inadequate. As a result, we have developed a novel reporting guideline for SRs/MAs of animal experimental in the field of TCM (PRISMA-ATCM) to overcome these problems. Methods: PRISMA-ATCM reporting guidelines were formed by analyzing both the status and quality of published SRs/MAs of animal experiments and consulting experts in the related fields, and then by Delphi consultation, consensus meeting and revision. Results: Among the 27 items on the PRISMA checklist, Title (1), Structured summary (2), Rationale (3), Objectives (4), Protocol and registration (5), Eligibility criteria (6), Data items (11), Planned methods of analysis (14), Study characteristics (18), Summary of evidence (24), Limitations (25), and Funding (27) have been extensively revised and expanded, to specifically include the details about TCM intervention and animal characteristics. In addition, illustrative examples and explanations have been provided for each item. Conclusion: PRISMA-ATCM could markedly improve the quality SRs/MAs of animal experiments in the field of TCM.
Article
The purpose of this study was to create concrete, practical recommendations for remediation with students struggling to effectively counsel members of the LGBTQ+ populations as a result of faith‐based values conflicts that are both effective and ethical. A panel of seven experts in counselor education participated in this Delphi study.
Article
Full-text available
Delphi is a scientific method to organize and structure an expert discussion aiming to generate insights on controversial topics with limited information. The technique has seen a rise in publication frequency in various disciplines, especially over the past decades. In April 2021, the term Delphi method yielded 28,200 search hits in Google Scholar for the past five years alone. Given the increasing level of uncertainty caused by rapid technological and social change around the globe, collective expert opinions and assessments are likely to gain even more importance. Therefore, the paper at hand presents technical recommendations derived from a Delphi study that was conducted amid the outbreak of the COVID-19 pandemic in 2020. ● The paper comprehensively demonstrates how to prepare, conduct, and analyze a Delphi study. In this regard, it combines several methodological advancements of the recent past (e.g., dissent analyses, scenario analyses) with state-of-the-art impulses from other disciplines like strategic management (e.g., fuzzy clustering), psychology (e.g., sentiment analyses), or clinical trials (e.g., consensus measurement). ● By offering insights on the variety of possibilities to exploit Delphi-based data, we aim to support researchers across all disciplines in conducting Delphi studies and potentially expand and improve the method's field of application.
Chapter
Although the Delphi method was designed as a forecasting tool for the RAND Corporation in the 1950s, in the last several decades, this research methodology is commonly used for facilitating consensus in many fields such as business, education, and nursing. Because of the increased use of the Delphi method, more information is needed for researchers to precisely execute a successful Delphi study. This chapter briefly introduces the Delphi method, reviews the methodology, discusses types and variations in Delphi studies, addresses the advantages and limitations, and provides clear, step-by-step guidelines for employing a Delphi method research study.
Article
Background: The National Academies of Sciences, Engineering, and Medicine 2016 trauma system report recommended a National Trauma Research Action Plan (NTRAP) to strengthen and guide future trauma research. To address this recommendation, 11 expert panels completed a Delphi survey process to create a comprehensive research agenda, spanning the continuum of trauma care. We describe the gap analysis and high priority research questions generated from the NTRAP panel on prehospital and mass casualty trauma care. Methods: We recruited interdisciplinary national experts to identify gaps in the prehospital and mass casualty trauma evidence base and generate prioritized research questions using a consensus-driven Delphi survey approach. We included military and civilian representatives. Panelists were encouraged to use the PICO (Patient/Population, Intervention, Compare/Control, and Outcome) format to generate research questions. We conducted four Delphi rounds in which participants generated key research questions and then prioritized the questions on a 9-point Likert scale to low, medium, and high priority items. We defined consensus as ≥60% agreement on the priority category and coded research questions using a taxonomy of 118 research concepts in 9 categories. Results: 31 interdisciplinary subject matter experts generated 490 research questions, of which 433 (88%) reached consensus on priority. The rankings of the 433 questions were: 81 (19%) high priority, 339 (78%) medium priority, and 13 (3%) low priority. Among the 81 high priority questions, there were 46 taxonomy concepts, including: health systems of care (36 questions), interventional clinical trials and comparative effectiveness (32 questions), mortality as an outcome (30 questions); prehospital time/transport mode/level of responder (24 questions), system benchmarks (17 questions), and fluid/blood product resuscitation (17 questions).Conclusions This Delphi gap analysis of prehospital and mass casualty care identified 81 high priority research questions to guide investigators and funding agencies for future trauma research. Level of evidence: not applicable.
Chapter
The paper focuses on long-term strategies to apply concepts of economic resilience in a practical context of Bavarian tourism regions. Based on the case study of the Ingolstadt region and tourism destinations in Bavaria (Germany), qualitative empirical methods are used to look at the specific barriers to implement resilience strategies on the regional level. Implications of transferring resilience understanding to challenges of tourism regions, the role of destination managers and organizational preconditions to introduce destination resilience are discussed to emphasise the value of the concept to regional practitioners.
Article
Our situation As pharmacy educators, we often encounter situations such as designing new curricula or establishing shared values for an organization that may be ambiguous or controversial. To generate effective solutions, it is often necessary to build group consensus with key stakeholders. The purpose of this paper is to describe and provide recommendations for using the Delphi method, a process for facilitating discussions and aiding in decision making. Methodological literature review An overview of the Delphi method, including its multiple variations, is presented. Steps necessary to complete a Delphi study (building a protocol, developing a research question, defining panelists and panel size, piloting the protocol, round one item creation and analysis, round two and beyond, consensus, increasing clarity, and reporting) is described. Our recommendations and their application(s) Practical recommendations are provided to support use the Delphi method to build consensus in research. These recommendations include: (1) clarify the purpose of the Delphi, (2) ensure the research questions are grounded in the literature and are relevant, (3) carefully consider panelist processes, (4) determine any definitions that should be given to or developed by panelists, (5) determine methods for enhancing clarity, and (6) employ methods to reduce attrition. Potential impact The Delphi method provides a systematic approach to generating consensus in pharmacy education for commonly encountered situations such as committee meetings, research studies, faculty retreats, classroom activities, and lab meetings.
Article
Rapid technological progress and digitalization have considerably changed the role of technology in sports in the past two decades. As the human limits of performance have been reached in many disciplines, reaching future limits will increasingly depend on technology. While this represents progress in how athletes train and compete, similar developments await sports managers in the way they lead sports organizations and sports consumers in the way they consume and engage with sports. Using the SportsTech Matrix (i.e., a framework to capture how different types of technologies provide solutions to different user groups in sports), we examine how technology will impact sports in the future. We present a Delphi-based prospective study with quantitative and qualitative assessments from 92 subject matter experts for six future projections and 35 non-Delphi prospective survey items. We find that, by 2030, technology will significantly impact all three user groups in sports: athletes, consumers, and managers. For athletes, experts anticipate technology to play a major role for sporting performance improvements. For consumers, the consumption of sports content will continue to change significantly. For management, new types of manager profiles in terms of backgrounds and skill sets would be desirable. We discuss two possible future scenarios: (1) a probable future and (2) a game changer. Our findings should provide relevant insights for decision-makers and other stakeholders in sports and raise promising directions for future research.
Conference Paper
Full-text available
The Suggested Upper Merged Ontology (SUMO) is an upper level ontology that has been proposed as a starter document for The Standard Upper Ontology Working Group, an IEEE-sanctioned working group of collaborators from the fields of engineering, philosophy, and information science. The SUMO provides definitions for general-purpose terms and acts as a foundation for more specific domain ontologies. In this paper we outline the strategy used to create the current version of the SUMO, discuss some of the challenges that we faced in constructing the ontology, and describe in detail its most general concepts and the relations between them.
Article
Full-text available
Abstract: We discuss the development of an environment for formal knowledge engineering. The Sigma system is an advance over previously developed systems in that it integrates a number of modern ontology development tools, which has motivated a number of research issues. Primary components include an ontology browsing and editing environment, a first order logic inference system and a natural language to logic translator. Although largely independent of any particular ontology, it supports a number of publicly available formal ontologies.
Article
There are a number of differences in procedures when a large scale Delphi is undertaken. Some of the most important differences are discussed in the light of Delphi embracing a panel of 385 members responding to a study sponsored by the National Center for Higher Education Management Systems (NCHEMS) at the Western Interstate Commission for Higher Education (WICHE) in 1972. Such topics as panel fatigue, panel consister cy, the consensus phenomena, and early plurality are considered in the report.
Article
The preparation of research and technology development (RTD) programmes calls for the analysis of prospective technological advances and the consideration of ''lessons learned'' from earlier programs. In this paper, we report experiences from a pilot project in which an internet-based survey and five participatory workshops were organised in conjunction with a Finnish RTD program in telecommunications. At these workshops, leading representatives from industry and the research community were invited to report their experiences from a concluding RTD program and to address future research needs, whereby they were assisted by a group support system (GSS). The workshop process was well received, which suggests that the preparation of RTD efforts can benefit from the deployment of GSS tools in support of face-to-face discussions. To our knowledge, this is the first account on GSS use in the foresight context.
Conference Paper
Ontologies provide advantages of knowledge reusability, sharing, and greater robustness when used to build large knowledge-based applications. Unfortunately, translating between English statements and a specific ontology requires skill in knowledge engineering and an understanding of formal logic and the ontology itself. A knowledge engineer must be familiar with the concepts in the ontology, the fine distinctions between terms, and the specific way the ontology conceptualizes the world. We have developed a tool, CELT (controlled English to logic translation), to enable nonprogrammers to add knowledge expressed in terms of an ontology. CELT is an automatic translation tool to convert controlled English to KIF formulas using ontologies built with the suggested upper merged ontology (SUMO). Wordnet provides a base lexicon and a default preference for word senses. We do not expect CELT to obviate the need for knowledge engineers but to instead better leverage their time, as current machine translation tools assist professional human translators.
bComputer Based Delphi ProcessesQ an invited chapter Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health
  • Murray Turoff
  • Starr Roxanne Hiltz
Murray Turoff, Starr Roxanne Hiltz, bComputer Based Delphi ProcessesQ an invited chapter, in: Adler Michael, Ziglio Erio (Eds.), Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, Kingsley publishers, London, 1996.
The Delphi Method, Futures Research Methodology V2, CD ROM, The Millennium Project
  • T Gordon
T. Gordon, The Delphi Method, Futures Research Methodology V2, CD ROM, The Millennium Project, American Council for the United Nations University, 2003.
Issues in large scale Delphi studies
  • Huckfeldt