No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
RT Delphi: An efficient, “round-less” almost real time Delphi method
Abstract
The authors have recently developed a new approach to performing a Delphi study that does not involve the use of sequential “rounds” and as a result, greatly improves the efficiency of the process and shortens the time to perform such studies. This paper describes this process, RT Delphi, and illustrates its use in a decision-making application drawn from the Millennium Project of the American Council for the United Nations University. The illustrative application involves setting priorities among strategies for dealing with anticipated terrorist activities that might be initiated by a single deranged individual.
... Real-Time Delphi allows an interactive and around-the-clock alternative to traditional Delphi in an online environment (Gordon and Pease, 2006). Real-Time Delphi has the advantage of accelerating and enhancing decision making, as the process is not interrupted to analyze intermediate results (Gordon and Pease, 2006;von der Gracht et al., 2011). ...
... Real-Time Delphi allows an interactive and around-the-clock alternative to traditional Delphi in an online environment (Gordon and Pease, 2006). Real-Time Delphi has the advantage of accelerating and enhancing decision making, as the process is not interrupted to analyze intermediate results (Gordon and Pease, 2006;von der Gracht et al., 2011). Another strength of Real-Time Delphi is its interactivity, as feedback to an expert panel that participates anonymously is direct, as is the opportunity for experts to debate with each other (Gordon and Pease, 2006). ...
... Real-Time Delphi has the advantage of accelerating and enhancing decision making, as the process is not interrupted to analyze intermediate results (Gordon and Pease, 2006;von der Gracht et al., 2011). Another strength of Real-Time Delphi is its interactivity, as feedback to an expert panel that participates anonymously is direct, as is the opportunity for experts to debate with each other (Gordon and Pease, 2006). Lastly, Real-Time Delphi simulates a hypothetical decision-maker to whom relevant information influencing decisionmaking is provided (Gordon and Pease, 2006). ...
Weak signals, the first signs of a potentially emerging issue that could disrupt current trends and megatrends, are hidden among disconnected pieces of information. Analyzing weak signals can help decision makers generate preparedness and a rapid-response capacity for possible systemic changes in the forest bioeconomy, where weak signals have not been studied extensively. This exploratory study sought to understand how to identify and interpret weak signals of change that may have an impact on the forest bioeconomy. An international panel of experts in the forest bioeconomy and in foresight provided their views through a two-stage Real-Time Delphi method which utilized the multiple rounds of traditional Delphi and the instantaneous feedback of the Real-Time Delphi method. This study introduces four conceptual frameworks to use when searching for weak signals: Internal/external operating environment, the Multi-level perspective, the Three Horizons Framework, and Complexity theory. These conceptual frameworks were applied in the analysis of collected data with the aim of developing the theory and practice on how to identify and interpret weak signals. According to the results, observations beyond the traditional dominating operating environment and a combination of observations from various sources could help to identify weak signals that may affect the forest bioeconomy. The Real-Time Delphi insights suggest that weak signal analysis becomes stronger via collaboration and interaction between futures specialists and forest bioeconomy experts.
© 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
... In recent years, the concept of a 'Real-Time' Delphi (RTD) has emerged. The functionality of a RTD can potentially improve the shortfalls of engaging in a long Delphi survey process with multiple rounds [6,7]. Some groups developing RTD functionality describe the group response feedback as being available when the participant revisits the survey [7]. ...
... The functionality of a RTD can potentially improve the shortfalls of engaging in a long Delphi survey process with multiple rounds [6,7]. Some groups developing RTD functionality describe the group response feedback as being available when the participant revisits the survey [7]. In contrast, others describe feedback, in the form of group responses, as being provided immediately after a participant has answered a question [8]. ...
... At this point, the participant can change their rating or keep it the same before moving on to rate the next outcome. As in Gordon and Peases' Fig. 1 Timeline of randomised trial model [7], participants can revisit and re-rate outcomes as often as they wish during the survey period. ...
Background
Delphi surveys are commonly used to prioritise critical outcomes in core outcome set (COS) development. This trial aims to compare a three-round (Multi-Round) Delphi (MRD) with a Real-Time Delphi (RTD) in the prioritisation of outcomes for inclusion in a COS for neonatal encephalopathy treatments and explore whether ‘feedback’, ‘iteration’, and ‘initial condition’ effects may occur in the two survey methods.
Methods
We recruited 269 participants (parents/caregivers, healthcare providers and researchers/academics) of which 222 were randomised to either the MRD or the RTD. We investigated the outcomes prioritised in each survey and the ‘feedback’, ‘iteration’, and ‘initial condition’ effects to identify differences between the two survey methods.
Results
In the RTD, n = 92 participants (83%) fully completed the survey. In the MRD, n = 60 participants (54%) completed all three rounds. Of the 92 outcomes presented, 26 (28%) were prioritised differently between the RTD and MRD. Significantly fewer participants amended their scores when shown stakeholder responses in the RTD compared to the MRD (‘feedback effect’). The ‘iteration effect’ analysis found most experts appeared satisfied with their initial ratings in the RTD and did not amend their scores following stakeholder response feedback. Where they did amend their scores, ratings were amended substantially, suggesting greater convergence. Variance in scores reduced with subsequent rounds of the MRD (‘iteration effect’). Whilst most participants did not change their initial scores in the RTD, of those that did, later recruits tended to align their final score more closely to the group mean final score than earlier recruits (an ‘initial condition’ effect).
Conclusion
The feedback effect differed between the two Delphi methods but the magnitude of this difference was small and likely due to the large number of observations rather than because of a meaningfully large difference. It did not appear to be advantageous to require participants to engage in three rounds of a survey due to the low change in scores. Larger drop-out through successive rounds in the MRD, together with a lesser convergence of scores and longer time to completion, indicate considerable benefits of the RTD approach.
Trial registration
NCT04471103. Registered on 14 July 2020.
... It provides the benefit of allowing the experts to discuss and communicate with each other in a structured setting, making the plurality of opinions more visible in the process (Landeta and Barrutia, 2011;Story et al., 2000). In fact, scholars have suggested that investigating the underlying arguments in case of substantial dissent in experts' views is more insightful than striving for consensus (Gordon and Pease, 2006;Story et al., 2000). ...
... In particular, we employ an online, real-time variation of the Delphi method (Gnatzy et al., 2011;Gordon and Pease, 2006). Instead of explicit rounds, this provides instantaneous feedback to the participants on the aggregated quantitative assessments and qualitative views of the experts who have participated so far. ...
... Reliability and validity of Delphi studies are strongly influenced by the selection and participation of appropriate experts (Tersine and Riggs, 1976;Welty, 1972). Given the nature of the study, we explicitly sought the views of practitioners with significant experience in strategy to build an understanding of how they assess established strategy tools (Gordon and Pease, 2006;. This included industry experts who have worked for at least 10 years in the strategy domain. ...
Strategy practitioners and executives face increasingly dynamic environments in which their firms operate. In pursuit of better company performance and competitive advantages in such environments, the onus is on them to make better strategic decisions. To this end, it is critical that they achieve a fit between their information requirements and their information processing capacities. On one hand, the dynamics in their business environments are resulting in exacerbated information requirements. On the other hand, we are now in an information age with an endless flow of data inputs from various sources. Against this background, this dissertation examines how executives and strategy practitioners faced with such challenging circumstances can achieve a fit between their increasing information requirements and their growing difficulty in identifying and analyzing relevant information. The thesis suggests that this fit can be achieved by developing a Decision Intelligence approach. The individual articles comprising this work provide an overview and then delve deeper into individual core elements of a Decision Intelligence approach. In addition to the overall approach, this entails how executives gather, filter, interpret, and synthesize information inputs from multiple sources, as well as innovating which strategy tools and frameworks they leverage to gain a contextual understanding of their environments and for processing decisions-relevant information. Further, the last article examines how executives and strategy practitioners can innovate their decision models to develop business models which can lead to competitive advantages for their companies.
... In 2006, Gordon & Pease introduced the Real-Time Delphi (RTD) as an alternative to multi-round Delphi surveys, aiming to improve the process's efficiency and shorten the required time. In this method, participants can view the group's responses for each question in real time and the total number of responses simultaneously [43]. RTD is round-less; however, in light of changing feedback, participants can be recommended to revisit and re-rate questions. ...
... The number of papers using RTD is increasing. The RTD has been used to develop consensus across a range of settings, including economics, [43,44] transportation [44], education [45,46], religion [47], professional development [48], and sustainability [49]. RTD has been used in healthcare studies that aim to develop a medication adherence technologies repository [50] and determine the negative impacts of substance use disorders among people with HIV in the United States [51]. ...
Neonatal sepsis is a serious public health problem; however, there is substantial heterogeneity in the outcomes measured and reported in research evaluating the effectiveness of the treatments. Therefore, we aim to develop a Core Outcome Set (COS) for studies evaluating the effectiveness of treatments for neonatal sepsis. Since a systematic review of key outcomes from randomised trials of therapeutic interventions in neonatal sepsis was published recently, we will complement this with a qualitative systematic review of the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers. We will interpret the outcomes of both studies using a previously established framework. Stakeholders across three different groups i.e., (1) researchers, (2) healthcare providers, and (3) patients’ parents/family members and parent representatives will rate the importance of the outcomes in an online Real-Time Delphi Survey. Afterwards, consensus meetings will be held to agree on the final COS through online discussions with key stakeholders. This COS is expected to minimize outcome heterogeneity in measurements and publications, improve comparability and synthesis, and decrease research waste.
... So, sampling strategies could easily be studied within the SGRP by, for instance, using an information board (e.g., Mouselab: Payne et al., 1993) to record the sequence in which judges search for information and/or what sources they choose to consult in what order (perhaps with costs to searching imposed). Another obvious manipulation would be to compare sequential information presentation-as can be the case in 'real time Delphi' (Gordon & Pease, 2006), where judges can post their judgments and see those judgments already posted, whenever they like-with parallel presentation, as in a traditional Delphi where panelists can only post and review judgments once per round. The sequence in which information is received is not just an issue in evaluating different forms of Delphi, but also in JASs, for example in the work of DeKay (2015) discussed above, on how decisions can affect the evaluation of subsequent information. ...
... Investigate Discuss Estimate Aggregate (IDEA) is very similar to the NGT except that the discussion is usually performed online and may not be completely free 25 (i.e., in IDEA's original formulation its main purpose is simply to ensure that all participants have a similar understanding of the problem)-the final aggregation is weighted by performance of each participating expert on a pre-test. Some forms of Delphi, such as real-time Delphi (RTD: Gordon & Pease, 2006) also have less strictly controlled interaction between participants than classic Delphi. ...
Although the individual has been the focus of most research into judgment and decision-making (JDM), important decisions in the real world are often made collectively rather than individually, a tendency that has increased in recent times with the opportunities for easy information exchange through the Internet. From this perspective, JDM research that factors in this social context has increased generalizability and mundane realism relative to that which ignores it. We delineate a problem-space for research within which we locate protocols that are used to study or support collective JDM, identify a common research question posed by all of these protocols—‘ What are the factors leading to opinion change for the better (‘virtuous opinion change’) in individual JDM agents? ’—and propose a modeling approach and research paradigm using structured groups (i.e., groups with some constraints on their interaction), for answering this question. This paradigm, based on that used in studies of judge-adviser systems, avoids the need for real interacting groups and their attendant logistical problems, lack of power, and poor experimental control. We report an experiment using our paradigm on the effects of group size and opinion diversity on judgmental forecasting performance to illustrate our approach. The study found a U-shaped effect of group size on the probability of opinion change, but no effect on the amount of virtuous opinion change. Implications of our approach for development of more externally valid empirical studies and theories of JDM, and for the design of structured-group techniques to support collective JDM, are discussed.
... It is particularly useful for forecasting and decisionmaking in situations where time is a critical factor or when there is coordination difficulty or a lack of sufficient expert engagement. Gordon and Pease (2006) introduced this method to address the time-consuming nature of the classical Delphi method, which can take several rounds of questionnaire iteration to reach a consensus or stability. The RTD aims to achieve a group consensus in a single round of questionnaire iteration (named by the authors as "round- (Aengenheyster et al., 2017;Gnatzy et al., 2011;Gordon & Pease, 2006;Meyer et al., 2022). ...
... Gordon and Pease (2006) introduced this method to address the time-consuming nature of the classical Delphi method, which can take several rounds of questionnaire iteration to reach a consensus or stability. The RTD aims to achieve a group consensus in a single round of questionnaire iteration (named by the authors as "round- (Aengenheyster et al., 2017;Gnatzy et al., 2011;Gordon & Pease, 2006;Meyer et al., 2022). ...
The Delphi technique is a commonly applied method for (among the various uses) achieving consensus from a group of knowledgeable experts. This approach is frequently employed to generate and prioritize ideas, identify potential solutions, and make decisions in various contexts through a series of iterative rounds. In the Futures Studies (FS) context, the Delphi method is regularly used in combination with the scenario method to explore different futures, implementing strategies in the present with the aim of averting dystopian outcomes and/or facilitating normative scenarios. Nevertheless, assuming that the convergence of opinions can also occur in spatial contexts, a shortcoming of the method is the deficiency of spatial references useful in the planning process. In this paper, we introduce the Real-Time Geo-Spatial Consensus System, a novel web-based open platform useful to develop Delphi-based Spatial Scenarios (DBSS), in an interactive and innovative interface. The platform adopts the Real-Time spatial Delphi technique to obtain a spatial convergence of opinions among experts to offer researchers, decision-makers, policymakers, and local authorities a new tool for complex spatial decisions. The primary innovations of the platform, including its architecture, statistical algorithms, tools, features, and outcomes, are demonstrated through a preliminary application focused on potential future climatic hazards in Dublin, Ireland.
... 38 The RT-Delphi approach was developed by Gordon and Pease to improve efficiency of the process and shorten the time of performance. 39 Since then, several online tools have been developed to facilitate the RT-Delphi design 40 and literature describing the use of RT-Delphi and comparison with the traditional multiround Delphi approach is growing. 23 41-44 In contrast to the traditional Delphi, the real-time approach is round-less and offers a constant iteration by providing immediate (real-time) individual and aggregated feedback. ...
... Participants are encouraged to revisit and engage in the survey several times during the study period. 39 40 42 44 In comparison with the traditional approach, the real-time approach encompasses all key Delphi features 43 and is similar from all key perspectives. 23 41 43 44 Furthermore, the real-time approach is particularly suitable for managing larger groups, decreases moderators' workload, simplifies inclusion of people from different geographic locations and can be leaner in costs. ...
... Delphi is a proven method in examining prospective developments of entire industries (Förster, 2015;Kluge et al., 2020;Merkel et al., 2016;Schuckmann et al., 2012;von Briel, 2018). To facilitate an effective group discussion among the experts (Belton et al., 2019;Linstone and Turoff, 2011), we decided to use the real-time variant of the Delphi method Gordon and Pease, 2006). To ensure a rigorous application of the research method with the highest standards, we have followed state-of-the-art methodological and technical papers from Beiderbeck et al. (2021b) and Schmalz et al. (2021), who suggest clear quality criteria for a three-step Delphi-procedure including study preparation, study conduction, and study analysis. ...
... A real-time format of the Delphi method Gordon and Pease, 2006), was used, which is a variation of the traditional Delphi approach (Linstone and Turoff, 1975). It lets participants proceed with the questionnaire at their convenience with immediate access to other respondents' responses after completing the survey, thus permitting an effective online group communication process (Belton et al., 2019;Linstone and Turoff, 2011). ...
Rapid technological progress and digitalization have considerably changed the role of technology in sports in the past two decades. As the human limits of performance have been reached in many disciplines, reaching future limits will increasingly depend on technology. While this represents progress in how athletes train and compete, similar developments await sports managers in the way they lead sports organizations and sports consumers in the way they consume and engage with sports. Using the SportsTech Matrix (i.e., a framework to capture how different types of technologies provide solutions to different user groups in sports), we examine how technology will impact sports in the future. We present a Delphi-based prospective study with quantitative and qualitative assessments from 92 subject matter experts for six future projections and 35 non-Delphi prospective survey items. We find that, by 2030, technology will significantly impact all three user groups in sports: athletes, consumers, and managers. For athletes, experts anticipate technology to play a major role for sporting performance improvements. For consumers, the consumption of sports content will continue to change significantly. For management, new types of manager profiles in terms of backgrounds and skill sets would be desirable. We discuss two possible future scenarios: (1) a probable future and (2) a game changer. Our findings should provide relevant insights for decision-makers and other stakeholders in sports and raise promising directions for future research.
... 36 As part of the review, we foresee discussing how to assess the potential risk of bias (which can lead to misuse and patients' harm), and if current tools such as 'PROBAST' are appropriate. 37 Stage 3: real-time Delphi exercise We will conduct a real-time Delphi survey 38 to review and refine the items of the PRECOG reporting checklist. Participants will be identified initially through the professional network of the core working group and of the external advisors, and further via literature search (including but not limited to the existing scoping review and the planned systematic review), social media screening and snowballing by the active participants. ...
... The participant selection will be based on area expertise grouping (computer science, biostatistics, biomedical informatics, statistics, epidemiology, standards, causal inference, ethics), used to determine the sample size (discussed below). We choose a computer-based, realtime Delphi, 38 since it offers some operational advantages with respect to conventional multi-round Delphi Open access techniques, for example, responder's attrition. 39 In brief, real-time Delphi is a 'roundless' exercise based on an online survey platform. ...
Introduction
While there are guidelines for reporting on observational studies (eg, Strengthening the Reporting of Observational Studies in Epidemiology, Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement), estimation of causal effects from both observational data and randomised experiments (eg, A Guideline for Reporting Mediation Analyses of Randomised Trials and Observational Studies, Consolidated Standards of Reporting Trials, PATH) and on prediction modelling (eg, Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis), none is purposely made for deriving and validating models from observational data to predict counterfactuals for individuals on one or more possible interventions, on the basis of given (or inferred) causal structures. This paper describes methods and processes that will be used to develop a Reporting Guideline for Causal and Counterfactual Prediction Models (PRECOG).
Methods and analysis
PRECOG will be developed following published guidance from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network and will comprise five stages. Stage 1 will be meetings of a working group every other week with rotating external advisors (active until stage 5). Stage 2 will comprise a systematic review of literature on counterfactual prediction modelling for biomedical sciences (registered in Prospective Register of Systematic Reviews). In stage 3, a computer-based, real-time Delphi survey will be performed to consolidate the PRECOG checklist, involving experts in causal inference, epidemiology, statistics, machine learning, informatics and protocols/standards. Stage 4 will involve the write-up of the PRECOG guideline based on the results from the prior stages. Stage 5 will seek the peer-reviewed publication of the guideline, the scoping/systematic review and dissemination.
Ethics and dissemination
The study will follow the principles of the Declaration of Helsinki. The study has been registered in EQUATOR and approved by the University of Florida’s Institutional Review Board (#202200495). Informed consent will be obtained from the working groups and the Delphi survey participants. The dissemination of PRECOG and its products will be done through journal publications, conferences, websites and social media.
... In recent years, different implementations of the Delphi method have been made, such as the possibility of having a valid and fast method to elicit the experts' judgments. In Real-Time Delphi (Gordon & Pease, 2006), participants have access to an online web-based questionnaire, where based on the answers of the panellists, an automatic calculation of the median and other statistical summaries takes place, so that the actors involved can access the questionnaire several times during the time window of the study. The rounds occur simultaneously and there are no more iterative rounds, which is why the two scholars call the method "round-less". ...
In the Futures Studies context, the Delphi-based scenario (DBS) is a valuable method for setting future-oriented strategies and actions by gathering expert opinions in multiple iterative rounds, in order to make better decisions in the present. However, one of the main challenges is to find a suitable representation of the scenarios both for evaluation and decision-making. The development of future scenarios requires high creativity and simple textual descriptions could not be enough to grab the attention of the experts and decision-makers, due to a probable lack of communication and/or technical language. Since the textual descriptions concern the future, the human mind may have difficulty imagining events that have not already happened. With the spread of AI models, different implementations can support the DBS process, such as the use of Generative Adversarial Networks (GANs) to generate specific images starting from the narratives of the scenarios. In this paper, we propose a new method for the development of Delphi-based Visual Scenarios (DBVS) adopting GANs, with the aim to support the experts and decision-makers in planning and communication, in order to have immediate images of what the future could be like. We apply this method with regard to the future of Italian families.
... The original Delphi methodology has been adapted for different conditions and purposes. Among the most noteworthy modifications to the initial design-for their early development and their subsequent and current success-are the Policy Delphi (De Loë et al., 2016;Turoff, 1970) and the Real-Time Delphi (Gnatzy et al., 2011;Gordon, 2009;Gordon & Pease, 2006;Turoff, 1972). The Policy ...
The Delphi method is a technique of social research that seeks to obtain a reliable group opinion from experts. It was first created for military purposes in the mid‐1950s. Since then, its use in the scientific field has continued to spread to different disciplines and aims. Despite this expansion, however, not set of indicators of the quality of Delphi studies has yet to be developed that might provide the reader—whether an expert in the technique or not—with some framework of reference whereby to gauge what credibility should be afforded to the results of the study. In this paper, following a thorough review of the literature on the criteria used to assess Delphi studies and the items of evaluation recommended for inclusion in Delphi reports, we determine what characteristics a quality evaluation indicator for this technique should have and propose a battery of indicators based on these characteristics, which should for preference be included in the final report of a Delphi study. The proposed indicators focus on three areas that are particularly relevant to the quality of Delphi research: the quality of the panel of participating experts, the way in which relevant information is obtained from the experts, and the quality of the interaction generated among the experts.
... There are many variations of the classic Delphi method, with the option for researchers to modify the approach to reflect research aims [23]. The current study utilised a modified realtime Delphi method to develop consensus on the core elements of the FFD model of care [24]. The real-time Delphi method comprises a round-less approach, continuously calculating and updating aggregated responses in real-time until the end of study timeframe [25]. ...
Background
Innovations in models of care for older adults living with frailty presenting to the emergency department (ED) have become a key priority for clinicians, researchers and policymakers due to the deleterious outcomes older adults experience due to prolonged exposure to such an environment. This study aimed to develop a set of expert consensus-based statements underpinning operational design, outcome measurement and evaluation of a Frailty at the Front Door (FFD) model of care for older adults within an Irish context.
Methods
A modified real-time Delphi method was used. Facilitation of World Café focus groups with an expert panel of 86 members and seperate advisory groups with a Public and Patient Involvement panel of older adults and members of the Irish Association of Emergency Medicine generated a series of statements on the core elements of the FFD model of care. Statements were analysed thematically and incorporated into a real-time Delphi survey, which was emailed to members of the expert panel. Members were asked to rank 70 statements across nine domains using a 9-point Likert scale. Consensus criteria were defined a priori and guided by previous research using 9-point rating scales.
Results
Fifty members responded to the survey representing an overall response rate of 58%. Following analyses of the survey responses, the research team reviewed statements for content overlap and refined a final list of statements across the following domains: aims and objectives of the FFD model of care; target population; screening and assessment; interventions; technology; integration of care; evaluation and metrics; and research.
Conclusion
Development of a consensus derived FFD model of care represents an important step in generating national standards, implementation of a service model as intended and enhances opportunities for scientific impact. Future research should focus on the development of a core outcome set for studies involving older adults in the ED.
... Various methods have been developed to assess and appraise risks (Covello and Merkhofer, 1993;Horvath and Zuckerman, 1993), especially those related to technology (Lefley, 1997;Stirling, 2008). Methods such as RT Delphi (Gordon and Pease, 2006) or cost-effective models (Hubbard, 2014) have been applied to prioritise technologies. However, the multi-criteria mapping (MCM) method provides an extensive view of potential options for assessing the uncertainties of new technological developments (Ely et al., 2014). ...
The deregulation of energy markets in many countries has stimulated the development of energy communities. Members of these communities generate and trade energy in the general market. In addition to pursuing financial goals, they aim to achieve positive environmental and social effects. Despite the recent surge in such communities, they often lack resources and depend on subsidies to compete with traditional energy providers. New technologies could support these communities with optimising their production and consumption, providing smart services and increasing their competitiveness. By applying the multi-criteria mapping (MCM) method to data collected in focus group discussions and interviews with experts, this study explored the potential of technologies and their prioritisation for the adoption of smart services by energy communities. The study proposes an overall ranking of design options for smart energy services based on four stakeholder perspectives and on opposing views and uncertainties. The results provide insights into the potential of each option for the design of innovative information systems (IS) for energy communities.
... The Delphi process was conducted in an online format, as all real-time Delphi, and owing to the COVID-19 pandemic context. Note that online Delphi performs similarly to in person [18,21,27,28]. ...
Background
The increased scrutiny on public health brought upon by the ongoing COVID-19 pandemic provides a strong impetus for a renewal of public health systems. This paper seeks to understand priorities of public health decision-makers for reforms to public health financing, organization, interventions, and workforce.
Methods
We used an online 3-round real-time Delphi method of reaching consensus on priorities for public health systems reform. Participants were recruited among individuals holding senior roles in Canadian public health institutions, ministries of health and regional health authorities. In Round 1, participants were asked to rate 9 propositions related to public health financing, organization, workforce, and interventions. Participants were also asked to contribute up to three further ideas in relation to these topics in open-ended format. In Rounds 2 and 3, participants re-appraised their ratings in the view of the group’s ratings in the previous round.
Results
Eighty-six public health senior decision-makers from various public health organizations across Canada were invited to participate. Of these, 25/86 completed Round 1 (29% response rate), 19/25 completed Round 2 (76% retention rate) and 18/19 completed Round 3 (95% retention rate). Consensus (defined as more than 70% of importance rating) was achieved for 6 out of 9 propositions at the end of the third round. In only one case, the consensus was that the proposition was not important. Proposition rated consensually important relate to targeted public health budget, time frame for spending this budget, and the specialization of public health structures. Both interventions related and not related to the COVID-19 pandemic were judged important. Open-ended comments further highlighted priorities for renewal in public health governance and public health information management systems.
Conclusion
Consensus emerged rapidly among Canadian public health decision-makers on prioritizing public health budget and time frame for spending. Ensuring that public health services beyond COVID-19 and communicable disease are maintained and enhanced is also of central importance. Future research shall explore potential trade-offs between these priorities.
... This Delphi method has been demonstrated to be well suited for many different business planning and product development applications. This choice of method is also one of the most appropriate for face-to-face meetings and when working with geographical limitations (Geist, 2010), and online applications (Gordon & Pease, 2006). Other heuristic research strategies were considered, including systematic literature and policy reviews and extensive quantitative data collection but were rejected on the grounds of pragmatism and opportunity. ...
The current convergence of global challenges, particularly the climate change emergency, the Covid-19 pandemic, and the Black Lives Matter movement, have highlighted the need for a new lens to challenge and interrogate key urban planning assumptions related to spatial urban inequality. Yet urban inequality is often and invariably described from a limited economic perspective, commonly interpreted and measured as income inequality. This is an overtly statistical measure, or Gini-Type index, often giving limited and unsatisfactory results. Yet, in practice, the spatial distribution and concentration of income inequality is a multi-scalar, multi-variant, and multi-disciplinary issue and has links with other and wider dimensions of inequality and well-being. As such, this article argues for a holistic understanding of urban inequality that goes beyond narrow empirical and quantitative models. It presents collaborative research that aims to impact the actions of urban professionals, to accurately identify and adequately respond to urban inequalities. Through the establishment of an interdisciplinary expert panel, we have uncovered a series of provisional mechanisms and responses to aid practitioners to achieve more spatial equality. We introduce an integrated analytical method, the “litmus test,” that acts as a planning tool for understanding, evaluating, and responding to inequalities and segregation present in the built environment. This novel methodology and procedural framework will assist us in (a) identifying and defining different forms of inequality and segregation beyond the current scope of physical and agency-based forms; (b) measuring and demonstrating the latter with a combination of qualitative, empirical sources that are materially significant in supporting and evidencing planning strategies; and (c) setting out a series of planning and built environment specific responses.
... An extended period between rounds makes it questionable whether participants can recall why or what was said in the previous round. Software has been developed that uses real-time Delphi, which relies on rapid completion of the rounds by the expert panels within a specific time after a survey is open (Gordon & Pease, 2006). Use of this software would align with our recommendation of keeping the shortest amount of time possible between rounds. ...
... Just one round of responses was required because the Delphi study was used just for validation. Furthermore, Gordon and Pease (2006) highlighted the need to reduce the rounds to improve the efficiency of the process in Delphi studies. ...
This paper focuses on university–firm relationships in terms of individual interactions between researchers and practitioners. More specifically, we focus on an analysis of the main factors that influence the use of the action research (AR) to achieve a successful doctoral thesis. In order to achieve this, we developed a Delphi study with 15 panelists whose common characteristic is that they defended or supervised an AR-based thesis in the field of business and management. The primary contribution of the research is the development of a reference framework that should be considered in the design of a doctoral thesis for which an AR methodology is put into practice. Four dimensions were defined: profiles of both the PhD candidate and supervisor, PhD program/university, and firm/organization. Three main conclusions were reached. First, it is crucial to have a cooperative “eye-to-eye” relationship between the university and the company. Second, the AR process must respond unequivocally to its own dichotomous nature. Third, there must be a straightforward academic process for the PhD thesis. We believe that this study may impel the development of doctoral theses based on AR as a tool to potentiate collaborative university–firm relationships.
... As the study was mainly an internet-based inquiry, it is consistent with e-Delphi method (Donohoe et al., 2012). Use of e-Delphi methodology provided the researchers with several benefits comprising: (1) access to a wide range of experts from different locations, (2) cost-effectiveness, (3) good response and retention rates, and (4) offering equal opportunity to the committee members to express their opinions (Gordon and Pease, 2006). This e-Delphi involved a systematic review of the literature and several rounds of discussion and independent voting (via internet) by a committee of 15 invited experts and facilitated by the staff at Milken Institute. ...
... Through three rounds, a total of thirteen valid responses were collected and the response rate was 86.67%. The sample size was satisfactory as 10-15 is adequate for a Delphi study to facilitate effective assessments with good consensus [51,52]. The experts were consultants who had professional experience in different AECO disciplines, such as architecture, building engineering, MEP, and HVAC, with at least three years of experience as consultants in AECO. ...
Although building information modelling (BIM) is a widely acknowledged information and communication technology (ICT) in the architecture, engineering, construction, and operation (AECO) industry, its implementation is hindered by the hybrid practice of BIM and non-BIM information processing, and sometimes, it fails to add value to the AECO business. It is crucial to define, on a scientific base, how to ensure the effective use of BIM regarding the various conditions in which to apply BIM in AECO practices. Although several studies have investigated similar topics, very few have focused on the adoption of distinct BIM applications over the conventional practice from the perspective of business intelligence (BI) as a theoretical framework to justify the effective value of BIM use in the AECO. This study proposes a framework relying on BI principles to justify effective BIM use and explicates the contextual factors in AECO practices. The data were acquired from a three-round Delphi survey. The framework suggests that effective BIM use in AECO practices should follow the two principles of BI: achieving technical effectiveness and realizing business value. The pursuit of technical effectiveness should consider business objectives, business issues, business sustainability and regulatory eligibility, and the realization of business value involves willingness to adopt BIM, human-computer interoperability, visualization-based data quality and sources, data processing and system integration, and application maturity. This study provides a new perspective by which to address the issue of the technological iteration in the current hybrid BIM and non-BIM practice and could help to improve BIM implementation in the AECO industry.
... This "round-less" adaptation enables experts to receive controlled feedback based on all previous expert answers in realtime after answering a question. Hence, they are given the opportunity to reevaluate their own choices immediately (Aengenheyster et al., 2017;Gordon and Pease, 2006) and, thus, combine steps (ii) and (iii) of traditional Delphi studies into one single step. Gnatzy et al. (2011) compared the traditional and RT Delphi method and concluded that while results of the two methods are generally of similar quality, RT Delphis offer greater convenience in terms of process, appearance and reduced (time) effort, one of the drawbacks of traditional Delphi studies (Beretta, 1996;Scapolo and Miles, 2006). ...
Implementing the circular economy (CE) in the plastic packaging sector is considered a crucial step towards the adoption of more sustainable practices. However, despite the presence of regulatory CE targets, the plastic packaging sector's transition is not speeding up sufficiently to reach them.
Individual companies need to reconfigure their resource base and develop complementary resources and capabilities to improve the systems performance. Based on the resource-based view, this study identifies and prioritizes the most important resources and capabilities for a circular transition along the plastic packaging supply chain. Accordingly, expert opinion was analyzed using a combination of a real-time Delphi method and an analytic hierarchy process. Two aspects were of utmost importance for the experts: collaboration capabilities between upstream and downstream actors, and design capabilities that create simpler and recyclable products. When prioritizing resources and capabilities, potential conflicts between reactive and proactive CE strategies were found. Further research is proposed to (i) develop adequate framework conditions to enable higher waste management targets, (ii) apply dynamic, natural and extended resource-based views to better understand the effects of collaboration networks in CE and their dynamics, and (iii) investigate supply chain power relationships to understand the impacts of supply chain transformations.
... As a result, sustainability variables were obtained, which consisted of identifying the affirmations or definitions stated by different authors regarding what should be considered in a sustainable development model. After identifying the variables, the construction and development of qualitative and quantitative questions were carried out, both of multiple choice, to facilitate their evaluation by interested parties, which were reviewed by experts in the sustainability field, applying the Delphi methodology by assessing its relevance and congruence [8][9][10][11]. Subsequently, the questionnaire was applied in the study region to evaluate its sustainability innovation and to propose this as a tool that could serve as a guideline for anyone interested in evaluating sustainability innovation in a region. This study was part of a master's thesis in sustainability and postdoctoral research related to innovation systems oriented towards sustainability. ...
In recent years, the need for changes in the current consumption and development patterns
has become evident. To achieve this, it is necessary to innovate, create, and devise new methodologies
and ways of thinking that allow for a reorientation towards economically prosperous, socially equitable, and environmentally reasonable practices. The purpose of this study is to identify sustainability
variables within the framework of the innovation system concept, and to propose a methodology for
diagnosing regions and identifying their gaps in a sustainability-oriented innovation system. The
methodology was based on a literature review of different documents, where sustainability variables
related to innovation were extracted from this search, consisting of the identification of concepts
made by different authors regarding what should be considered an innovation system aligned with
sustainable development. These concepts were identified as study variables and a questionnaire
was formulated based on them, which was reviewed by experts to determine their relevance and
congruence. After obtaining the final questionnaire, which was subsequently referenced as the
diagnostic tool, it was applied in a Colombian study region consisting of the states of Quindío and
Risaralda, with sustainability stakeholders. As a result, a great lack of knowledge on the subject on
the part of the respondents, low accessibility to sources of information, and a high percentage of
disarticulation between policies were found, which led to the conclusion that the evaluated region has
a low understanding of sustainability. Based on the information discussed in this research, it can be
concluded that there are currently no specific methodologies to measure the sustainability of a region
or territory. Therefore, the tool is determined to be a guide for the measurement of sustainability in
the context of innovation in any region. Finally, from the studies reviewed, the potential to include
sustainability in the innovation systems of a region was detected, enabling economic development, the production of goods and services, and strengthening the socio-environmental considerations involved in the adequate use of natural resources and the increase in the quality of life.
Keywords: innovation system; sustainable development; sustainability; Colombia
... To account for varying language skills and preferences among the panelists, we decided to conduct the survey in three different languages: English, French, and Spanish (i.e., three out of the four official FIFA languages except for German, assuming that German-speaking TDs would be sufficiently familiar with English). Therefore, we opted for the classic sequential Delphi format, which consists of two consecutive rounds (Gordon and Pease, 2006;Schmalz et al., 2021). In the first round, participants evaluated a set of futureoriented projections, in terms of their respective expected probability (EP), subjective desirability (D), and estimated impact (I) in case of occurrenceboth quantitatively and qualitatively. ...
The speed of technological innovation in football has drastically increased in recent years. Therefore, we examine the impact of (digital) technologies on the future of association football up until the year 2026. In this regard, we also take into account diverging socio-economic circumstances in different parts of the world. We use a two-round sequential Delphi method to gather both quantitative and qualitative data from an expert panel consisting of 85 technical directors from official FIFA member associations. In total, we test ten future-oriented projections and collect information about participants' work experience, sentiments, as well as their individual attitude towards technology. While experts generally agree that the importance of technology in football will continue to increase, we find differences in perception among sub-groups of experts (e.g., smaller associations express an even higher desire for technologies to be implemented) and identify two distinct future scenarios. Thus, our study sheds light on multifaceted opinions towards technology diffusion in football. This unprecedented global perspective allows governing bodies to understand the industry's expectations, desires, and reservations, which helps building a better understanding for the needs within the global football ecosystems with respect to emerging technology innovations.
... The theses and design guidelines were then the basis for a real-time Delphi [44,45] conducted between 7 October and 10 November 2019. We invited 95 national experts from the German Federal Institute for Occupational Safety and Health from various disciplines with research backgrounds in digital transformation and OSH to participate via email and a link to the web-based survey. ...
With the increasing digital transformation, work tasks are changing—in some cases, significantly. Our study addresses the question of whether the established criteria for work design are still sufficient or if they should get updated and additional criteria become necessary in the context of digitalization. In a multistage consensus process involving interdisciplinary groups of experts, we have identified specific criteria for the humane design of work in a world increasingly permeated by digitalized work tools. Starting with an expert workshop using a combined nominal group/focus group technique, followed by a real-time Delphi study, a content analysis and a five-stage peer comment process, we detected 13 criteria and 38 design guidelines for human-centered work in digital transformation. Mapping these with established criteria, it became apparent that some established criteria have experienced a new dynamic because of the digital transformation. For other criteria, a need for digitization-sensitive design is discernible. In addition, criteria have emerged whose necessity is rooted in the digital transformation. A diffusion and stronger interconnection of the various levels of the work system in connection with the digital transformation of work is apparent.
... Delphi technique developed by (T. Gordon & Pease, 2005). This method provides instant feedback as experts add to the data, giving a real-time up-date of median and inter-quartile ranges, as well as the reasons for experts to provide the scores that they have given. ...
This thesis takes an interdisciplinary approach to the study of the impact that constructions of meaning and personal identity have upon the processes of professional decision-making, in the delivery of community safety services. The research draws upon the previous work undertaken in the fields of psychology, sociology, social anthropology, criminology and community safety.
The research was composed of five separate studies. Study one was a Delphi exercise to determine consensus of meaning for different community terms in common usage for policy makers, practitioners and academics. The research was able to define consensual meanings for ten of the thirteen terms presented, including crime prevention, crime reduction and community safety. Consensus was not achieved for the terms community engagement, respect and quality of life and suggestions are made which may account for this result. Study two utilised repertory grids to investigate the ways that community safety professionals might construe the decisions that they have to make as part of their duties. Studies three and four utilised bespoke ISA/Ipseus instruments, whose structures were informed by the results from Study Two. These instruments were used to further explore the construals and worldviews of a variety of community safety professionals through six process postulates. It was found that whilst an individual’s initial job role or gender did not have significant impact upon their professional decision-making, the training that they had received in community safety and the time that they had spent working in the field did have a significant impact upon their professional decision-making. It was also found that the groups of community safety professionals differed in their attitudes towards those members of society who are the target of community safety activity. Study five involved the generation and piloting of a survey instrument whose various sections were designed to validate the findings generated from the previous studies, as well as providing further data on the decision-making processes of those working within community safety.
The final chapter presents the Warren Person Process Priority (WaPPP) layered model of decision-making that was derived from the data collected to inform the current thesis. The outer Person layer is defined by the four-way typology derived from the Procedural / Free-form and Cautious / Adventurous bi-polar constructs of identity types that were identified from the ISA/Ipseus studies. The middle layer of the model describes a number of different decision-making processes that professionals may follow when making a judgement or coming to a conclusion. The order of the processes was given by the results of the survey pilot. The central portion of the model presents a number of factors that may impact upon professional decision-making, determined from the ethnographic work that informed the ISA/Ipseus studies. The order of these factors was determined from the preparatory data collection instrument that was used with the ISA/Ipseus studies and confirmed by the results of the survey pilot.
Suggestions are made for further research that may expand upon the results presented in this thesis. These include a larger version of the Delphi, with an international panel of experts; correlation of the ISA/Ipseus instruments with other validated instruments for the measurement of personality, identity and decision- making and an expansion of the survey pilot.
... La encuesta se desarrolla en dos rondas de forma anónima. Se aplica la metodología Delphi (Gordon y Pease, 2006) para generar consenso en los temas definidos, donde el grupo final no debía ser inferior a 25 participantes (Goyeneche y Parodi, 2017). Se invita a participar del proyecto a 99 expertos, donde aceptaron 45. ...
La extensión agropecuaria debe conllevar a entender al productor como miembro fundamental del sistema agroalimentario. Esta premisa hace que sea necesario direccionar el trabajo que se viene implementando con la aprobación de la Ley 1876 de 2017, específicamente con el subsistema de extensión agropecuaria. Con el objetivo de establecer el escenario apuesta para la aplicación de las estrategias de la organizaciones que se encargan de la extensión agropecuaria en Colombia con visión al año 2025. Se define una matriz de métodos (individuales, grupales y masivos) y factores (infraestructura, tecnología y otros), así como definir los factores débiles para ser priorizados por expertos mediante dos rondas Delphi, definiendo los escenarios mediante una ronda de matriz de impacto cruzado. Se establecen cuatro escenarios al 2025 con una tendencia del 94 %, el escenario probable: política en camino diferente con un 29 %; el escenario apuesta: todos por el mismo camino con un 22.4 %; el escenario alterno: sin política y sin camino con un 21.3 % y escenario indeseado: la política llega tarde con un 21.3 %. Los principales métodos individuales, grupales y masivos que a futuro generan mayor impacto son las capacitaciones y visitas a todos los productores, organización de productores y giras técnicas y/o días de campo, aplicaciones móviles y radio, dentro de los factores necesarios las vías rurales, inteligencia de negocios y el factor débil las políticas sin continuidad y sin seguimiento. La extensión agropecuaria debe intervenir aspectos técnicos, económicos, políticos, sociales, infraestructuras rurales y tecnológicas para lograr impactar la calidad de vida de los pequeños productores.
... First, we used a truly interdisciplinary panel of researchers, many of them co-authors of the chapters of this book, to develop a set of projections in next generation manufacturing, drawing on our own research in the Cluster, but also incorporating the input of numerous external professionals in form of indepth expert interviews. Using a novel real-time Delphi approach (Gordon & Pease, 2006;Gnatzy et al., 2011), we then validated these projections with the help of a large international set of experts from multiple fields, e.g. engineering, information systems, social sciences, and management, asking them about their evaluation of the projections (with a projection period of 2030), but also about their qualitative feedback and commentary on the projections. ...
Transferring the idea of the Internet to the manufacturing landscape—the Internet of Production (IoP)—fundamentally changes our understanding of how products are developed, produced, and utilized. A key concept of the IoP is digital shadows that connect data, products, and equipment and are shared in cross-organizational data spaces. These developments are also core ideas driving the evolution of the current Industry 4.0 paradigm into its next generation (“Industry 4.U”) and have far-reaching implications that go beyond mere technical issues. From a company-internal perspective, managers and workers need to deal with new forms of collaboration and cooperation between humans, robots, smart machines, and algorithms. From a company-external (network) perspective, data-based value creation and capture in platform-based ecosystems change the logic of many manufacturing business models. These changes have been reinforced by the COVID-19 pandemic, which acted as a catalyst for many transformation processes. Given the high uncertainty in the likelihood of occurrence and of the technical, economic, and societal impacts of these concepts, we conducted a technology foresight study in the form of a real-time Delphi analysis to derive reliable future scenarios featuring the next generation of manufacturing systems. This chapter introduces the conceptual and technical background of this study, defines important terms and frameworks, and provides an overview of the Delphi projections that are presented and analyzed in greater detail in the remaining chapters of this book.[Abstract generated by machine intelligence with GPT-3. No human intelligence applied.]
... For this study, we used a real-time Delphi study, which calculates the group opinion instantly and displays it to the experts, which gives them the chance to reassess their answers immediately (Gordon and Pease, 2006;Gnatzy et al., 2011). Real-time formats are increasingly used as they deliver results similar to conventional Delphi studies while being more efficient (Di Zio et al., 2017;Aengenheyster et al., 2017;Geist, 2010). ...
Increasing sustainability is one of the core objectives of future supply chain networks. However, the ambitious economic, ecological, and social goals can only be achieved through novel technological frameworks accompanied by disruptive organizational change. The five essays of this dissertation investigate the adoption of sustainability-enhancing technologies and their underlying organizational transformations in order to reduce carbon emissions, improve economical performance, and increase social quality in logistics networks. Established methodologies in the form of case studies, managerial surveys, conceptual developments, Delphi surveys, and quantitative reviews are applied to ground this research empirically and to derive new concepts and research strategies. The results support researchers, practitioners, and politics in driving the adoption of sustainable measures and prepare organizations for future challenges. The first research paper, “Blockchain technology enabling the Physical Internet – A synergetic application framework”, studies the Physical Internet as an upcoming logistics concept that aims for a paradigm shift towards economically, environmentally, and socially sustainable logistics. The concept receives growing interest but requires transformations in infrastructure, technology, and business models, which existing literature intends to answer with centralized platform solutions. The paper demonstrates that centralized solutions do not meet the requirements to efficiently operate open logistics networks. Therefore, a decentralized solution is proposed which builds on a four-layered Blockchain-based framework. The framework describes in detail the configuration of the Blockchain and its integration into the Physical Internet. The implementation of the framework into the Ethereum environment is validated by an application use case that guides researchers for further analysis and practitioners for adoption opportunities. Thus, this scientific work not only contributes to a deeper understanding of the promising Physical Internet concept but also extends the knowledge about applying Blockchain technology in logistics and supply chain management in general. The second research paper, “The bumpy road to the adoption of the Physical Internet – Overcoming barriers from a stakeholder perspective”, empirically grounds drivers and barriers of the Physical Internet through a case study with 14 relevant stakeholders. This approach is distinctive, as the field is so far dominated by conceptual and modeling research. The case study sample consists of logistics service providers, shippers, and includes companies working on Physical Internet products and software. This way, the study holistically embraces the majority of the concepts’ relevant stakeholders. Based on technology adoption theory, insights about stakeholder intentions, organizational and technological readiness as well as enablers and hindrances of the concept are pointed out. As such, the study answers the two research questions regarding how and why relevant stakeholders might adopt the Physical Internet. Due to the expected far-reaching consequences of the concept at the firm level, the study contributes to a better understanding of how innovative concepts and technologies induce organizational transformations. This empirical study is the first, which scientifically investigates stakeholder intentions in the adoption process of the Physical Internet. The third essay, “How organizations prepare for the future: A comparative study of firm size and industry”, analyses future preparedness as a joint construct consisting of a company’s exploitation and exploration capabilities. While previous publications focused their research on large companies and single industries, this essay explicitly differentiates future preparedness strategies between large organizations and small and medium-sized companies, as well as between 12 different industries. Data from a managerial survey with 602 companies of the German economy identify firm size and industry as predictors for future preparedness strategies and implicate that the scope of future preparedness augments with firm size. Next to the managerial implications, which should motivate practitioners to engage in corporate foresight activities and to take a forward view for being prepared for environmental uncertainty, the study extends theory on the upcoming future preparedness phenomenon. The fourth paper, “Technology foresight for sustainable road freight transportation”, examines technological measures to improve the triple bottom line sustainability of road freight transportation. The speed at which the adoption of relevant sustainability technologies in transportation is currently progressing is too slow in order to sufficiently reduce carbon emissions and increase social quality in logistics networks. Hence, this study offers deeper knowledge about the adoption process by examining barriers hindering a broader market penetration. Based on these barriers, expected timeframes are derived answering the question when mass adoption of 14 technologies is expected to occur. Combined with the technologies’ impacts on sustainability, recommendations are made, on which technologies relevant stakeholders might focus on in the future. The study builds on a global Delphi survey with 116 experts from 25 different countries and in particular outlines the crucial role of politics in developing necessary regulative frameworks to drive sustainability in road freight transportation. The fifth paper, “Decarbonizing road freight transportation – A bibliometric network analysis”, extends the findings from the fourth paper by analyzing in detail the literature on environmental sustainability measures in road freight transportation. The study is the first attempt that quantitatively synthesizes the extensive multidisciplinary area of green road freight transportation. In the study, bibliographic coupling and network analysis techniques are leveraged to identify emerging areas and trends of the research field. As such, the study outlines and visualizes key research clusters to understand the underlying knowledge structures regarding strategies, methods, theories, and concepts. As a result, a detailed research agenda and explicit recommendations for decision-makers are derived.
... The recovery rate, which is proportional to the level of expert concern, is also closely related to the research credibility 33,34 and generally ≥75% guarantees questionnaire accuracy. 35 ② Concentration of expert opinion: i.e., importance score (X), expressed as mean ± standard deviation. The larger the X, the greater is the importance of the discussed item and the more concentrated the expert opinion is. ...
Objective:
In recent years, there are several systematic reviews published on animal experiments of Traditional Chinese medicine (TCM). PRISMA (preferred reporting items for systematic reviews and meta-analysis) guidelines provide a guarantee for significantly improving the reporting quality of systematic reviews (SRs) and meta-analysis (MAs) to a certain extent; however, there are still certain defects found in the quality of SRs/MAs of animal experiments of TCM. It has been found that especially, the descriptions of the rationale and animal characteristics of TCM interventions are inadequate. As a result, we have developed a novel reporting guideline for SRs/MAs of animal experimental in the field of TCM (PRISMA-ATCM) to overcome these problems.
Methods:
PRISMA-ATCM reporting guidelines were formed by analyzing both the status and quality of published SRs/MAs of animal experiments and consulting experts in the related fields, and then by Delphi consultation, consensus meeting and revision.
Results:
Among the 27 items on the PRISMA checklist, Title (1), Structured summary (2), Rationale (3), Objectives (4), Protocol and registration (5), Eligibility criteria (6), Data items (11), Planned methods of analysis (14), Study characteristics (18), Summary of evidence (24), Limitations (25), and Funding (27) have been extensively revised and expanded, to specifically include the details about TCM intervention and animal characteristics. In addition, illustrative examples and explanations have been provided for each item.
Conclusion:
PRISMA-ATCM could markedly improve the quality SRs/MAs of animal experiments in the field of TCM.
Background and objective
Consensus methods are crucial in developing clinical guidelines. Different methods, such as the Delphi and nominal group techniques, are commonly used, but there is a lack of detailed instructions on how to implement them effectively. The survey aims to explore the opinions and attitudes of the chair, panel and working group on the critical elements of the consensus methods during guideline development.
Methods
We used a cross-sectional design to conduct this study and sent a structured questionnaire to stakeholders, including the chair, panel members, and working group participants, through the popular mobile phone application WeChat.We selected participants using a combination of purposive and snowball sampling. The questionnaire gathered information on demographics, experiences, opinions, and concerns regarding consensus methods and guideline development.
Results
The sample comprised 290 participants representing 31 provinces or municipalities. Among them, the most significant number of respondents (n = 107, 36.9%) were from Beijing. Most participants, specifically 211 (72.76%), held senior professional titles, while 186 (64.14%) adhered to ongoing guidelines. The Delphi method was the most commonly used consensus method (n = 132, 42.31%), but the respondents had only a preliminary understanding of it (n = 147, 47.12%). The consensus process also revealed the insufficiency of involving pharmacoeconomists, patients, and nurses.
Conclusions
Consensus methods have to be standardised and used consistently in the guideline development process. The findings of this study offer insights into diverse roles and more effective ways to apply the consensus process during guideline development.
This contribution explains new variants of Argumentative Delphi surveys that can also be used in sociological research, some examples and the learnings from and limits of argumentative surveys with feedback. Argumentative Delphi surveys are not new. As Christian Dayé explains in his book, the early expert surveys and especially the Delphi surveys used explanations and arguments for exchanging knowledge - but always without direct interation (Dayé, C. (2020). Experts, Social Scientistss and Techniques of Prognosis in cold war of America. Socio-Historical Studies of the Social and Human Sciences, palgrave McMilan, Switzerland:41, see also Cuhls, K. (1998). Technikvorausschau in Japan. Ein Rückblick auf 30 Jahre Delphi-Expertenbefragungen. Physica. [Technology Foresight in Japan]). The very first approaches of Delphi surveys did not only make use of expert knowledge in judging issues under uncertainty or were trying to make accurate predictions with statistical analysis, but there were also tests in groups of students. In some of them, the groups did not only choose and tick boxes, but gave reasons or comments for their judgments. Modern Argumentative Delphi surveys do ask for comments AND use a variety of open questions for adding information to the statistical findings. This way of performing a Delphi survey gets more and more ground and can be analysed in a fast way by new means of text mining and Delphi software tools. But they have their limits - especially as they are very demanding for the participants and the analysts. If many people participate, many arguments are given, and they can quickly go beyond the limits of the participants‘ understanding and their time availability. Some lessons learned from recent Delphi projects are reported. This is closing the cycle to Dayé’s description of expert knowledge inclusion in policy-making - a way of integration of expert opinion without direct interaction.
Background
Telemedicine-delivered medication for opioid use disorder (TMOUD) has become more prevalent during the COVID-19 pandemic, particularly in North America. This is considered a positive development as TMOUD has the potential to increase access to evidence-based treatment for a population heavily affected by the opioid crisis and consequent rising mortality and morbidity rates in relation to opioid use disorder. Despite the increase in the use of TMOUD, there are no established service- and process-focused models to guide the implementation of this intervention.
Objective
This study aims to develop a process- and service-focused implementation model in collaboration with key stakeholders and bring together peer-reviewed literature, practice-based knowledge, and expert opinions.
Methods
The simple rules for evidence translation in complex systems framework was applied to guide the development of a 6-step qualitative study. The steps were definition of the scope and objectives of the model, identification of evidence, stakeholder engagement, draft model development, key informant consultation, and final model specification.
Results
The final specification for the TMOUD implementation model incorporated key strategic priorities, service delivery prerequisites, service design elements, stakeholder identification and engagement, key process domains, and iterative cycles of evaluation and improvement.
Conclusions
Through stakeholder engagement and key informant consultation, we produced a process- and service-focused TMOUD implementation model. The model is modifiable to different contexts and settings while also in keeping with the current evidence base and national and international standards of high-quality opioid use disorder care.
Digital twins (DTs) are virtual representations of real‐world entities like production assets, processes, or products. They are updated at a defined fidelity and frequency along the entire life cycle from development and engineering over the production or implementation of a product or process until its usage stage. Interconnected digital twins (IDTs) are DTs shared and connected across organizations with the objective to create holistic simulation and decision models of an entire physical system. In this paper, we investigate how IDTs shape future digital manufacturing scenarios and impact innovation management. We present the results of a real‐time Delphi study, analyzing quantitative and qualitative estimates on a set of 24 projections, forecasting the future of digital manufacturing with a projection horizon towards 2030. Using this data and 22 additional use cases of IDTs in manufacturing companies, we present a baseline scenario where our Delphi panel reached a consensus, representing a likely future of digital manufacturing in 2030. By analyzing projections where our expert panels' evaluations vary widely, we identify key design decisions that may impact innovation management along the dimensions of variation, choice, and control in digital manufacturing. We explain how IDTs will impact external knowledge inflows, the emergence and governance of industrial data spaces, and the potential of data‐driven and AI‐enabled applications for prediction and regulation to drive better decision‐making and continuous innovation.
In the classic Delphi method, results are processed only after the conclusion of a round, and the numerical and qualitative information is reported back to the experts. This article describes a “real-time Delphi” approach, which eliminates the conventional “round logic” in favor of intermediate result feedback. The article describes the requirements of this efficiency-centered approach and discusses the biases and problems as well as the specifics of the survey procedure. In addition, the paper presents providers of software tools and identifies current developments regarding real-time Delphis.KeywordsReal-time DelphiExpert interviewQuestionnaire methodReal-time Delphi tools
The Delphi method has developed from a “classic” to a variety of Delphi methods or “types”, which can have very different functions and are used in different subject areas. Online variants, especially the Real-time Delphi with instant feedback are becoming more and more popular. This introduction explains the different definitions, types and fields of application and shows the most important points to consider when using one of the Delphi methods. Special attention should be paid to the participants and their different backgrounds and expertise as well as to the design of the “questionnaire”. For example, typical questions in a Delphi are about the importance or time horizon of realizing a statement about a future issue, e.g. a problem solution in the health care system, a technology, or an educational measure. The Delphi method will continue to have a place in the canon of methods used in Foresight and Futures Research as well as in general empirical research in various disciplines where uncertainty is an issue (e.g. business administration). The method will be integrated more and more into overall processes and is a building block in the method mix of futures sciences, e.g. coupled with scenarios.KeywordsDelphiForesightFutures researchHistorical Delphi methodsDelphi variantsApplication examples
Illegal dumping behavior widely exists in the construction industry and is difficult to manage. What factors and how they influence the illegal dumping behavior of construction and demolition waste remains to be clarified. Specifically, this study employs an ISM-DEMATEL approach to investigate the critical factors that influence the illegal dumping behavior. It is found that the factors, ‘capital investment of enterprises’ and “stakeholders' willingness and attitude”, are vital to tackle the illegal dumping behavior of construction and demolition waste. Yet, compared with business-level factors, the macro-level factors such as ‘social norms and culture’ and ‘government incentive policies’ tend to be less influential. Moreover, enhancing the industry-wise administration system and practitioners' dedication to the issue can be an option to avoid illegal construction waste dumping. The results of this study serve to elucidate the influential factors and offer practical implications for addressing the issue of illegal dumping of construction and demolition waste.
In the Future Studies context, the scenario development process is an established method for the identification of future projections, useful to avoid future threats and take different actions in the present. The development of future scenarios is often combined with different participatory approaches, one among many is the Delphi method, widely adopted for its systematic and interactive nature. In this context, the recent climate challenges lead society to an exponential growth of uncertainty about the future where Delphi-based scenarios (DBS) could be helpful to identify interesting mid and long-term projections. For the purpose of conducting a systematic review of Delphi-based future scenarios applied to climate change context, we used a quantitative bibliometric analysis aimed at investigating the scientific literature path, implementing it with a multiple correspondence analysis and a semantic network analysis. We illustrate the results of the case studies focusing on the combination of methods, rounds of the process, panellists‘ sampling, time horizon, and techniques used, to establish new guidelines for future Delphi-based climate research projects.
In this article, we explore and theorise how actors in the Sheffield City Region envision the development and orchestration of an innovation ecosystem in order to capture value from industry 4.0 technologies for manufacturing firms. This is theoretically motivated by the need to harmonise the knowledge combination and recombination practices that are required for the successful existence of innovation ecosystems at a regional level, in the applied context of industry 4.0 regional transformations. Drawing on data originating in semi‐structured interviews, a focus group and a real‐time Delphi with companies, companies associations, Higher Education Institutions, policy makers and Research and Technology Organisations, we examine specifically the role played by boundary organisations and how they respond by dynamically managing and articulating demands from within the network of partners. Our study deepens the theoretical and operational understanding of boundary organisations in the context of regional innovation systems and discusses how to sustain their role of bridging diverse epistemic communities.
Managing networks for eco‐innovation remains a top priority in manufacturing due to industrial collaborations being the de facto mode of functioning for modern manufacturing firms. Accordingly, an awareness of critical network factors is imperative for strategies that lead to successful sustainability‐focused partnerships and profitable collaborations. This study applies a group‐based multi‐criterion decision‐making technique—the Delphi methodology—to evaluate critical network factors for eco‐innovation in manufacturing from the viewpoint of 116 experts within the triple helix, that is, university, industry and government experts. The study uniquely frames the eco‐innovation strategy challenge for manufacturing from a network perspective. Review of literature and an online survey identifies 63 network motivational, configurational and transformational factors. Subsequent polled results show heterogeneity during Delphi selection (Cochran's Q: p < 0.001) and prioritisation (Friedman's Q: p < 0.001) rounds with ‘environmental concern by top management’, ‘innovative organisational strategies’, and ‘integrity norms of signed contract’ as the top motivational, configurational and transformational factors. The study also discusses the theoretical and managerial implications of the study for triadic orientations and triple helix framing of eco‐innovation networks and concludes by identifying research limitations and potential future research areas.
Door-to-door (D2D) air travel is gaining momentum for airlines, airports, and feeder traffic providers. The mobility industry and researchers are broadening their scope to include the entire travel chain, from origin to final destination. Intermodal mobility products are already on the market. At the same time, widespread trends affect transport service providers (as the suppliers) and passengers (regarding demand). Acquiring a better understanding of future D2D air travel trends is crucial for the mobility sector for long-term planning, product adaptation, the services provided and the pricing of these, and improvements in the passenger experience. Focusing on the European market, the overall objective of this doctoral thesis is to identify and understand the future trends of D2D air travel. It is divided into three parts; these provide different perspectives on trends and employ a range of methods that lead to results that develop from each other. In Part One, the Delphi technique is utilized to identify future travel trends. The study considers projections of European air passengers and their requirements for their entire air travel chain, including airport access, a long-haul flight, and airport egress. The research focuses on 2035 and is based on a two-round Delphi survey involving 38 experts from the transport industry, academia, and consultants. The Delphi survey is supplemented with findings from a preliminary study, consisting of a literature review, interviews with 18 experts in the field of air travel, and a workshop attended by experts. Results reveal that digitalization and personalization will be the main drivers in 2035 and that passengers might demand value-added use of their travel time. In addition, environmentally friendly travel products are considered desirable but only somewhat probable by 2035. Passenger type, age, origins, and travel budget will still be influential factors in 2035. Based on the results from a hierarchical cluster analysis, Part One presents three possible future scenarios: (1) personalized D2D travel, (2) integrated D2D travel, and (3) the game-changer. A technical chapter elaborates on the Delphi technique and individual research steps. Part Two explores the supply aspect and to what extent transport service providers consider strategically relevant trends. The scope of D2D air travel is adapted by applying multi-labeled text classification models to 52 corporate reports from a sample of transport service providers that operate in the European market. Trends identified in the first Delphi study and from an additional literature review are used to develop seven classes. Two prototype models are developed: a dictionary-based classifier and a supervised learning model using the multinomial naive Bayes and linear support vector machine classifiers. The latter yields the best model output, revealing which trends have a higher, medium, or lower relevance on the supply side. The results show that providers consider environmentally friendly air transport and related products to be highly relevant while disruption management, leveraging passengers' data, and improving airport feeder traffic through innovative mobility initiatives are considered to be of medium relevance. Part Three explores air passengers' preferences and willingness to pay for ancillary services in the current transition into the new normal, brought about by the ongoing COVID-19 pandemic, high uncertainty, and changing market dynamics. A choice-based conjoint analysis is used to test six attributes within a hypothetical travel scenario for a long-haul one-way air trip. Choice data from 269 German business and leisure passengers are analyzed using a hierarchical Bayes estimator. Results reveal that the total ancillary service upgrade price influences passengers' choices the most, followed by a seat upgrade for greater comfort and the CO 2-compensation of a flight. Hygiene-related ancillar-ies bring low utilities. Female and senior passengers care more for environmentally friendly ancillaries. Confirming previous research, business passengers and frequent flyers care more for onboard comfort.
Download: https://opus4.kobv.de/opus4-whu/frontdoor/index/index/docId/934
In der Dissertation wird der Frage nachgegangen, welche globalen
bildungspolitischen Maßnahmen erforderlich sind, um auch bislang
exkludierten Menschen den Kompetenzerwerb zu ermöglichen, der benötigt wird, eine positive User Experience in benutzergenerierten, digitalen Lernumgebungen auszubilden, damit sie an der modernen Weltgesellschaft selbstbestimmt teilhaben können. Zu diesem Zweck wurden Castells ‘Netzwerkgesellschaft’ und Csikszentmihalys ‘Theorie der optimalen Erfahrung’ als analytische Grundlagen zur Einordnung der sozialen Netzwerk-Aktivitäten herangezogen. Dies ermöglichte es, unter
Rückgriff auf aktuelle Lerntheorien, Kompetenzdebatten, ökonomische
Analysen des Bildungssystems und User Experience-Forschungen, einige
individuelle und gesamtgesellschaftliche Voraussetzungen abzuleiten,
um in der Netzwerkgesellschaft konstruktiv überleben zu können. Mit
Blick auf unterschiedliche sozio-kulturelle Bedingungen für persönlichen Flow im ‘space of flows’ liessen sich schließlich differenzierte Flow-Kriterien entwickeln, die als Grundlage für die Operationalisierung im Rahmen einer Real-Time Delphi (RTD)-Studie mit einem internationalen Expertinnen-Panel dienen konnten. Ziel war es, bildungspolitische Ansatzpunkte zu finden, den bislang Exkludierten bis zum Jahre 2020 erste Rahmenbedingungen zu bieten, damit sie potentiell teilhaben können an der Gestaltung der zukünftigen Netzwerkgesellschaft. Das Ergebnis der Expertinnen-Befragung wurde unter Rückgriff auf aktuelle Global und Educational Governance-Studien und das Einflusspotenzial der Zivilgesellschaft auf den Digital Divide reflektiert. Vor diesem Hintergrund konnten abschließend vier bildungspolitische Verlaufsszenarien entworfen werden, die es ermöglichen könnten, bis 2020 die Kluft zu den global Exkludierten wenigstens etwas zu schließen.
Since its introduction decades ago, the Delphi method has become a popular instrument of broad application within the field of technology foresight and for wider science, technology, and innovation policy‐making. The Delphi method has also attracted interest from a methodological perspective. The literature on the Delphi method demonstrates a research gap on the issue of Delphi statements and respondents' understanding of those statements. Based on a systematic literature review, this paper examines the existing knowledge and practices in constructing Delphi statements. A key contribution of the paper is a set of practical recommendations for constructing Delphi statements for technology foresight. Furthermore, the paper suggests future research on the topic.
Sedentary lifestyles and the lack of physical activity (PA) are a major concern among all age groups, and current generations tend to be less fit than the previous ones in the Western World. At the same time, there is an urgent need to cut transport-related carbon dioxide (CO2) emissions. Major gains can be foreseen if current car-centred lifestyles and sedentary behaviour are addressed from an integrated perspective. In this study, we explore future scenarios in the intersections of PA and active lifestyles as well as related environmental and health benefits in Finland. We used a disaggregative Delphi approach to examine the topic. Although frequently used in health-related research, Delphi has rarely been used in exploring alternative futures or non-consensus. The study design was based on a mixed-methods approach where we combined both qualitative and quantitative data analysis. Building on the experts’ perceptions on alternative futures, we formulated four scenarios for PA up to 2030, which we named Mismatch, Empowerment, Fatigue and Balance. The scenarios may be utilised as guides in developing future policies and decision-making, and to build better futures. Our scenarios demonstrate that alternatives do exist, and actions can be realigned with the positive scenarios of Empowerment and Balance. The physically inactive scenarios of Mismatch and Fatigue represent avoidable scenarios.
Der Begriff der Resilienz hat eine zunehmend wichtige Bedeutung in der gegenwärtigen Diskussion in Wissenschaft und Praxis erlangt. Vor dem Hintergrund sich dynamisch verändernder Rahmenbedingungen stellt sich zunehmend die Frage, wie der Resilienzansatz dabei helfen kann, neue Sichtweisen auf Entwicklungstrends (u. a. Digitalisierung) und Herausforderungen (u. a. Klimawandel, demografischer Wandel) zu entwickeln und eine Regionalentwicklung mit dem Ziel einer ausgeprägteren Robustheit und Krisenfestigkeit, und damit Resilienz, voranzutreiben. Der Beitrag möchte verdeutlichen, dass durch die Beteiligung von unterschiedlichen Akteuren (Wirtschaft, Wissenschaft, Politik und Gesellschaft) und durch die von diesen Akteuren beeinflussten regionalen Netzwerke, nicht nur die Resilienz gestärkt, sondern auch die Innovationsfähigkeit einer Region gefördert werden kann. Neben einem Überblick über das Verständnis von Resilienz in der Regionalentwicklung liegt aus diesem Grund ein besonderer Schwerpunkt des Beitrags auf regionalen Netzwerken, die ausschlaggebend für die Innovationsfähigkeit sind. Im Rahmen einer empirisch grundgelegten Fallstudie zur regionalen Resilienz am Beispiel der Region Ingolstadt geht das Kapitel der Frage nach, welche Akteure und Systeme eine Rolle im strategischen Umgang mit Fragen der regionalen Resilienz einnehmen und wie regionale Netzwerke und Innovationssysteme zur Resilienz in der Regionalentwicklung beitragen. Die mithilfe der qualitativen Methode GABEK® gewonnenen Ergebnisse verdeutlichen die Potenziale, die sich aus regionaler Vernetzung entlang des Quadruple-Helix-Modells ergeben.
The Delphi method is a structured scientific approach used to organize and structure an expert discussion in order to gain insights about the future. In order to develop scenarios for the future of Next Generation Manufacturing, an innovative real-time Delphi survey was conducted with 35 experts from industry and academia. The survey involved evaluating a set of 24 projections on the future of Next Generation Manufacturing, and the results of the survey were used to develop reliable future scenarios. Our main objective was to create a picture of the elements of Next Generation Manufacturing in 2030, guided by developments in the context of Industry 4.0. By using an innovative real-time Delphi approach in the context of Next Generation Manufacturing, we extend this established tool of strategic technology management from predicting technological developments and their impact on firms and society to providing a strategic guide for decision-makers in times of high uncertainty. Our study thus serves as a template for further applications of forecasting studies in interdisciplinary settings with high degrees of technical uncertainty.[Abstract generated by machine intelligence with GPT-3. No human intelligence applied.]
Background: The increased scrutiny on public health brought upon by the ongoing COVID-19 pandemic provides a strong impetus for a renewal of public health systems. This paper seeks to understand priorities of public health decision-makers for reforms to public health financing, organization, interventions, and workforce.
Methods: We used an online 3-Round Delphi method of reaching consensus on priorities for public health systems reform. Participants were recruited among individuals holding senior roles in Canadian public health institutions, ministries of health and regional health authorities. In Round 1, participants were asked to rate 9 propositions related to public health financing, organization, workforce, and interventions. Participants were also asked to contribute up to three further ideas in relation to these topics in open-ended format. In Rounds 2 and 3, participants re-appraised their ratings in the view of the group’s ratings in the previous round.
Results: Eighty-six public health senior decision-makers from various public health organizations across Canada were invited to participate. Of these, 25/86 completed Round 1 (29% response rate), 19/25 completed Round 2 (76% retention rate) and 18/19 completed Round 3 (95% retention rate). Consensus (defined as more than 70% of importance rating) was achieved for 6 out of 9 propositions at the end of the third round. In only one case, the consensus was that the proposition was not important. Proposition rated consensually important relate to targeted public health budget, time frame for spending this budget, and the specialization of public health structures. Both interventions related and not related to the COVID-19 pandemic were judged important. Open-ended comments further highlighted priorities for renewal in public health governance and public health information management systems.
Conclusion: Consensus emerged rapidly among Canadian public health decision-makers on prioritizing public health budget and time frame for spending. Ensuring that public health services beyond COVID-19 and communicable disease are maintained and enhanced is also of central importance. Future research shall explore potential trade-offs between these priorities.
Technology and innovation management is vital emerging research fields. Technological Forecasting and Social Change (TFSC) has worked as a major forum in this field and is currently regarded as the leading journal. However, an increasing number of publications hamper a comprehensive understanding of the field and journal. In this study, we conducted a systematic review of TFSC with the support of bibliometric analysis. We used citation network analysis and topic models to extract research landscapes and trends. Our results illustrate how technology and innovation management research has developed through the interactions among theories, methods, and cases, both qualitatively and quantitatively. Based on our analysis and findings, we discuss the major branches of research, topics in the journal, and future perspectives.
Rapid expansion of digitalization and in the volume of data available constitutes a major driver toward circular economy. In the textile industry, with its vast quantities of waste and huge environmental impact, transformation toward such circularity is necessary but challenging. To explore how the use of data could support building sustainability-aligned pathways to circular economy of textiles, a study employing a two-round disaggregative Delphi approach (engaging 33 experts in the first round, in May 2021, and 26 in the second, in June 2021) articulated alternative images of the future. The three images, dubbed Transparency, Conflicting Interests, and Sustainable Textiles, imply that the role for data is intertwined with sustainability aspirations. The results highlight that exploiting data in pursuit of circular economy is a collaborative effort involving business value networks that include consumers and regulators. Availability and sharing of accountability-affording, meaningful data on textiles' life cycle and value network function as a key enabler. By working with the images developed, actors can better assess their circular-economy commitments, planned actions, and the consequences of these. Furthermore, the images provide a tool for mutual discussion of the development desired and of related responsibilities and uncertainties.
Background:
Given substance use disorders (SUDs) among people with HIV are highly prevalent, integrating SUD services within HIV service settings is needed to help end the HIV epidemic. In this study, we assessed the setting-intervention fit (SIF) of 9 evidence-based SUD interventions: acamprosate, disulfiram, oral naltrexone, injectable naltrexone, oral buprenorphine, injectable buprenorphine, contingency management, motivational interviewing, and cognitive behavioral therapy (CBT).
Setting:
Clinical and nonclinical HIV service organizations (HSOs) in the United States.
Methods:
In May 2020, a stakeholder-engaged real-time Delphi was completed with 202 HSOs. HSO respondents rated the extent to which each SUD intervention was fundable, implementable, retainable, sustainable, scalable, and timely for their HSO, and these 6 items were summed into an SIF score (possible range of 0-18).
Results:
Motivational interviewing had the highest average SIF score (11.42), with SIF scores above the midpoint (9.5) for clinical (11.51) and nonclinical HSOs (11.36). For nonclinical HSOs, none of the other interventions were above the midpoint. For clinical HSOs, the average SIF scores were above the midpoint for CBT (10.97) and oral buprenorphine (9.51). Multivariate regression analyses, which controlled for characteristics of the HSO respondent, revealed geographic region of the United States and whether the HSO currently offered any substance use services as 2 of the best predictors of SIF scores.
Conclusions:
Notwithstanding the need to improve the SIF for the other evidence-based SUD interventions, motivational interviewing, CBT, and oral buprenorphine are currently the evidence-based SUD interventions with greatest perceived fit for integration within HSOs in the United States.
The Suggested Upper Merged Ontology (SUMO) is an upper level
ontology that has been proposed as a starter document for The
Standard Upper Ontology Working Group, an IEEE-sanctioned working
group of collaborators from the fields of engineering, philosophy,
and information science. The SUMO provides definitions for
general-purpose terms and acts as a foundation for more specific
domain ontologies. In this paper we outline the strategy used to
create the current version of the SUMO, discuss some of the
challenges that we faced in constructing the ontology, and describe
in detail its most general concepts and the relations between them.
Abstract: We discuss the development of an environment for formal knowledge engineering. The Sigma system is an advance over previously developed systems in that it integrates a number of modern ontology development tools, which has motivated a number of research issues. Primary components include an ontology browsing and editing environment, a first order logic inference system and a natural language to logic translator. Although largely independent of any particular ontology, it supports a number of publicly available formal ontologies.
There are a number of differences in procedures when a large scale Delphi is undertaken. Some of the most important differences are discussed in the light of Delphi embracing a panel of 385 members responding to a study sponsored by the National Center for Higher Education Management Systems (NCHEMS) at the Western Interstate Commission for Higher Education (WICHE) in 1972. Such topics as panel fatigue, panel consister cy, the consensus phenomena, and early plurality are considered in the report.
The preparation of research and technology development (RTD) programmes calls for the analysis of prospective technological advances and the consideration of ''lessons learned'' from earlier programs. In this paper, we report experiences from a pilot project in which an internet-based survey and five participatory workshops were organised in conjunction with a Finnish RTD program in telecommunications. At these workshops, leading representatives from industry and the research community were invited to report their experiences from a concluding RTD program and to address future research needs, whereby they were assisted by a group support system (GSS). The workshop process was well received, which suggests that the preparation of RTD efforts can benefit from the deployment of GSS tools in support of face-to-face discussions. To our knowledge, this is the first account on GSS use in the foresight context.
Ontologies provide advantages of knowledge reusability, sharing, and greater robustness when used to build large knowledge-based applications. Unfortunately, translating between English statements and a specific ontology requires skill in knowledge engineering and an understanding of formal logic and the ontology itself. A knowledge engineer must be familiar with the concepts in the ontology, the fine distinctions between terms, and the specific way the ontology conceptualizes the world. We have developed a tool, CELT (controlled English to logic translation), to enable nonprogrammers to add knowledge expressed in terms of an ontology. CELT is an automatic translation tool to convert controlled English to KIF formulas using ontologies built with the suggested upper merged ontology (SUMO). Wordnet provides a base lexicon and a default preference for word senses. We do not expect CELT to obviate the need for knowledge engineers but to instead better leverage their time, as current machine translation tools assist professional human translators.
bComputer Based Delphi ProcessesQ an invited chapter Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health
- Murray Turoff
- Starr Roxanne Hiltz
Murray Turoff, Starr Roxanne Hiltz, bComputer Based Delphi ProcessesQ an invited chapter, in: Adler Michael, Ziglio Erio (Eds.), Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, Kingsley publishers, London, 1996.
The Delphi Method, Futures Research Methodology V2, CD ROM, The Millennium Project
- T Gordon
T. Gordon, The Delphi Method, Futures Research Methodology V2, CD ROM, The Millennium Project, American Council for the United Nations University, 2003.
Issues in large scale Delphi studies
- Huckfeldt