No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Delphi Technique: Making Sense Of Consensus
Abstract
The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study.
... (e-Delphi) survey method to identify linguistic categories relevant to hearing aid experiences by leveraging preselected linguistic categories from the LIWC software (Boyd et al, 2022), dimensions identified through Meaning Extraction Method (MEM) and principal component analysis (PCA) of open-ended text on hearing experiences , and systematically derived dimensions from qualitative studies on hearing aid experiences (Oosthuizen et al, 2022). Delphi studies often employ multiple rounds of surveys to obtain feedback from a panel of informed or expert individuals regarding their opinions on specific issues or concerns until consensus has been reached (Hsu and Sandford, 2007b;Vázquez-Ramos et al, 2007). Modified Delphi studies differ from classical techniques in allowing modifying items to be discussed with panel members (Keeney et al, 2011). ...
... Panel selection for this study involved purposive sampling, targeting 23 professionals with extensive experience in providing or researching hearing aids for adults and having a substantial publication record on the topic. A target sample of 23 experts was chosen to account for possible attrition between rounds and stay within the recommended number of 10-15 panel members for a homogenous panel (Hsu and Sandford, 2007b). ...
... Table 5) for the future development of a customized hearing aid LIWC text processing module. Despite the small sample size, it was still within the recommendation for a homogenous group of experts in a Delphi approach (Hsu and Sandford, 2007b). Thus, this approach allowed for integrating varied perspectives, enhancing the study's findings with insights from different cultural and professional contexts. ...
Background: Recent advancements in automated natural language processing (NLP) methods and tools have enhanced the efficacy and accuracy of quantitative analysis of natural language data. NLP offers significant potential for audiology by providing valuable insights from open-text responses about users’ lived hearing aid experiences. Purpose: This study aimed to establish linguistic categories pertinent to the experiences of adults using hearing aids, with the ultimate goal of developing a specialized text processing module to facilitate natural language analysis of textual data on hearing aid experiences. Research Design: A modified electronic Delphi (e-Delphi) design was employed. Study Sample: A panel of 16 audiology experts from seven countries. Data Collection and Analysis: Two survey rounds were conducted. In round 1, experts rated categories from the Linguistic Inquiry and Word Count (LIWC) software and categories from principal component analysis of open-ended text data on hearing aid experiences. Experts also responded to open-ended prompts regarding categories derived from qualitative studies on hearing aid experiences. Responses were condensed and refined into items for rating during round 2. In round 2, experts reconsidered their ratings in light of the group answers of round 1. All ratings used a four-point Likert scale of importance. Measures of central tendency, levels of dispersion, and Cronbach’s α reliability coefficients were conducted in both rounds for comparative purposes. Results: The open-ended section generated 26 items. Consensus was met on 53 linguistic categories encompassing social (e.g., social situations, social support), emotional (e.g., positive tone, negative tone), cognitive (e.g., cognitive processes, causation), lifestyle (e.g., work, leisure), hearing aid–related (e.g., sound quality, use and handling), and service delivery–related dimensions. Conclusions: The linguistic categories identified provide a foundation for developing a customized LIWC text processing module tailored to the analysis of hearing aid experiences. Future research is needed to refine and validate the custom text processing module. Clinical Relevance Statement: The findings provide a framework to enable the development of a customized hearing aid LIWC dictionary. Such a tool may enhance clinicians’ ability to explore hearing aid outcomes and improve patient-centered care by integrating real-world experiences into clinical practice.
... The Delphi method typically commences with a brainstorming session, which serves to gather specific information about the topic under examination (Hsu & Sandford, 2007). Once the topic and its crucial aspects are framed, the technique involves creating a well-structured questionnaire or set of items to be submitted to panelists for voting. ...
... As explained above, the Delphi method requires the generated questionnaire or set of items to be submitted to panelists for voting. The selection of panelists can affect the quality of the results generated, and there is no clear definition of a "Delphi panelist" (Hsu & Sandford, 2007;Welty, 1972). To overcome this issue, panelists were chosen based on the relevance of their background and expertise to the project's objectives (Belton et al., 2019;Hsu & Sandford, 2007). ...
... The selection of panelists can affect the quality of the results generated, and there is no clear definition of a "Delphi panelist" (Hsu & Sandford, 2007;Welty, 1972). To overcome this issue, panelists were chosen based on the relevance of their background and expertise to the project's objectives (Belton et al., 2019;Hsu & Sandford, 2007). Particularly, the Proponent Group identified 1152 potential panelists by selecting all members of the editorial boards of leading scientific journals in behavioral and cognitive psychology and neuropsychology, as well as researchers from their personal networks, all of whom held at least a PhD. ...
Are scientific papers providing all essential details necessary to ensure the replicability of study protocols? Are authors effectively conveying study design, data analysis, and the process of drawing inferences from their results? These represent only a fraction of the pressing questions that cognitive psychology and neuropsychology face in addressing the “crisis of confidence.” This crisis has highlighted numerous shortcomings in the journey from research to publication. To address these shortcomings, we introduce PECANS (Preferred Evaluation of Cognitive And Neuropsychological Studies), a comprehensive checklist tool designed to guide the planning, execution, evaluation, and reporting of experimental research. PECANS emerged from a rigorous consensus-building process through the Delphi method. We convened a panel of international experts specialized in cognitive psychology and neuropsychology research practices. Through two rounds of iterative voting and a proof-of-concept phase, PECANS evolved into its final form. The PECANS checklist is intended to serve various stakeholders in the fields of cognitive sciences and neuropsychology, including: (i) researchers seeking to ensure and enhance reproducibility and rigor in their research; (ii) journal editors and reviewers assessing the quality of reports; (iii) ethics committees and funding agencies; (iv) students approaching methodology and scientific writing. PECANS is a versatile tool intended not only to improve the quality and transparency of individual research projects but also to foster a broader culture of rigorous scientific inquiry across the academic and research community.
... Due to the availability and accessibility of experts, the participant number was set between 10 and 15 [35]. Selection was based on the following criteria: residence in the German-speaking region of Switzerland (CH) or Germany (D); professional background in physiotherapy, medicine, or sports science; more than 10 years of clinical and/or research experience in the cervical spine and/or head region, as well as in SMT. ...
... Selection was based on the following criteria: residence in the German-speaking region of Switzerland (CH) or Germany (D); professional background in physiotherapy, medicine, or sports science; more than 10 years of clinical and/or research experience in the cervical spine and/or head region, as well as in SMT. Additionally, participants with a research background were required to have more than three publications (including primary and secondary studies, as well as books and book chapters) [33,35]. Fifteen experts were contacted via e-mail between November and December 2021, of whom 13 agreed to participate. ...
... Two declined for professional reasons, and one expert did not complete the questionnaire. The final 12 participants formed a homogenous expert group in terms of professional expertise, with members representing universities from Switzerland and Germany [35]. ...
Introduction: Neck pain is a common condition associated with high treatment costs, reduced productivity, and work limitations. It represents the second-largest group requiring physiotherapy services. Given its high prevalence, it is crucial to evaluate intervention effectiveness and ensure optimal implementation in practice. Literature highlights sensorimotor training (SMT) as an important approach in managing neck pain, though dosage recommendations for clinical application are only partially defined. This study aims to develop practical dosage guidelines for SMT in neck pain to support its implementation in daily physiotherapy.
Methods: A Delphi study was conducted. Based on literature and the expertise of 12 specialists from Switzerland and Germany, dosage recommendations were developed. Two questionnaire rounds were required to reach consensus (≥ 80%).
Results: After the second round, 92–100% consensus was achieved on key aspects: targeted abilities, training cycle, frequency, training duration, number of exercises, sets per exercise, repetitions, time per exercise, and rest time. Additional background questions and comments contributed to a factsheet designed as a practical clinical guide. Recommended parameters include: training cycle of 4–12 weeks, 3–21 sessions/week, 10–20 minutes/session, 2–6 exercises/session, 2–5 sets/exercise, 7–15 repetitions or 20–60 seconds/exercise, and 20–40 seconds rest between sets. Further guidance is provided on targeted abilities, clinical indicators of positive effects, criteria for progression, and required adjustments for effective SMT.
Conclusion: This study closes a knowledge gap regarding SMT dosage. The resulting factsheet aims to simplify SMT implementation for neck pain and promote its integration in daily physiotherapy practice.
... Following the co-design of potential intervention elements with a diverse project steering group, consensus on these elements was sought using e-Delphi methodology. Within an e-Delphi, experts rate statements within an electronic survey, and statements meeting a minimum threshold for inclusion are taken forward [5]. ...
... A modified e-Delphi methodology was selected to establish consensus by expert SLTs on the core intervention elements generated by the project steering group. An e-Delphi methodology was selected as it is a structured way of gaining consensus across a group of panellists, whilst minimising respondent bias by ensuring anonymity [5]. The Delphi was conducted electronically (an e-Delphi), to enable responses to be given at a time most convenient to panellists and thus facilitate uptake [48]. ...
Introduction
Although frequently seen in clinical services, there are few interventions which have been developed specifically to meet the needs of pre-school children with co-occurring features of a phonological speech sound disorder (P-SSD) and developmental language disorder (DLD). This study aims to achieve consensus on the core elements of a novel intervention for pre-school children with co-occurring features of P-SSD and DLD (“SWanS”- Supporting Words and Sounds), where expressive vocabulary and speech comprehensibility are joint outcomes of interest.
Methods
Forty-seven potential core intervention elements, based on a priori findings and the wider literature, were generated by a diverse steering group of professionals and people with lived experience within a systematic co-design process. This was followed by a modified, two round, e-Delphi with expert Speech and Language Therapists (SLTs) to achieve consensus on the elements. Consensus was defined as over 75% of participants (minimum 30 SLTs) rating the elements as either appropriate or very appropriate on a Likert of 1–5, with an inter-quartile range of one or below. If consensus was not achieved in round 1, free text comments were used to generate amended statements for the second round.
Results
Consensus was achieved on 42/47 statements in round 1. During the revision process, one statement was discarded; six statements which did not achieve consensus were re-worded; two statements which required further clarity had examples added; four statements were merged into two statements. Consensus was reached on 8/8 statements presented in round 2, resulting in 44 final statements achieving consensus in total.
Conclusions
Core elements of a novel intervention have been identified through co-design with a diverse group of stakeholders followed by consensus with expert SLTs. Additional flexibility was required within some core elements in order to achieve consensus. Implications for future implementation are discussed.
... In developing the first Chinese evaluation framework for GenAI in ECE, three experts in the fields of ECE and educational technology convened to analyze children's educational needs, as well as existing evaluation frameworks identified in the literature review. An expert panel approach-widely recognized in instrument development and framework construction (Hsu & Sandford, 2007;Okoli & Pawlowski, 2004)was employed, involving multiple rounds of discussion and iterative refinement until consensus was reached. Through extensive discussions, they reached a consensus on three key dimensions for evaluating GenAI platforms-educational adaptability, technical performance, and ethical safety-with assigned weights of 40%, 30%, and 30%, respectively. ...
... With the exception of the response speed metric, scores for all other indicators were derived from qualitative assessment. To ensure rigor in this qualitative scoring, an expert panel approach was employed (Hsu & Sandford, 2007;Okoli & Pawlowski, 2004). This involved multiple rounds of discussion and iterative refinement to achieve consensus on the scoring rubrics and application. ...
Generative artificial intelligence (GenAI) offers transformative potential for early childhood education (ECE), yet concerns remain regarding its suitability and ethical implications for young children. This study evaluated 10 freely available Chinese GenAI platforms for suitability in ECE, considering technical performance, pedagogical adaptability, and ethical/safety considerations. Using mixed methods (quantitative scoring and qualitative content analysis), platforms were assessed on multimodal support, response speed, teaching content generation, activity organization, personalization, ethical compliance, content appropriateness, and algorithmic fairness. Results showed significant variability in performance, with some platforms excelling in technical aspects while others demonstrated stronger pedagogical adaptability. Doubao and iFlytek Spark demonstrated strong technical performance, while DeepSeek excelled in pedagogical adaptability. However, all platforms presented areas for improvement, particularly in multimodal support and the transparency of ethical guidelines. This research offers a novel framework for evaluating GenAI in ECE, which allows for a more nuanced and context-specific assessment compared to existing frameworks. This framework highlights the need for specialized ECE datasets, algorithmic transparency, and robust ethical guidelines to protect young learners. Findings provide practical guidance for educators, developers, and policymakers navigating GenAI's potential in early childhood development.
... Consensus-based methods, such as the Delphi process and group concept mapping (GCM), are commonly used to develop quality appraisal tools by systematically gathering expert input [8][9][10][11]. The Delphi method relies on iterative, anonymous surveys to achieve consensus. In contrast, GCM enables participants to contribute to both idea generation and the organisation of concepts, making it valuable for capturing collective thinking on complex, multidimensional issues [10,12]. ...
... The Delphi method relies on iterative, anonymous surveys to achieve consensus. In contrast, GCM enables participants to contribute to both idea generation and the organisation of concepts, making it valuable for capturing collective thinking on complex, multidimensional issues [10,12]. While concept mapping in educational contexts is often used to visually organise and communicate knowledge [13], in participatory research, GCM is a structured, multi-step process designed to systematically collect, analyse, and visually represent the perspectives of a group to support consensus-building or conceptual framework development [14]. ...
This study developed and validated the Spatial Methodology Appraisal of Research Tool (SMART) using group concept mapping with discipline experts. The 16-item tool comprises four domains: (1) methods preliminaries, (2) data quality, (3) spatial data problems, and (4) spatial analysis methods. Validity testing demonstrated excellent content validity and expert agreement. Future studies will assess its usability and reliability to ensure consistent results. Its application in spatial epidemiology and health geography will enable more rigorous and transparent evidence synthesis. This contribution represents a significant step forward in improving the standards of quality appraisal in spatial research.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12942-025-00401-1.
... These statements were then refined and validated through the Delphi consensus technique, a well-established method for achieving expert agreement to support evidence-based decision-making. [39][40][41] This study was conducted and reported in accordance with the DELPHI-STAR reporting guidelines for Delphi studies in health research. 42 A completed DELPHI-STAR checklist is provided in online supplemental file 1. ...
... [44][45][46] Furthermore, panel members were given the opportunity to reflect on their individual ratings in round one and round two in conjunction with the group responses, which facilitated a thorough consideration of statements where consensus was not initially achieved. 39 However, some limitations remain. As with all Delphi studies, reproducibility can be limited due to the reliance on the specific experts selected for the panel. ...
Objective
Antimicrobial resistance is a significant global health challenge, exacerbated by unnecessary antibiotic prescribing. Respiratory tract infections (RTIs) are common reasons for antibiotic prescribing in primary care, despite most being viral or bacterial infections that are self-limiting. C-reactive protein (CRP) point-of-care tests (POCTs) are promising tools to support antibiotic stewardship by guiding the management of lower RTIs (LRTIs). The aim of this study was to develop best practice guidance for using CRP POCT in the management of LRTIs in primary care.
Design
Scoping review findings informed guidance statements, which were then evaluated through a three-round Delphi process with an expert panel via web-based questionnaires. Statements focused on intended use, detection of bacterial LRTIs, communication strategies, device features, performance and ease of use of CRP POCT.
Setting and participants
The panel of experts included 19 healthcare professionals across several specialties, including general practitioners, community pharmacists, hospital pharmacists and respiratory physicians.
Main outcome measures
Panellists rated each guidance statement using a 5-point Likert scale, with acceptance, revision or rejection determined using predefined cut-off scores for medians and interquartile ranges. Statements were revised between rounds using the feedback provided by panellists.
Results
In the first round, 49 statements were evaluated; 16 were accepted, nine removed and 24 revised for the second round. Of the 24 statements evaluated in the second round, 17 were accepted and seven were revised. In the third round, consensus was reached on four of the seven statements presented, resulting in 37 final guidance statements. These statements covered key areas, including the appropriate use of CRP POCTs to guide antibiotic prescribing, CRP cut-off values, integration with clinical decision rules, device performance and operational considerations, training requirements and financial reimbursement. The panel emphasised the need for structured guidelines to align CRP POCT use with clinical context and highlighted its role in improving diagnostic confidence while supporting antibiotic stewardship.
Conclusions
This study provides a set of best practice guidance statements to support the use of CRP POCT in the management of LRTIs in primary care. Dissemination and further research are required to assess their impact.
... 59 60 Given the gaps identified in the published literature and guidelines, a modified Delphi study with international clinicians was conducted with the aim of reaching consensus on the factors that should guide decisionmaking regarding whether, when, and how to retreat with anti-PD-(L)1 therapies after patients have already been treated with those therapies in an early-stage setting. 61 ...
... 62 It involves a group of clinical experts and a series of sequential questionnaires; after each round of questions, the responses are aggregated and shared with the group, which allows the clinical experts to adjust their answers based on how they interpret the group response. Our modified approach, while maintaining the traditional Delphi principles, 61 included more interaction between participants through real-time discussions using video conferencing. The first round of questionnaire responses was sent to a larger group of experts than just those participating in the consensus panels; participation in the panels was based on clinicians' willingness to participate, along with availability, and the consensus process was done in real time through a series of virtual meetings. ...
Anti-programmed death (ligand) 1 (anti-PD-(L)1) therapies were first introduced in the metastatic setting and have since been approved and reimbursed for treating early-stage cancers in the adjuvant, perioperative, and neoadjuvant settings in many cancer types. Current evidence supporting anti-PD(L)-1 retreatment after relapse with prior neoadjuvant and/or adjuvant anti-PD(L)1 therapy is limited and inconclusive. There is no guidance for clinicians on how and when to retreat with anti-PD-(L)1 therapies when anti-PD-(L)1 therapy was administered in the neoadjuvant and/or adjuvant setting. This study aimed to reach consensus on factors to guide decision-making regarding retreatment with anti-PD-(L)1 therapies after prior therapy with an anti-PD-(L)1 agent. This modified Delphi study consisted of a clinician survey across 10 countries followed by three real-time virtual Delphi panels involving clinical experts who had completed the survey. Clinical experts were experienced in using anti-PD-(L)1 treatments in early-stage cancers and/or as retreatment of patients with recurrences following early-stage treatment with anti-PD-(L)1 therapies. Of 28 clinicians providing survey responses, 20 participated in one of three Delphi panels. There was consensus that retreatment can be defined as ‘repeated treatment with the same therapeutic class following relapse after or during neoadjuvant and/or adjuvant treatment.’ All three panels agreed that decisions around retreatment should consider ‘prior immune-related adverse events/toxicity,’ ‘time-related factors’ (eg, time since completion of full treatment course and since discontinuation) and ‘previous patient response’ (often referred to by clinicians as tumor response, which may have reflected their experience with metastatic disease). Other factors identified as important included country-specific practices, treatment availability, and reimbursement. Generally, the clinical experts considered that retreatment could be considered from ≥3 to 6 months after stopping initial anti-PD-(L)1 treatment, or from ≥6 months after relapse/recurrence. In conclusion, clinicians across different regions recognized a role for retreating patients with anti-PD-(L)1 therapies after initial anti-PD-(L)1 treatment for early-stage cancers. Consensus was reached on some factors to consider regarding whether and when to retreat, although differences in clinical practice between countries/geographical regions made it difficult to achieve consensus for some more nuanced elements of retreatment. Further evidence could help better inform retreatment decisions.
... Our study used the modified Delphi method to enhance the construct validity of the EPAs by drawing on the collective expertise of field professionals. This consensusbuilding approach, as described by Hsu and Sandford (2007), is important in areas where professional judgment plays a significant role in defining professional skills [35]. Several studies reported the use of this method for development of EPAs frameworks either independently and in combination with other tools or methodologies [8,14,16,17,36]. ...
... Our study used the modified Delphi method to enhance the construct validity of the EPAs by drawing on the collective expertise of field professionals. This consensusbuilding approach, as described by Hsu and Sandford (2007), is important in areas where professional judgment plays a significant role in defining professional skills [35]. Several studies reported the use of this method for development of EPAs frameworks either independently and in combination with other tools or methodologies [8,14,16,17,36]. ...
Background
While competency-based education has gained prominence in preparing professionals for practice, clinical microbiology residency programs face a challenge in defining specific, observable tasks that align with Entrustable Professional Activities (EPAs). The current lack of a standardized set of EPAs tailored to clinical microbiology creates a gap in assessing learner proficiency and educational outcomes.
Objectives
This study aims to develop and validate a set of specific EPAs for clinical microbiology using a multi-step national expert consensus-building process.
Methods
This study was conducted in Egypt, involving experts from various medical schools across the country. As the first step, a thorough literature review was undertaken to identify potential EPAs pertinent to clinical microbiology residency programs. Then, evaluation of EPAs for quality and structure using EQual rubric involved five experts in medical education and clinical microbiology, resulting in the confirmation of relevant EPAs. Subsequently, three rounds of the modified Delphi method were employed, engaging ten clinical microbiology experts from various medical schools. Simultaneously, content validity was assessed based on these ratings. Participants also determined the appropriate year of entrustment for each EPA item, and an 80% Validity index agreement threshold was calculated to ensure consensus among participant groups.
Results
The use of the literature review and initial expert evaluation using EQual rubric confirmed 39 out of the initially identified 43 EPAs. Following the modified Delphi method rounds, 16 EPAs gained acceptance, signifying their relevance and appropriateness for clinical microbiology residency training. These EPAs were categorized into key areas, including preanalytical testing and quality assurance, microbiological techniques and diagnostics, infection control and safety practices, clinical leadership and teamwork, research and development, and laboratory management and communication.
Conclusions
This study developed 16 EPAs for clinical microbiology residency programs. These EPAs were developed using a robust multi step validation study. This provides a further step towards competency-based postgraduate training in clinical microbiology.
Clinical trial number
Not applicable.
... This exposure to the opinions of other participants aimed to enhance consensus (Mc Laughlin 1969). In Round 2, a cut-off median score of ≥ 3.25 was used to achieve consensus in both the physiotherapists and the users of physiotherapy services panels (Hsu and Sandford 2007). To minimize attrition during Round 2, regular reminders were sent to all invited physiotherapists. ...
... Our two-round Delphi study used a median cut-off score of ≥ 3.25 for both panels, which ensured that the perspectives of physiotherapists and users of physiotherapy services were equally considered in determining the "Top 10" research priorities. This criterion, supported by previous studies (Hsu and Sandford 2007;Rankin et al. 2012), highlights the value of combining professional knowledge with user experiences to create a relevant and balanced research agenda. However, this criterion was not used in previous physiotherapy research agendas (Nast et al. 2016;Rankin et al. 2012;Stevens-Lapsley et al. 2023). ...
Background
As physiotherapy research advances, identifying knowledge gaps and setting priorities is crucial for enhancing its efficiency in national and international collaborations. This study aimed to establish a physiotherapy research agenda in Portugal by integrating perspectives from physiotherapists and users of physiotherapy services.
Methods
An online two‐round Delphi study was conducted from April to August 2024 with two panels: physiotherapists and users of physiotherapy services. Participants rated 66 research priorities in the two rounds using a four‐point Likert scale (1 = not important, 4 = very important), with 13 new priorities introduced in the second round. These priorities were organized into nine research categories identified from a prior scoping review: (1) (cost)effectiveness; (2) service delivery; (3) education, professional development and quality; (4) measurement instruments; (5) mechanisms behind disability, physiotherapy treatments and patient classification; (6) patients' needs, expectations, experience and context; (7) prognostic outcomes and responses to therapy; (8) decision‐making strategies; and (9) technology and big data. Consensus was defined as a cut‐off median score of ≥ 3.25.
Results
A total of 479 physiotherapists and 70 users of physiotherapy services participated in Round 1 with final retention rates of 64% and 43%, respectively. The final ‘Top 10' priorities focused on three research categories: researching optimal service delivery models, structures, and processes ( n = 5); establishing the (cost)effectiveness of different physiotherapy interventions ( n = 3); and exploring the best models of physiotherapy education, professional development, and quality ( n = 2).
Conclusions
This study established a physiotherapy research agenda for Portugal, focusing on (cost)effectiveness, service delivery, education, professional development, and quality, integrating the perspectives of physiotherapists and users of physiotherapy services. The final priorities provide a foundation for future research and policy development.
... After each round, a facilitator summarized the responses, and participants were encouraged to revise their answers based on the group's feedback. This iterative process continued until consensus was achieved or no further changes were noted (13)(14)(15)(16). ...
The article outlines the development and adaptation of lifestyle medicine as a medical specialty tailored to the unique needs of Saudi Arabia and our (MENA) region. It also emphasizes the faith and the spirituality and highlights the importance of concepts like Al-Eiman, Al-Ehsan for this area of the world.
... 21 Consensus is obtained through a series of questionnaires delivered on multiple occasions to collect data from an expert panel of participants. 22 This Delphi process was led by a steering group of three academics and two paediatric dietitian specialists who sought guidance from the host organisations' Indigenous Health Nurse Navigator to ensure that the wording of the finalised Delphi statements was consistent with the Lowitja Institute 23 and other relevant national guidelines regarding research autonomy for Aboriginal and Torres Strait Islander Peoples. Subject matter experts (panel members) were identified and recruited from research institutes with active researchers in paediatrics, tertiary and quaternary paediatric hospitals across all States and Territories, community health services, government, non-government organisations and advocacy networks at a state and national level and research grant recipients from nationally competitive schemes. ...
Aim
The Australian Academy of Science's Decadal Plan for the science of nutrition emphasised research investment combined with system wide changes to improve the population's nutritional health. This study applied a Delphi consensus process with the aim to develop a definition for ‘paediatric nutrition research’ alongside identifying research themes and specific key priorities.
Methods
Multidisciplinary health professionals with expertise in paediatric nutrition were invited to participate in an on‐line survey. Panellists contributed to refining a new definition of paediatric nutrition research. Consensus was sought on nine priority areas under which more specific topics were listed, with panellists using a 7‐point Likert scale to rate priority statements. Approval was considered where agreement was reached by ≥70% of participants, with a ranked score ≥6.
Results
Participants ( n = 104) were invited with n =31 agreeing (response rate: 30%). The majority ( n = 28) also completed round 2. The proposed definition of paediatric nutrition research was agreed by 86% of first round respondents. This process generated 7 research themes containing 30 specific topic priorities. The themes were: growth and development; health equity and priority populations; food systems and food access; Aboriginal and Torres Strait Islander Peoples; strengthening prevention and treatment; empowering children, parents, and caregivers to maximise healthy development; and strengthening the workforce.
Conclusions
This study will support the prioritisation of critical nutrition research questions and inform the skills and expertise required within cross‐disciplinary teams needed to tackle the identified priorities. Consultation with Aboriginal and Torres Strait Islander leaders and consumer representatives is required to understand the specific needs of priority communities.
... The study population included specialists and subspecialists from four fields of (1) Pediatric diseases, (2) pediatric allergy, (3) pediatric gastroenterology and (4) health information management working in educational and medical centers associated with Kerman University of Medical Sciences. As in utmost Delphi-based research, the number of experts has been frequently 15-20 [17], 15 specialists and subspecialists in the aforementioned fields were chosen to partake in the study. The following inclusion criteria applied to choose participants: ...
Background and Aims
Cow's milk protein allergy (CMPA) occurs when the body recognizes the protein in cow's milk as a foreign substance and attacks it. This study aims to design a minimum data set (MDS) for CMPA in children, as a prerequisite for implementing related registries and applications.
Methods
Initially for this descriptive cross‐sectional study, a literature review was performed. Then the required data elements were obtained from the studies and a questionnaire was designed. Subsequently, to approve the data set, the designed questionnaire was distributed to 20 relevant specialists throughout a two‐round Delphi technique. To analyze the data, descriptive statistics were used.
Results
Initially 137 data elements were obtained in five categories. After performing the primary round of Delphi, 108 data elements were accepted, and 29 data elements entered the second round of Delphi. Then, 12 data elements were accepted, and 17 data elements were removed. Totally, the final accepted data elements reached 120.
Conclusion
Determining a qualified MDS for CMPA in children is the most important step to implement health information systems such as registries. It can provide standardized data elements helping health managers identify the necessary information to create a successful system, also for epidemiologists, researchers and health policy makers obtaining the necessary statistics and information.
... Expert feedback was analyzed using key indicators, including the median (Mdqi), interquartile range (Qqi), mean (Mqi), and variance (Vqi), with Vqi representing the percentage of experts who modified their ratings between rounds. This analysis was conducted following the guidelines presented in Hsu and Sandford [35]. After the first round of evaluation, four questions failed to meet the criteria due to having Mqi>3.5 but Qqi≥0.75, indicating a lack of consensus among the experts. ...
span lang="EN-US">The Central Highlands (Vietnam) harbors a diverse array of traditional musical instruments belonging to 13 ethnic minority groups. However, this invaluable cultural heritage faces the imminent threat of being forgotten and lost due to the lack of generational transmission among the youth. Consequently, the preservation and development of these instruments have become a critical priority for the education sector. This study implemented context-based learning (CBL) approach in teaching the topic of sound-natural science in 7th-grade, to encourage students to engage in addressing issues related to the preservation of traditional musical instruments within their local communities. Through observations and in-depth interviews with nine “E De” ethnic minority students, the study found that students exhibited a strong interest in learning to craft and play traditional instruments. They recognized the importance of this learning in contributing to cultural preservation and demonstrated the ability to apply scientific knowledge effectively in the crafting and utilizing of these instruments. These findings underscore the potential of CBL as a powerful educational model for teaching cultural heritage in other countries. This approach not only deepens students’ understanding of cultural values and social responsibility but also fosters mutual understanding and respect among different cultures on a global scale.</span
... The key screening criteria are summarized in Table 1. Based on the panel sizes used by other modified Delphi studies, the panel size (n = 17) is sufficient to draw robust conclusions and reach consensus on telehealth in PAH [14][15][16]. ...
Telehealth utilization increased during the coronavirus disease‐2019 (COVID‐19) pandemic, which encouraged remote patient management. The identification of optimal strategies to enhance telehealth and address barriers to its use for patients with pulmonary arterial hypertension (PAH) may improve patient outcomes. Physicians (n = 11) and advanced practice providers (n = 6) based in the United States with experience with telehealth in PAH were recruited to a double‐blinded modified Delphi panel (two survey rounds and a virtual consensus meeting). Consensus was defined as ≥ 80% of panelists rating their agreement or disagreement using a 9‐point Likert scale. A consensus in agreement was reached that telehealth is the use of virtual or remote methodologies to interact with, monitor and assess patients and deliver health care. Telehealth methodologies may be used in PAH for returning visits, diagnostic test follow‐up and medication management; telehealth is convenient, enables additional visits, and provides patient access to care. There are barriers to telehealth such as socioeconomic status, patient digital literacy, visual and/or hearing impairments, developmental disabilities, geographical location, and reimbursement. Solutions to improve patient access to telehealth include health insurance reimbursements, financial resources for healthcare professionals (HCPs), and improved connectivity infrastructure for patients. There are benefits, barriers and solutions for telehealth in general, and more research is required to evaluate the impact on PAH care. Further research and insights from the present study may improve PAH management and telehealth services. Improvements in telehealth care and delivery may allow better monitoring of patients with PAH and assist in more equitable PAH care.
... Despite the wide range of personal and professional expertise on our existing AASPIRE Outcomes Project Team, we felt that for this phase of the project, it was important to use the expertise of an even broader group of people who may not normally be a part of our CBPR partnership, be it due to time constraints, differing ideological perspectives, interest, professional roles, or other factors. To incorporate the perspectives of this broader group, we used a "modified Delphi process," a technique that researchers commonly use to reach consensus among experts on challenging topics (Hsu & Sandford, 2007). ...
People are increasingly recognizing the need for service interventions to improve the lives of autistic adults. However, less is known about how to best evaluate such services. We aimed to identify (1) which self-reported outcomes are most important to measure when evaluating the effectiveness of services for autistic adults and (2) what survey instruments would be needed to measure them. We nested a traditional researcher-driven “Delphi process” within our community-based participatory research approach in what we are calling a “CBPR-Nested Delphi Process.” The process allowed us to reach a full consensus among 53 experts with professional and lived experience as autistic adults, family members, health and disability service providers, autism community leaders, and researchers. The final list of outcomes included quality of life, overall health, emotional wellbeing, anxiety, depression, suicidality, autistic burnout, social support, employment satisfaction, community participation, self-determination, access to communication, activities of daily living, satisfaction with social services, and satisfaction with healthcare services. Experts felt almost all available instruments to measure these outcomes would need adaptations to be used with autistic adults (or proxies). Researchers and service providers should consider targeting interventions to these measurable outcomes and evaluating them using instruments that have been co-developed with autistic adults.
Lay abstract
Why was this project done?
People are starting to recognize the need for services to improve the lives of autistic adults. But less is known about how to best evaluate such services.
What were the goals of the project?
To identify (1) which outcomes are most important to measure when evaluating the effectiveness of services for autistic adults and (2) how we can successfully measure them using surveys.
What did the researchers do?
We used a method called a “Delphi process” that gets input from lots of different experts. We used that method inside our own long-standing community-based participatory research (CBPR) process so that we could share power between the academic and community members of our team. We reached a full consensus (agreement) among 53 experts. These experts had professional and/or lived experience as autistic adults, family members, health and disability service providers, autism community leaders, and researchers.
What does this study add?
The final list of outcomes included quality of life, overall health, emotional wellbeing, anxiety, depression, suicidality, autistic burnout, social support, employment satisfaction, community participation, self-determination, access to communication, activities of daily living, satisfaction with social services, and satisfaction with healthcare services. Experts felt almost all available surveys that try to measure these outcomes would need adaptations to be used with autistic adults (or if needed, with their caregivers).
What are the implications?
Researchers and service providers should consider targeting services to these outcomes. They should evaluate the effectiveness of services using surveys that have been created with and for autistic adults.
... Dalkey and Helmer (1963), pioneers of the Delphi method, suggested that a group of 10 to 15 experts may be sufficient. Similarly, Hsu and Sandford (2007) stated that a panel size between 10 and 50 participants is generally appropriate for Delphi studies and that this range may be adjusted depending on the purpose of the study. Okoli and Pawlowski (2004) also noted that panels consisting of 10 to 18 members are commonly used, although larger groups ranging from 20 to 30 participants may be preferable for achieving greater diversity of perspectives. ...
The aim of this study is to develop a needs analysis scale for the Out-of-School Learning Environments (OSLE) course. An exploratory sequential mixed-methods design was used in the study. The qualitative phase of the study involved 34 experts, while the quantitative phase included 255 university students. In the first stage, the Delphi technique was utilized to determine the items of the Out-of-School Learning Environments Needs Analysis Scale. In the second stage, validity and reliability analyses were conducted. Specifically, Kaiser-Meyer-Olkin (KMO) coefficient, Bartlett's Test of Sphericity, Exploratory Factor Analysis (EFA), Cronbach's Alpha, item-total correlation, and Confirmatory Factor Analysis (CFA) were performed sequentially. Following Exploratory Factor Analysis (EFA), the final 42-item scale structure of Out-of-School Learning Environments Needs Analysis Scale (OSLENAS) was examined, revealing a two-factor structure. The Cronbach’s Alpha coefficient for the entire scale was calculated as 0.95, with the first sub-dimension ("Learning Outcomes") yielding 0.95, and the second sub-dimension ("Course Content") yielding 0.94. When these reliability coefficients were further evaluated and tested using Confirmatory Factor Analysis (CFA), it was confirmed that the Out-of-School Learning Environments Needs Analysis Scale has a two-factor structure. The scale serves as a valuable instrument for conducting the needs analysis of the OSLE course, providing evidence-based insights for curriculum planning and instructional improvements and assessing the competencies of education faculty students regarding their knowledge and skills required for out-of-school learning.
... Different organisms present in water bodies have different saprobic rates and this principle was the basis for the determination of SI Lumbet al., (2011). The SI is the measure of the level of organic pollution in water.Most of the WQI model components have been developed based on expert views and local guidelines, therefore, many models are therefore region-specific (see Hsu and Sandford, 2007;Saidet al., 2004) The primary aim of this work is to critically review the most commonly used WQI models in different regions of the world ( Table 1). The review identified seven basic WQI models from which most other WQI models have been developed. ...
The water quality index (WQI) is a statistical tool for evaluating water quality which converted a large data set in to single numerical value. Most of the WQI models involves four successive steps viz. selection of parameters, estimation of values, generation of sub-indices for each parameter, calculation of the parameter weighting values, and aggregation of sub-indices to compute the overall water quality index. In this paper a study has been carried out in the genesis and evaluation of WQI model and the statistical techniques used in different models. The uncertainty in the conversion of large amounts of water quality data into a single index associated with different aggregation methods also discussed here. Finally a comparative discussion of the most commonly used WQI models viz. Horton's WQI, NSF WQI, SRDD-WQI including the different model structures and its components.
... In this study, first-time parents, who stayed on the postpartum ward after birth, were considered as experts in the topic investigated, due to their lived experience. A modified Delphi method was employed: the traditional open-ended questions in Round 1 for item generation (Keeney et al., 2011;Hsu and Sandford, 2007) was replaced with a pre-existing set of items sourced from the existing literature and qualitative findings from one of the studies described above (Schobinger et al., 2024). This approach allowed for the compilation of a list of professional support behaviours, thereby ensuring a more focused and relevant starting point. ...
... The answers from round I were already very complete concerning the gait patterns and impairments that were responsible for certain gait patterns. Earlier research indicated that 3 Delphi rounds are often enough to gather the information and to reach consensus (32). Indeed, in our study, we used 3 rounds, leading to consensus on most items. ...
Objective
This study aims to identify characteristic gait patterns in people with multiple sclerosis, to describe their key characteristics, and to identify their potential underlying causes.
Design
a 3-round Delphi study.
Participants
An international panel of 20 experts, including physiotherapists, a neurologist, rehabilitation physicians, biomechanical engineers, and movement scientists with expertise in multiple sclerosis or gait analysis.
Methods
A comprehensive list of gait characteristics and underlying impairments was compiled and analysed to identify common gait patterns and their primary features and potential causes. Consensus was defined as 67% agreement.
Results
Consensus was reached on 6 gait patterns in multiple sclerosis: (i) drop foot; (ii) insufficient push-off; (iii) stiff knee during swing; (iv) knee hyperextension during stance; (i) knee flexion in midstance; and (vi) enhanced gait variability. At least 69% agreement was achieved on the naming of the final gait patterns, their key characteristics, and the potential causes of each pattern.
Conclusion
Consensus was achieved on 6 gait patterns, their characteristics, and potential underlying causes. The identification of these gait patterns may support clinical decision-making regarding diagnostic and treatment measures, and deepen understanding of impairments that underlie walking problems in people with multiple sclerosis.
... To achieve this objective, our study used the Delphi method, an evidence-based approach to deriving expert consensus on complex topics through iterative surveys 17 . This method was selected for its capacity to integrate diverse forms of academic and experiential knowledge 18 , to facilitate discourse across vast geographical regions 19 and to enable the productive exchange of knowledge across power differentials and hierarchies by preserving anonymity 20 . To guide future research, the study also aimed to evaluate how set and setting are currently conceptualized in the field. ...
Psychedelic substances exhibit complex interactions with the ‘set and setting’ of use, that is, the mental state of the user and the environment in which a psychedelic experience takes place. Despite these contextual variables’ known importance, psychedelic research has lacked methodological rigor in reporting extra-pharmacological factors. This study aimed to generate consensus-based guidelines for reporting settings in psychedelic clinical research, according to an international group of psychedelic researchers, clinicians and past trial participants. We conducted a Delphi consensus study composed of four iterative rounds of quasi-anonymous online surveys. A total of 89 experts from 17 countries independently listed potentially important psychedelic setting variables. There were 770 responses, synthesized into 49 distinct items that were subsequently rated, debated and refined. The process yielded 30 extra-pharmacological variables reaching predefined consensus ratings:i.e., ‘important’ or ‘very important’ for ≥70% of experts. These items compose the Reporting of Setting in Psychedelic Clinical Trials (ReSPCT) guidelines, categorized into physical environment, dosing session procedure, therapeutic framework and protocol, and subjective experiences. Emergent findings reveal significant ambiguities in current conceptualizations of set and setting. The ReSPCT guidelines and accompanying explanatory document provide a new standard for the design and documentation of extra-pharmacological variables in psychedelic clinical research.
... The researchers selected 10 experts to complete the questionnaire developed. The number of experts for studies using the Fuzzy Delphi Method (FDM) is based on previous empirical research, Hsu and Sandford (2007) suggest that the ideal number of experts for the FDM is between 10 and 15, as this is sufficient to obtain a variety of opinions without overburdening the consensus process. The selection was based on their experience of more than five years in relevant areas according to Jamil et al. (2020). ...
This study aims to develop a Tadabbur al-Quran Model integrating Augmented Reality (AR) technology to enhance Arabic vocabulary learning for hearing-impaired Muslim adults. Conducted in collaboration with Persatuan Orang Pekak Islam Malaysia (PRISMA), a national NGO for the Deaf Muslim community, the research built upon a previous needs analysis findings to design a structured learning model. A questionnaire was developed to gather expert opinions on the model's * Corresponding author: Ummu-Hani Abas; ummuhani@usim.edu.my 251 http://ijlter.org/index.php/ijlter key components, focusing on Arabic vocabulary learning and AR's potential in Quranic education. It was distributed to 10 experts in Arabic Education, Quranic and Special Needs Education, Educational Technology, and Model Development. The Fuzzy Delphi Method (FDM), using Fuzzy Delphi Analysis V1.5, analysed expert responses to refine the model. The findings showed consensus on all components, meeting three key Fuzzy criteria: the threshold value (d) was ≤ 0.2, the expert agreement percentage was ≥ 75%, and the Defuzzification (alpha cut) value was ≥ 0.5. This study contributes to fostering a more inclusive and effective learning environment for hearing-impaired learners by advocating for the integration of interactive and visually supportive tools, such as AR books and digital sign language resources, into Arabic vocabulary education.
... However, the effectiveness of the approach depends on how well the stimulus question is formulated, and it is imperative that the researcher is clear about the information they hope to obtain from the procedure. In their 1975 study, Delbecq, Van de Ven, and Gustafson contrasted NGT with Fuzzy FDM [20]. ...
Traditional auditory-based teaching approaches limit the effectiveness of practical skills acquisition for Deaf or Hard-of-Hearing (DHH) students in Technical and Vocational Education and Training (TVET). Despite increased interest in immersive technologies like augmented reality (AR), the field lacks validated, inclusive instructional models tailored to DHH learners. This study addresses this gap by integrating the Nominal Group Technique (NGT) and Fuzzy Delphi Method (FDM) to design and validate an Immersive Learning Practical Skills (ILPS) model. The novelty lies in the combined use of NGT and FDM for consensus-building among experts in AR, gamification, and DHH education-an approach not commonly applied in inclusive model development. Results revealed a strong expert consensus (>97%) on 15 items across three core constructs: Learning Input Medium, Practical Skills Module, and AR Gamification Features. This research offers a replicable and participatory model development process and introduces a validated framework for inclusive immersive learning in TVET. The study contributes new knowledge by demonstrating how expert-driven methods can operationalize inclusive pedagogy through immersive technologies. This study demonstrates how combining FDM and NGT may successfully evaluate inclusive design elements for immersive learning. The results support the development of a practical skills model with a DHH focus and provide a repeatable framework for inclusive curriculum co-creation. This combination strengthen consensus among 11 panel of experts and according to the study's findings, the NGT and FDM approach has made it simple and quick for researchers to confirm crucial details that should be highlighted. To help DHH students learn more effectively, it is advised that more research be done in collaboration with course designers. To provide a scalable approach for developing immersive, accessible learning environments in specialized educational contexts, this study hopes to demonstrate how effectively NGT and FDM collaborate for inclusive instructional design.
... (3) possession of an intermediate or higher title; (4) had a master's degree or higher, or a master's supervisor or higher; and (5) willingness to participate in the study, including completing expert consultation. Due to time and budget constraints, we enlisted 21 panel members as recommended by the literature [35]. Experts from Phase 1 were excluded from the Delphi rounds. ...
Background
Dental hygienists are professionals engaged in oral disease prevention, oral health care, and auxiliary diagnosis and treatment. Despite the continuous improvement of dental services, the current training model for dental hygienists in China may not be able to meet the needs of dental diagnosis and treatment in China in the next two to three decades. It is urgent to carry out continuing education training for dental nurses currently performing roles similar to those of dental hygienists so that they can truly address the public oral health needs during the transition period of dental hygienists cultivating in China. Therefore, the aim of this study was to construct a core knowledge system for dental hygienist training to provide a theoretical basis for continuing education.
Methods
We used a modified Delphi study design. A literature review, theoretical research and expert panel discussion (n = 15) were used to construct an initial core knowledge system. Through two rounds of Delphi expert consultation (n = 21), consensus was reached using online or paper questionnaires, and descriptive statistical analysis was used. Consensus criteria were defined as the mean (≥ 4.0), coefficient of variation (< 0.25), and consensus level of agreement (≥ 70%).
Results
The final core knowledge system of dental hygienists training included 3 first-level indicators (theoretical knowledge, professional skills, and professional ethics and attitudes), 37 s-level indicators, and 56 third-level indicators. The effective response rates of the two expert consultation rounds were 100%. The expert authority coefficients of the first and second rounds of consultation were both 0.919. In the second consultation round, Kendall’s coefficient of concordance for the primary, secondary, and tertiary indicators were 0.333, 0.536, and 0.592, respectively (P ≤ 0.001).
Conclusion
The developed core knowledge system of dental hygienist training was scientific, reasonable and comprehensive, which can provide reference for the training and assessment of dental nurses to be competent for the position of dental hygienist.
... Qualitative data was gathered through stakeholder interviews and structured questionnaires. A modified Delphi approach was employed to gather expert insights on the deployment and performance of Carrier Aggregation in LTE networks [22]. Although only a single round of structured questionnaires and interviews was conducted, the method retained key Delphi principles such as expert selection, anonymity, and systematic data collection. ...
Carrier Aggregation (CA) is pivotal for enhancing LTE network performance in urban environments. This research investigated the practical impact of 2CA inter-band contiguous CA, combining 20 MHz and 10 MHz component carriers (CCs), on LTE networks in urban Zambia, focusing on downlink (DL) throughput and Physical Resource Block (PRB) utilization. While theoretical peak DL throughput for Category 6 User Equipment (UE) was 300 Mbps, this study examined real-world deviations and challenges faced by Mobile Network Operators (MNOs). Employing a mixed-methods approach, including drive and stationary tests, and analysis of network Key Performance Indicators (KPIs), the study assessed network performance before and after CA activation. Key challenges identified were spectrum availability and high spectrum auction costs. However, empirical data revealed significant end-user benefits. Peak user data rates reached 126.2 Mbps, a 129% increase over pre-CA conditions. Average throughput improved by 88%, from 36.17 Mbps to 68.14 Mbps.
Additionally, PRB utilization decreased from 44.30% to 41.40%, indicating enhanced network efficiency. The findings confirm CA’s crucial role in optimizing LTE performance and meeting escalating urban data demands. This research provides valuable insights into CA deployment, highlighting its contribution to network advancement and improved user satisfaction, demonstrating that CA is a viable solution for enhancing LTE networks in densely populated areas.
Keywords:
Carrier Aggregation (CA), Component Carriers (CC), Downlink (DL)
Throughput, Inter-Band Contiguous, Long Term Evolution (LTE), Mobile Network Operator (MNO), Physical Resource Block (PRB), User Equipment (UE)
... The Delphi technique is a systematic method, consisting of a series of surveys, to develop consensus amongst a designated panel of domain-specific experts. While there are no clear guidelines for the design of a Delphi study, typical elements include anonymity amongst the panel members, several iterative rounds, and the analysis of group results (33,34). In the present study, we used a modified e-Delphi approach, consisting of an electronic approach with a priori defined maximum of two rounds. ...
Introduction
Following the lack of widely implemented interventions to mitigate Relative Age Effects (RAEs) in sports, the Royal Netherlands Football Association (KNVB) called on stakeholders to propose relative age solutions in youth soccer (Part One). This initial study yielded 13 lower-order potential solutions, many of which remain hypothetical. Therefore, this study aimed to evaluate these solutions to overcome RAEs in youth soccer using a two-round adapted e-Delphi study.
Methods
Fifteen international experts, including both researchers and practitioners, rated (out of 9) each solution on how likely it is to directly and indirectly mitigate RAEs (Round 1) and how feasible it is to implement (Round 2).
Results
Findings indicated that “rotating cut-off dates” was perceived as the most effective solution to mitigate direct and indirect RAEs (6.2 ± 1.6), although it was not rated particularly feasible (4.6 ± 2.5). In comparison, while “cueing differences in age” was perceived as the most feasible solution (6.7 ± 2.1), it was deemed less useful for mitigating RAEs (5.2 ± 2.3). Taken together, “cueing differences in age” was considered the most viable solution across both rounds (5.8 ± 2.3).
Discussion
Interestingly, highly rated solutions perceived to effectively moderate RAEs were generally expected to be more challenging to implement. Results also showed regular disagreement amongst the international experts, highlighting that creating consensus on possible relative age solutions may be difficult to achieve in youth soccer. Moving forward, the highest rated solutions should be designed, implemented, and evaluated based on their effectiveness and feasibility in practice.
... Its emphasis on anonymity and the avoidance of face-to-face interaction minimizes hierarchical bias and peer pressure, encouraging independent, unbiased contributions (Cole, 2017). The Delphi method aims to define "what should be" through a structured, iterative process that refines expert consensus on key issues (Hsu & Sandford, 2007). ...
This study focuses on developing a consumer-centric Environmental, Social, Governance (ESG) evaluation scale for the aviation industry, highlighting the growing importance of corporate sustainability actions from a consumer perspective. With the aviation sector being significantly impacted by sustainability challenges, the need for a strategic approach to leverage ESG management for competitive advantage is evident. The research aims to fill the empirical gap regarding how consumers evaluate airlines' ESG performance, which is crucial for enhancing financial and non-financial value. Utilizing a Delphi survey and the Analytic Hierarchy Process, the study seeks to understand and integrate various stakeholders' perspectives, offering new insights into consumer attitudes towards ESG activities, and contributing to the discourse on sustainable and responsible business practices.
... This study aimed to gauge the level of consensus on a set of predefined statements using the Delphi methodology, following guidance from published Delphi reporting guidelines [10,11]. The Delphi technique is a structured communication method designed to achieve consensus or assess the degree of agreement among experts by using iterative rounds of questionnaires [12,13]. ...
Objective
The increasing availability of disease-modifying therapies (DMTs) may provide more personalized treatment options for multiple sclerosis (MS) based on various factors, including patients’ characteristics, prognostic indicators, comorbidities, and safety. In Italy, recent efforts focused on promoting interdisciplinary, patient-centered care and equitable access to optimized therapies, as reported in the 2023 Barometer of Multiple Sclerosis and Related Diseases from the Italian Multiple Sclerosis Association. A key challenge is ensuring equitable access to homogeneous and personalized therapeutic strategies.
Materials and methods
Using a Delphi methodology, a panel of Italian neurologists with expertise in MS evaluated consensus on specific aspects of MS treatments, including personalized therapy, patient involvement in decision-making, treatment flexibility, self-management of therapies, perception of treatment efficacy and safety and therapeutic sequence management.
Results
Of 166 votes, 116 statements reached consensus (68% positive, 2% negative), representing 70% of the total, whereas 50 (30%) highlighted areas of non-consensus. The findings emphasize the central role of neurologists, the importance of personalized therapy, the inclusion of patients in therapeutic choices to enhance adherence and quality of life, and managing both quality of life and caregiver burden. Most high-efficacy disease-modifying therapies (HE DMTs), like cladribine and anti-CD20 therapies, recognized for their efficacy and convenience of administration, received positive consensus, emphasizing their perceived value in individualized treatment approaches.
Conclusions
This research highlights best practices and provides a roadmap for improving patient outcomes through tailored, well-communicated therapeutic strategies.
... Factor labeling involves the notion of interpretive understanding of human subjects (i.e., researchers, a panel of experts) that are assigned to label the topics extracted. For instance, the labels of the topics extracted could be based on the development of a Thurstone scale [33,80] or the Delphi method [81,82]. To develop Thurstone scales, researchers initially assign preliminary labels for each topic based on the cluster of terms highly loaded (precoding), and then a group of experts are surveyed to evaluate the level of their agreement on the initially assigned labels. ...
Textual analysis is grounded in conceptual schemes of traditional qualitative and quantitative content analysis techniques that have led to the hybridization of methodological styles widely used across social scientific fields. This entry delivers an extensive review of the origins and evolution of text analysis within the domains of traditional content analysis. Emphasis is given to the conceptual schemas and operational structure of latent semantic analysis, and its capacity to detect topical clusters of large corpora. Further, this entry describes the operations of Entity–Aspect Sentiment Analysis which are designed to measure and assess sentiments/opinions within specific contextual domains of textual data. Then, this entry conceptualizes and elaborates on the potential of streamlining latent semantic and Entity–Aspect Sentiment Analysis complemented by Correspondence Analysis, generating an integrated operational scheme that would detect the topic structure, assess the contextual sentiment/opinion for each detected topic, test for statistical dependence of sentiments/opinions across topical domains, and graphically display conceptual maps of sentiments in topics space.
... The influence of authoritative experts was avoided (Okoli & Pawlowski, 2004). The Delphi method does not require the number of experts to be statistically representative, but rather focuses more on the quality of the expert panel (Hsu & Sandford, 2007). That is, it relies more on the experts' familiarity with the problem and their expertise. ...
... Panelists were given two weeks to complete the survey, and reminders were sent one week and one day prior to the submission deadline [17,18]. In round 2, the study team organized a virtual panel discussion using the Zoom platform. ...
... Prior literature indicates that Delphi panels commonly range from 8 to 20 participants (32). While larger panels can enhance stability by reducing the influence of individual experts, excessively large groups may introduce logistical challenges (33). Given these considerations, we aimed to secure a final panel within this recommended range. ...
Introduction
Burnout among physicians is a worldwide concern issue, impacting individual well-being and healthcare efficiency and jeopardizing the achievement of Universal Health Coverage. Using the Portuguese National Health Service scenario, we aimed at identifying the factors that contribute to burnout of physicians and interventions that can be helpful in its prevention.
Methods
We used a Delphi panel technique with three rounds of participation with 16 specialists, including physicians, psychologists, academics and hospital administrators.
Results
Organizational variables such system strain, staff overload, and unfavourable working circumstances were pointed as the main contributing factors of burnout. Consensus was reached that courses of action, such as resource allocation, legislative changes, and promotion of a healthier workplace environment can help prevent physician burnout. Non-organizational tactics such as workplace amenities and health literacy initiatives were also considered relevant. Identified knowledge gaps comprised long-term effects of burnout, leadership influence, and environmental repercussions.
Discussion
The study concludes that addressing organizational factors and implementing targeted interventions are crucial for improving physician well-being and aiding healthcare efficiency in Portugal but also in similar contexts.
... To improve future research, it is important to identify and agree on the most validated measure(s) for assessing well-being and burnout among homelessness staff. A Delphi exercise could facilitate consensus on the most appropriate measures for this group [33]. ...
Background
Homelessness staff often experience high job demands, limited resources, and significant emotional strains; with high levels of burnout, stress, and trauma being common within the workforce. Despite growing recognition of these issues, limited literature exists on interventions to address them. This study aims to conduct a systematic scoping review to map and identify interventions aimed at improving well-being and reducing burnout among homelessness staff.
Methods
All eligible studies needed to include an intervention addressing burnout and/or well-being in homelessness staff, published in English with primary data. Evidence sources were left open with no data restrictions. Following protocol registration, a systematic search of five electronic databases (Medline, APA PsychInfo, Global Health, ASSIA, CINAHL) and Google Scholar was conducted. Studies were double-screened for inclusion. Methodological quality was assessed using the Mixed Methods Appraisal Tool.
Results
Of the 5,775 screened studies, six met the inclusion criteria: two peer-reviewed and four non-peer-reviewed publications. No studies were retrieved from Google Scholar. The included studies comprised four quantitative non-randomised designs, one randomised controlled trial, and one mixed-methods study. All included studies were complex interventions. Three were therapy-based, two included supervision, and two were one-time educational sessions. Most were conducted in the United States (n = 4), with two in the United Kingdom. The total pooled sample was 347 participants, though four studies were missing demographic data (age and gender). The studies used heterogenous measures and outcomes. Limitations included restrictions to English-only publications, potential gaps in capturing well-being measures, and a limited grey literature scope.
Conclusion
There is a lack of research on well-being and burnout interventions in frontline homelessness staff. Identified studies were generally low quality, using heterogenous measures and outcomes to assess well-being and burnout, limiting the generalisability of findings. Future research should employ more robust study designs with standardised measures and outcomes.
Introduction
The conjunction of physical frailty and cognitive impairment without dementia is described as Cognitive Frailty (CF). Indications that CF is potentially reversible have led to proposals that risk factors, symptoms or mechanisms of CF would be appropriate targets for interventions for prevention, delay or reversal. However, no study has brought experts together across sectors to determine targets, content or mode of interventions, and most resources on interventions are from the perspective of academic or clinical researchers only. This international Delphi consensus study brings together experts from academic and clinical research, lay people with lived experience of CF, informal carers, and professional care practitioners/clinicians.
Methods
Three rounds of Delphi study were held to discern which factors and statements were agreed upon by the whole sample and which generated different views in those with differing expertise. A scoping review and Round 1 (29 participants) were used to gather initial statements. In Round 2, 58 people responded to statements and open text items, comprising 7 lab-based researchers, 27 researchers working with people, 14 people with lived experience or informal family carers, and 10 professional carers/clinicians. Percent agreement and qualitative responses were analyzed to provide a final set of statements which were checked by 38 respondents in Round 3.
Results
Analysis of Round 2 quantitative data provided 74 statements on which there was at least 70% agreement and qualitative data produced a further 24 statements. These were combined to provide 90 statements for Round 3. There was Consensus for 89 of the statements. A few differences between the groups were observed at both stages.
Discussion and conclusion
The consensus for statements associated with CF interventions provides a useful first step in defining health promotion activities and interventions. Given the prevalence and potential disability caused by CF in older populations, the consensus statements represent expert opinion that is inter-sectoral and will inform public health policies to support implementation of evidence-based prevention and intervention plans. This study is an important step toward changing current approaches, by including all stakeholders from the outset. Outcomes can be used to feed into co-creation of interventions for cognitive frailty.
In this study, clinical supervision within dance/movement therapy is considered in depth. The lack of and need for a consistent theory of supervision in dance/movement therapy is highlighted, including both supervision outcomes and common techniques and approaches used within the discipline. A Delphi method study is used to determine the consensus on the topic of dance/movement therapy supervision from a panel of experts. As a result of the study, the expert panel comes to a consensus of 71 statements regarding dance/movement therapy supervision. These statements are interpreted, dance/movement therapy supervision is defined, and connections are drawn to the value of body-based learning and specialized supervision for dance/movement therapists.
Focusing on issues related to SDG 11 (Sustainable Cities and Communities) and SDG 13 (Climate Action), this study aligns with the framework of the 2030 Agenda for Sustainable Development. This study explores the barriers unique to the industry that obstruct the adoption of low-carbon emission solutions in Jordan’s multi-family residential buildings. Multi-family apartments constitute 73% of the total housing stock and account for over 80% of all residential structures. A total of eight main barriers that are preventing the implementation of low-carbon emission techniques were evaluated. The Fuzzy Delphi Method was utilized to gather insights from the Consultancy Council members of the Jordan Housing Investors Association. The results suggest that a major obstacle is the insufficient knowledge among end-users regarding environmental concerns, along with financial limitations, resulting in a lack of enthusiasm for low-carbon multi-family apartments. Moreover, insufficient cooperation between consultants and contractors leads to subpar constructability, which is worsened by the prevailing conventional procurement method that prioritizes cost and schedule above environmental consequences. To further investigate, it is advisable to examine the utilization of contemporary procurement methods, such as Design–Build and Construction Management and modern family contracts such as NEC4 in the housing industry of Jordan. These alternative methods have the potential to solve the current difficulties by promoting more effective and environmentally friendly building practices.
Background
Occupational therapy practice focuses on occupation‐based interventions, considering the interaction between the person, environment, and task. In Australia, combining supervised therapy with self‐directed practice is feasible, even for health‐care consumers with cognitive impairments. This study aims to explore the self‐practice programs developed by occupational therapists and delivered in inpatient rehabilitation and develop core recommendations for such interventions.
Methods
Ethical approval for this Delphi consensus study was granted by Monash University Human Research Ethics Committee. The study involved an online survey and two focus groups with Australian occupational therapists. Participants were recruited via snowball sampling and required to meet specific experience criteria. Content analysis was used to analyse data, and consensus was reached on core recommendations for prescribing self‐practice in inpatient rehabilitation.
Consumer and community involvement
Occupational therapists who prescribe self‐practice to health‐care consumers were the community of interest and directly involved in the Delphi consensus process to inform study findings. Consumers using self‐practice activities were not included in the study.
Results
In Round 1 of the Delphi process, 21 occupational therapists participated in an anonymous online survey about prescribing self‐practice in inpatient rehabilitation. The survey identified key recommendations across various categories. Round 2 focus groups further refined these recommendations, and Round 3 achieved consensus, incorporating additional feedback and suggestions for implementing self‐practice programs. It found variability in implementation, with neurological and general rehabilitation health‐care consumers most likely to receive self‐practice tasks.
Conclusion
This Australian study explored how occupational therapists prescribe self‐practice during inpatient rehabilitation. The study emphasised the importance of clinical reasoning and environmental factors, offering recommendations to guide goal‐focused, client‐centred self‐practice interventions for better health‐care consumer outcomes.
PLAIN LANGUAGE SUMMARY
We looked at occupational therapists in Australia. We wanted to know how therapists gave self‐practice activities to people. Our focus was on people in recovery hospitals. People do self‐practice activities without a therapist. People do these activities outside therapy sessions. This study used a survey and focus groups. Skilled therapists agreed on ideas for self‐practice. The results showed differences in self‐practice methods. Self‐practice is common in stroke and general recovery. The study gave self‐practice tips for therapists. It included what, how, and why they prescribe these activities. Therapists should think about each person's goals. They should understand what helps self‐practice. We know what therapists are doing. We support therapists to include self‐practice programs in regular care.
Tourism plays a central role in Thailand’s economy, especially within its urban landscapes, but the rapid expansion of urban tourism presents significant challenges. From the lens of Easternisation and the broader context of the Asian Century, this study addresses the growing concerns of environmental degradation, sociocultural impacts, and governance issues linked to tourism. In response, we propose a Sustainable Tourism Index (STI) tailored for urban areas in Thailand, developed through the Delphi method. A panel of 25 experts reviewed 143 initial indicators across four rounds, ultimately validating 32 indicators covering economic, sociocultural, environmental, political, and technological dimensions. This framework offers a vital tool for assessing and improving sustainable tourism practices in Thailand’s urban regions, reflecting both Thailand’s strategic objectives and the United Nations’ Sustainable Development Goals (SDGs). As Asia continues to rise, the STI serves as a model for Easternisation in tourism, equipping policymakers with essential insights to foster sustainable tourism development in urban settings.
Introduction: There is an increased injury risk during pointe training if dance students are not physically or technically prepared. The objective of this study was to find consensus between dance experts pertaining to the most reliable screening protocols when determining pointe readiness in young adolescent female dance students. Methods: Dance experts (10+ years of professional dance ( pointe ) experience/teaching pointe experience/clinician to dancers), were recruited through multiple dance teacher/science groups on social media. The Delphi method was utilized via Online Surveys and distributed in rounds until consensus was found. Questions were created, and subsequent rounds were devised based on analyzed data from the previous round. Descriptive and thematic analysis was used to collate, rank, and analyze data. Consensus was determined at 60% agreement between the dance experts’ answers. Results and discussion: Fifteen dance experts completed Round 3 with 80% consensus. The most important physical attributes were ankle alignment, pelvic stability, ankle strength, ankle stability, and lower limb alignment. Screening measures consisted of medical history, teacher assessment, heel raise test, single leg plié , plantarflexion. Functional skills tests were heel raise test, single leg sauté , holding passé relevé on demi- pointe , balance, and pencil test. All agreed that teacher assessment is more reliable than a test battery. Overall, responses corresponded with statistically significant evaluators yet were not conclusive. Conclusion: Contradictory responses meant providing reliable outcomes to the objectives was challenging. Further research projects are needed to indicate any association between testing and teacher assessment for pointe readiness.
Background
In the United Kingdom, pregnant women are offered two scans: at 11–14 and 18–20 weeks’ gestation. Current guidance supports fetal anatomical screening at the second scan, but evidence suggests earlier detection is possible.
Objectives
To determine clinical and cost-effectiveness of a detailed two-dimensional ultrasound scan in the first trimester for detection of fetal anomalies, in addition to usual practice.
Design
Systematic review and meta-analysis.
Nationwide survey.
Analysis of National Congenital Anomaly Disease Registry data.
Consensus procedure.
Prospective survey of parental opinions.
Probabilistic decision-analytic model for cost-effectiveness.
Value-of-information analysis.
Setting
United Kingdom National Health Service.
Participants
Pregnant women and partners.
Interventions
Detailed anomaly ultrasound at 11–14 weeks’ gestation, in addition to usual practice.
Main outcome measures
Diagnostic accuracy, protocol development, health economic modelling and value-of-information analysis.
Data sources
MEDLINE (OvidSP), EMBASE (OvidSP), Science Citation Index and Conference Proceedings Citation Index-Science (Web of Science Core Collection); National Congenital Anomaly Disease Registry; European Congenital Anomalies Registry; Surveys of National Health Service Trusts; screening sonographers, midwives and doctors; and parents; National Schedule of National Health Service Costs (2019–20).
Review methods
Systematic review and meta-analysis for diagnostic accuracy.
Results
First-trimester ultrasound detects 93.3% (95% confidence interval 90.4% to 95.7%) of a pre-selected group of eight major anomalies with specificity of 99.99% (95% confidence interval 99.98% to 99.99%) and positive predictive value of 96.5% (95% confidence interval 93.3 to 98.8, 416,877 fetuses, 40 studies). For major cardiac anomalies, the respective data are 55.8% (95% confidence interval 45.9% to 65.5%), 99.98% (95% confidence interval 99.97% to 99.99%) and 94.85% (95% confidence interval 91.63% to 97.32%, 306,872 fetuses, 45 studies). Of NHS trusts surveyed, 77% currently perform first-trimester anatomy assessment, with evidence of inequity of care; earlier screening resulted in more diagnoses before 16 weeks’ gestation. A consensus procedure ( n = 172) developed an anatomical protocol and minimum targets for diagnosis. Parental survey ( n = 1374) indicated that over 90% would opt for such screening. Modelling of singleton pregnancies undergoing earlier anomaly screening using two-dimensional ultrasound was associated with increased mean healthcare costs per woman (£11, 95% confidence interval £1 to £29) and maternal quality-adjusted life-years (0.002065, 95% confidence interval 0.000565 to 0.00358), an incremental cost per quality-adjusted life-year of £5270, with likelihood of being cost-effective at £20,000 per quality-adjusted life-year of over 95%. Additional modelling predicted reductions in infant healthcare costs and quality-adjusted life-years. Decision uncertainty was low. Value-of-information analysis of cost-effectiveness results showed no groups of parameters for which further research to reduce uncertainty would likely prove cost-effective.
Limitations
Study heterogeneity; the lack of a universal reference standard; simplifying assumptions relating to economic model structure; and estimation of some parameters are documented and justified. The rarity of the conditions made estimation of longer-term maternal and infant costs and quality-adjusted life-years challenging, resulting in likely under-estimation of healthcare costs.
Conclusions
With standardisation and training, first-trimester ultrasound screening for fetal anomalies is clinically effective with over 90% detection for eight major conditions and low false-positive rates. Decision uncertainty around implementation is low and a prospective study would not be an efficient investment. Adding first-trimester anomaly screening to the current screening likely represents a cost-effective use of resources and is acceptable to parents.
Future work
Focus on developing an implementation framework to modify the current United Kingdom Fetal Anomaly Screening Programme.
Study registration
This study is registered as PROSPERO CRD42018111781 and CRD42018112434.
Funding
This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/19/10) and is published in full in Health Technology Assessment ; Vol. 29, No. 22. See the NIHR Funding and Awards website for further award information.
Background
Stepped care (SC) is being adopted in many countries as a framework for organising mental health care in diverse contexts. However, there is a lack of consistency in how SC has been defined and operationalised, limiting its effective application in practice. We describe the development of standards for implementing SC models in Canadian child and youth mental health (CYMH) contexts using a consensus‐based approach. These standards are intended to support systems planners in creating cohesive CYMH systems across Canadian settings.
Methods
This study employed learning alliance and Delphi methodologies. A pan‐Canadian multi‐round Delphi process conducted in English and French was used to derive consensus on the inclusion and wording of individual clauses in the standard. Consensus with a threshold of 70% was set to determine the inclusion of individual clauses in the final standard.
Results
Sixty‐eight individuals participated in the Delphi study (with a 76.48% retention rate) representing lived experience, service delivery, policy, and research expertise. Over three rounds, 29 clause items were revised and reduced to a final list of 24 clause items comprising SC implementation standards. Participant feedback indicated a desire for reduced ambiguity, considerations of the limitations of patient autonomy, and the need to clarify roles and responsibilities in system‐wide activities.
Discussion
The results of this Delphi study represent the first multi‐stakeholder, consensus‐driven set of standards for implementing SC in CYMH settings across Canada. With these standards, we aspire to provide a blueprint for mental health systems advocacy and reform toward stronger, more coordinated CYMH systems.
Objective
Current treatment for skull base chordomas utilizes both surgical resection and adjuvant radiation, but recent studies have demonstrated evidence that has brought the use of adjuvant radiotherapy into question. Chordomas differ greatly in molecular makeup and proliferation. These factors have led to significant variation in management across providers. We used a modified Delphi approach to work towards consensus on standardized operative definitions and evidence-based management of chordomas.
Methods
Our multidisciplinary panel included participants representing the AANS/CNS Tumor Section and North American Skull Base Society (NASBS) with a track record of publishing on chordoma management. Our approach involved a four-step process: one statement-generation round, two voting rounds to establish consensus and refine statements, and a final external validation round by NASBS members. Anonymous voting was completed via Qualtrics surveys.
Results
The statement-generation process produced 65 statements. Through the Delphi process, 36 statements reached consensus during the first round and an additional 17 were refined for further consensus in the second round. Moderate (67-80%) or strong (> 80%) consensus was achieved for 43 final statements. Forty-one items were externally validated. There was consensus that an endoscopic endonasal approach should be utilized whenever possible. They defined the tumor characteristics (molecular and cytogenic) to consider before offering adjuvant radiotherapy.
Conclusions
This modified Delphi study generated consensus on 41 statements regarding skull base chordoma management. These statements aim to shed light on the consensus among providers regarding the use of surgery, neoadjuvant radiation, adjuvant radiation, adjuvant systemic therapies, and treatment of recurrence for chordoma.
Research guidelines for the Delphi survey technique
Consensus methods such as the Delphi survey technique are being employed to help enhance effective decision-making in health and social care. The Delphi survey is a group facilitation technique, which is an iterative multistage process, designed to transform opinion into group consensus. It is a flexible approach, that is used commonly within the health and social sciences, yet little guidance exists to help researchers undertake this method of data collection. This paper aims to provide an understanding of the preparation, action steps and difficulties that are inherent within the Delphi. Used systematically and rigorously, the Delphi can contribute significantly to broadening knowledge within the nursing profession. However, careful thought must be given before using the method; there are key issues surrounding problem identification, researcher skills and data presentation that must be addressed. The paper does not claim to be definitive; it purports to act as a guide for those researchers who wish to exploit the Delphi methodology.
The Examination of Anomalous Self-Experience ( EASE) is a symptom checklist for semi-structured, phenomenological exploration of experiential or subjective anomalies that may be considered as disorders of basic or 'minimal' self-awareness. The EASE is developed on the basis of self-descriptions obtained from patients suffering from schizophrenia spectrum disorders. The scale has a strong descriptive, diagnostic, and differential diagnostic relevance for disorders within the schizophrenia spectrum. This version contains interview-specific issues and psychopathological item descriptions ( Manual), a scoring sheet ( Appendix A), a reminder list of items for use during the interview ( Appendix B) and an EASE/BSABS ('Bonner Skala fur die Beurteilung von Basis-symptomen') item comparison list (Appendix C).
This paper describes a research design aimed at examining the risks and impacts of natural and anthropogenic catastrophic events on the future of the tourism industry. The focus is on the selection of a survey method, the Delphi technique, and the reasons behind selecting this qualitative forecasting tool. The Delphi technique relies primarily on the intuitive judgment of experts. This paper argues that with thorough pre-planning and rigorous management of the survey, the Delphi technique can provide futures information that other conventional extrapolative methods cannot reliably forecast.
This paper gives an account of an experiment in the use of the so-called DELPHI method, which was devised in order to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in depth interspersed with controlled opinion feedback.
Focusing on the general issues of reliability and validity, several specific problems evident in reported applications of the Delphi method to forecasting are discussed. Reliability threats, which are noted in a number of procedural aspects of the method, arise from ill-considered procedural variations and lack of standardization. While validity threats are also found in several procedural aspects of the tool, they arise principally from pressures for convergence of predictions. This feature of the method, along with certain structural characteristics, is found to undermine critically its forecasting ability. Having discussed in some detail the nature of these difficulties, the paper closes with consideration of the reasons for the continued use of Delphi in spite of its shortcomings and with comments on alternative approaches.
Recently, Keith Zoski and Stephen Jurs proposed a modified version of the scree test for analyzing Delphi survey data. Although the test has merit, three considerations or problems affecting its use are examined and examples of data related to those problems are given. Discussion is also provided regarding appropriate and inappropriate applications of the modified scree test to Delphi data.
This paper is concerned with describing the principal uses of the Delphi method in higher education. These areas of use are found to be (1) searching out goals and objectives, (2) campus and curriculum planning, and (3) development of evaluation criteria. Important to understanding the use of Delphi in a higher education setting are the methodological adaptations that have been employed. Delphi as used in technological forecasting requires modification when used in the generation and assessment of goals. Three principal problems in methodology are examined here. They are (1) selection of the panel, (2) the character of round one and (3) the consequences of feedback.
This report describes the results of an extensive set of experiments, conducted at RAND, concerned with evaluating the effectiveness of the Delphi procedures for formulating group judgements. The study represents a small beginning in the field of research that could be called ‘opinion technology’, and is of direct relevance for the use of experts as advisers in decision making, especially in areas of broad or long-range policy formulation.
Describes the Delphi Technique, an approach to elicit and refine the opinions of a group of people. Outlines the steps in the process and provides illustrations of the applications of the technique for schools and colleges in determining future research needs, assessing staff development needs, and identifying problems confronting administrators. (MB)
Leading professionals in the field discuss instructional design and systems, computer applications in education and training, research and evaluation in instructional technology, future prospects for instructional technology, and professional development. The book contains six parts. A brief outline of instructional development and educational technology is given in part 1. This is followed by consideration in part 2 of the current state of the art, trends and issues, and challenges in linking the cognitive sciences with the instructional design process. Instructional development, including needs assessment and task analysis procedures are covered in part 3, and part 4 reviews various applications of instructional technology, such as the Internet and distance education. Parts 4 and 5 present an overview of current research and cover certification and professional development in the field. New to this edition are seven chapters that address such current topics as educational systems development and instructional systems development, postmodernism and instructional technology, interactive technologies, the Internet and higher education, qualitative research, and instructional technology and attitude change. (Author/AEF)
Delphi technique as used by the University of Virginia School of Education. (LR)
This research study used the Delphi technique to determine the ideal content of a college-level outdoor leadership course for land-based outdoor pursuits in the Pacific Northwest. Topics were generated and value-rated by 61 Pacific Northwest outdoor leaders using a Likert-type scale in three separate questionnaires. Thirty-five topics were retained in the ideal content, of which the top 10 were: (1) risk management plans; (2) judgement; (3) wilderness ethics; (4) first aid; (5) analyzing risks; (6) minimum impact practices; (7) outdoor leadership objectives; (8) hazard analysis for hypothermia; (9) backcountry first aid; and (10) minimum impact philosophy. An outdoor leadership course outline constructed from research results included philosophy of outdoor leadership, risk management, decision making, outdoor leadership methods, teaching principles, small group dynamics in the outdoors, environmental awareness skills, pre-trip planning, hazards analysis, and map reading and interpretation. Tables list final rankings and mean scores of the 35 consensus topics and the 23 topics rated least important. An appendix provides a copy of the third and final questionnaire with ratings for 176 topics. (LFL)
This study determined how experts see the future of adult fitness by using the Delphi Technique as a measurement tool. Experts (N = 50), all Fellows of the American College of Sports Medicine, were selected on the basis of their reputations in adult fitness. Three questionnaires were submitted to the panel. In the first questionnaire each expert listed five or more events, trends, or developments they thought would take place and five or more events they would like to see take place in the next 25 years. From the events and trends listed on this questionnaire, 100 items were generated that formed the basis for the last two questionnaires. Responses to the items on the second questionnaire were evaluated on a 5-point Likert scale for probability and desirability. In the third questionnaire, group means and standard deviations were calculated and recorded for each item. Only those items which were significantly (p < .01) higher than 3.0, the average probability or desirability response on the Likert scale, were included in the analysis. The items were divided into seven categories: fitness in sport and recreation, fitness and wellness, fitness in industry, fitness in education, fitness in business, research in fitness, and fitness equipment and facilities. The highest single mean for any item for both probability (M = 4.31) and desirability (M = 4.83) was increased awareness of health and fitness among the elderly.
This is a review of the state-of-the-art of rational and primarily quantitative approaches to the utilization of expert opinion in the inexact sciences, among which the engineering sciences must be included. Of primary concern is the decision-maker's utilization of one or more experts, and the criteria and scoring rules used to evaluate their performance. Also reviewed are certain game-theoretic strategies used by experts to maximize their own goal satisfaction. With this review, it is hoped that chemical engineers, when acting as decision-makers, will be able to utilize expert opinion more effectively and will better understand the function of experts when it is necessary for them to adopt such roles themselves.
This paper presents the results of a two round Delphi survey conducted into expert opinion on the development of indicators to measure the movement of the tourism product at a company/resort level towards a position of greater or lesser sustainability. This research forms part of a wider project to develop indicators that consumers can use in the selection of their holidays and promote a more sustainable form of tourism. The results of this expert survey show considerable disagreement over “sustainability” and where the borders of the concept exist. In addition, the research identified contrasting views over the use of qualitative versus quantitative indicators and the role that consumer pressure can play. The use of the Delphi technique to address complex and uncertain issues is also explored.
This paper is concerned with a methodology for efficiently obtaining concensus from a panel of evaluators on questions which are shrouded in uncertainty and can not be measured or evaluated in the classical sense. The methodology and its historical development are described, it is placed in context vis-a-vis other methods of subjective scaling, and a critique is provided. An extensive and current annotated bibliography is presented.
Thesis (Ph. D.)--Ohio State University, 1994. Includes bibliographical references (leaves 163-170). Advisor: R. Kirby Barrick, Dept. of Agricultural Education. System requirements: World Wide Web browser and PDF viewer.
Thesis--Ohio State University. Bibliography: leaves 171-179.
Thesis (M.A.)--Ohio State University, 1971. Includes bibliographical references (leaves 88-91).
Thesis (Ed. D.)--West Virginia University, 1983. Vita. Includes bibliographical references (leaves 254-259). Microfilm. s
Delphi technique. Lecture, Applied evaluation design
- J W Altschuld
Altschuld, J. W. (2003). Delphi technique. Lecture, Applied
evaluation design. The Ohio State University.
Using the Delphi process to identify significant recreation research-based innovations
- D H Anderson
- I E Schneider
Anderson, D. H., & Schneider, I. E. (1993). Using the
Delphi process to identify significant recreation
research-based innovations. Journal of Park and
Recreation Administration, 11 (1), 25-36.
A comparative study of the effectiveness of decision making processes which utilize the Delphi and leaderless group methodologies. Unpublished doctoral dissertation
- D C Douglas
Douglas, D. C. (1983). A comparative study of the effectiveness of
decision making processes which utilize the Delphi and leaderless
group methodologies. Unpublished doctoral dissertation,
The Ohio State University, Columbus.
Essential assessment criteria for physical education teacher education programs: A Delphi study
- J M Jacobs
Jacobs, J. M. (1996). Essential assessment criteria for physical
education teacher education programs: A Delphi study.
Unpublished doctoral dissertation, West Virginia
University, Morgantown.
Priorities within the health care system: A Delphi survey
- C A Lindeman
Lindeman, C. A. (1981). Priorities within the health care system:
A Delphi survey. Kansas City, MO: American Nurses'
Association.
Rethinking the outlook of colleges whose roots have been in agriculture
- J H Meyer
Meyer, J. H. (1992). Rethinking the outlook of colleges whose roots
have been in agriculture. Davis, CA: University of
California.
Determining what could/should be: The Delphi technique and its application
- L E Miller
Miller, L. E. (2006, October). Determining what could/should
be: The Delphi technique and its application. Paper presented
at the meeting of the 2006 annual meeting of the
Mid-Western Educational Research Association,
Columbus, Ohio.
The Delphi method: Techniques and applications
- M Scheibe
- M Skutsch
- J Schofer
Scheibe, M., Skutsch, M., & Schofer, J. (1975).
Experiments in Delphi methodology. In H. A.
Linstone, & M. Turoff (Eds.). The Delphi method:
Techniques and applications (pp. 262-287). Reading, MA:
Addison-Wesley Publishing Company.
Tourism marketing and management handbook
- R E Taylor
- L L Judd
Taylor, R. E., & Judd, L. L. (1989). Delphi method applied
to tourism. In S. Witt, & L. Moutinho, (Eds.). Tourism
marketing and management handbook. New York: Prentice
Hall.
Gazing into the oracle: The Delphi method and its application to social policy and public health
- M Turoff
- S R Hiltz
Turoff, M., & Hiltz, S. R. (1996). Computer based Delphi
process. In M. Adler, & E. Ziglio (Eds.). Gazing into the
oracle: The Delphi method and its application to social policy and
public health (pp. 56-88). London, UK: Jessica Kingsley
Publishers.
Human resource development: The theory and practice of need assessment
- F L Ulschak
Ulschak, F. L. (1983). Human resource development: The theory
and practice of need assessment. Reston, VA: Reston
Publishing Company, Inc.
Planning and conducting needs assessment: A practical guide
- B R Witkin
- J W Altschuld
Witkin, B. R., & Altschuld, J. W. (1995). Planning and
conducting needs assessment: A practical guide. Thousand
Oaks, CA: Sage Publications, Inc.