Article

The Delphi Technique: Making Sense Of Consensus

January 2007

Authors:

Shih Hsin University

The Delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. The technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. The Delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. The Delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. Subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a Delphi study.

The Delphi technique: a tutorial

Article

Full-text available

Jan 2022

Victoria Naisola Ruiter

The Delphi technique is a recognised research technique designed as a futures-oriented methodology that gathers data by engaging a panel of experts to make judgements on a specific real-world issue. The technique enables the researcher to interact with key stakeholders and decision makers in the field, who are well experienced and trained in the specialised area of knowledge related to the target issue of the research, enabling researchers to gain a deeper understanding of real-world opinions. The Delphi technique has been described as a qualitative, quantitative, and mixed methods approach as the collection of narrative group opinion, coupled with the strictly structured nature of the process and quantitively described results, poses a challenge to situate the approach in a specific methodological category and its usage. This methods tutorial discussion aims to illustrate the usage of the Delphi technique by providing a worked example to demonstrate the proposed method in a hospitality industry setting. KEYWORDS: experts, future oriented, HR, hospitality industry, research approach

Evaluating insider threat indicators and mitigation measures: a Delphi study

Technical Report

Apr 2022

The report elaborates on the results of our Delphi study on insider threat mitigation. The goal of the study was to discover potential ‘red flags’ of insider threat incidents (i.e. factors that may point to imminent insider threat incidents) and good practices on insider threat mitigation. The study employed the Delphi technique to iteratively compare and contrast the opinions of insider threat experts. A multidisciplinary panel of 25 international experts in a field related to insider threats (corporate security, counterintelligence, insider threat training, …) completed three rounds of online questionnaires that contained questions on four insider threat mitigation phases, namely prevention, detection, pre-emption and the aftermath of an insider threat incident. Round 1 concerned open-ended, level-setting questions with panelists asked to share their expertise on both red flags and good practices. In round 2, experts were presented with a list of all important red flags/good practices shared by the panel in round 1 in the form of a structured questionnaire whereby they were asked to rate each individual red flag/good practice listed. The questionnaire of round 3, to conclude, provided the experts with a list of high-rated factors, after which they were asked whether they agreed or disagreed with the panel’s decision and to explain their reasoning in case of disagreement. The results of the study provide readers with useful insights on what experts consider to be red flags organizations should be vigilant of, as well as with mitigation measures to better secure organizations against insider threats.

The Emergency Care Competence Needed for Police Patrol Officers According to the Experts - a National Swedish Delphi Study Mats Holmberg

Article

Jun 2022

Police patrol officers (PPOs) face different types of encounters with acutely ill and injured people as first responders. The study aim was to explore and describe the emergency care competence needed for Swedish PPOs. The study had an exploratory design, and the data was collected using the Delphi technique from 43 experts in Sweden: police program educators (n=10), police authority instructors (n=11), and PPOs (n= 22). The results generated an understanding of emergency care competence that is relevant for PPOs. Basic assessment and basic life support meas-RESEAR CH PUBLICATION ures for traumatic injuries and cardiac arrest (e.g., airway management, cardiopulmonary resuscitation (CPR), and hemorrhage control) received the highest levels of agreement among the experts, whereas more advanced assessments (taking care of people suffering from mental/medical illness or with drug/alcohol abuse) received lower levels of agreement. These results provide support for educators at the police programs and for the police authority in preparing PPOs to perform relevant out-of-hospital emergency care.

Development and validation of the Adherence to Asthma Medication Questionnaire (AAMQ)

Article

Full-text available

May 2022

Background: Adherence to medication is the cornerstone to achieve the best treatment outcome. Pharmacists are healthcare professionals found in a pivotal position to assess asthmatic patients’ adherence to medication. A brief, reliable, and valid measure of patients adherence to mediations is useful to enable the pharmacists to deliver that vital service. Objective: To develop a reliable and valid adherence assessment tool for asthmatic patients. Methods: The Adherence to Asthma Medication Questionnaire (AAMQ-13) was developed based on an extensive literature review, followed by applying the Delphi technique, and then it was pilot-tested by 55 patients. The final AAMQ-13 was completed by 213 patients. Psychometric evaluation was assessed including reliability, criterion validity, and construct validity. Results: The AAMQ-13 is a feasible 13-item questionnaire, as it can be completed within an average of two minutes. It has high reliability (Cronbach’s alpha= 0.87). Criterion-concurrent validity was established by comparing the AAMQ-13 to the Test of the Adherence to Inhaler (TAI) and the pharmacy refill records. Criterion-convergent validity was established by comparing the AAMQ-13 to the Asthma Control Test (ACT) questionnaire and the Positive Health Behaviors Scale (PHBS). Construct validity was established through AAMQ-13 factor analysis which revealed two factors explaining 51.76% of the total variance. Conclusion: The AAMQ-13 is a reliable and valid questionnaire with several desirable characteristics as it has high reliability, good criterion validity, and strong construct validity. The AAMQ-13 is a suitable questionnaire that can identify non-adherent patients and reveal the reasons behind their non-adherence

Consensus on the Objectives of an Educational Intervention for Patients with Oropharyngeal Dysphagia and Their Informal Caregivers: A Delphi Study

Article

Full-text available

Jun 2022

Purpose: In the absence of literature allowing for an evidence-based approach to therapeutic patient education (TPE) in Oropharyngeal Dysphagia (OD), this study aims to reach a consensus of experts on the content of a competency framework of an educational program for OD patients and their informal caregivers (ICGs). Methods: We used the Delphi consensus-building method. Four categories of experts were recruited: 12 patients, 17 ICGs, 46 healthcare professionals (HCP) (experienced in OD, not necessarily certified in TPE), and 19 experts in TPE (trained individuals to set up and run TPE programs not necessarily HCPs). The content of the questionnaire of the first round (R) was established according to the result of a scoping review and the opinion of an expert committee. We carried out three rounds. In R1 and R2, we collected the opinions on the relevance (7-point Likert-type scale) and on comprehensiveness (YES/No question and asking participants to propose additional content). Participants were also invited to leave comments on each objective. In R3, we asked the participants to give their opinion about the relevance of the objectives again and asked them to rank the themes from highest to lowest priority. Results: Objectives were considered relevant for all participants if they reached consensus when the interquartile (IQR) ≤ 1, and if the median indicated agreement (Mdn ≥ 6) (6= appropriate, 7 = totally appropriate). Following three rounds, the final content of the educational program is composed of 23 educational objectives organized in 13 themes with an agreement about relevance amongst all participants (Mdn ≥ 6; IQR ≤ 1). The comprehensiveness criterion received also a consensus (IQR ≤ 1). The participants ranked the theme "normal swallowing vs difficulty swallowing" as the highest priority. Conclusion: This Delphi study resulted in a consensus, on the content of a competency framework of an educational program for OD patients and their ICGs. Further steps are needed to construct learning activities based on these objectives before testing their feasibility and efficacy.

State Policies that Impact the Design of Children’s Mental Health Services: A Modified Delphi Study

Article

Full-text available

Jun 2022
ADM POLICY MENT HLTH

To identify the state-level policies and policy domains that state policymakers and advocates perceive as most important for positively impacting the use of children’s mental health services (CMHS). We used a modified Delphi technique (i.e., two rounds of questionnaires and an interview) during Spring 2021 to elicit perceptions among state mental health agency officials and advocates (n = 28) from twelve states on state policies that impact the use of CMHS. Participants rated a list of pre-specified policies on a 7-point Likert scale (1 = not important, 7 = extremely important) in the following policy domains: insurance coverage and limits, mental health services, school and social. Participants added nine policies to the initial list of 24 policies. The “school” policy domain was perceived as the most important, while the “social” policy domain was perceived as the least important after the first questionnaire and the second most important policy domain after the second questionnaire. The individual policies perceived as most important were school-based mental health services, state mental health parity, and Medicaid reimbursement rates. Key stakeholders in CMHS should leverage this group of policies to understand the current policy landscape in their state and to identify gaps in policy domains and potential policy opportunities to create a more comprehensive system to address children’s mental health from a holistic, evidence-based policymaking perspective.

ACCORD guideline for reporting consensus-based methods in biomedical research and clinical practice: a study protocol

Article

Full-text available

Jun 2022

Background Structured, systematic methods to formulate consensus recommendations, such as the Delphi process or nominal group technique, among others, provide the opportunity to harness the knowledge of experts to support clinical decision making in areas of uncertainty. They are widely used in biomedical research, in particular where disease characteristics or resource limitations mean that high-quality evidence generation is difficult. However, poor reporting of methods used to reach a consensus – for example, not clearly explaining the definition of consensus, or not stating how consensus group panellists were selected – can potentially undermine confidence in this type of research and hinder reproducibility. Our objective is therefore to systematically develop a reporting guideline to help the biomedical research and clinical practice community describe the methods or techniques used to reach consensus in a complete, transparent, and consistent manner. Methods The ACCORD (ACcurate COnsensus Reporting Document) project will take place in five stages and follow the EQUATOR Network guidance for the development of reporting guidelines. In Stage 1, a multidisciplinary Steering Committee has been established to lead and coordinate the guideline development process. In Stage 2, a systematic literature review will identify evidence on the quality of the reporting of consensus methodology, to obtain potential items for a reporting checklist. In Stage 3, Delphi methodology will be used to reach consensus regarding the checklist items, first among the Steering Committee, and then among a broader Delphi panel comprising participants with a range of expertise, including patient representatives. In Stage 4, the reporting guideline will be finalised in a consensus meeting, along with the production of an Explanation and Elaboration (E&E) document. In Stage 5, we plan to publish the reporting guideline and E&E document in open-access journals, supported by presentations at appropriate events. Dissemination of the reporting guideline, including a website linked to social media channels, is crucial for the document to be implemented in practice. Discussion The ACCORD reporting guideline will provide a set of minimum items that should be reported about methods used to achieve consensus, including approaches ranging from simple unstructured opinion gatherings to highly structured processes.

Exploring the feasibility of a framework to align a competency framework in a dental curriculum: a Delphi study SADJ April 2022, Vol. 77 No. 3 p147 - p154

Article

Full-text available

May 2022

Aim The aim of this study was to test the feasibility of an implementation framework for an undergraduate competency framework by means of a Delphi study. Methods A modified Delphi research design consisting of two rounds was conducted and aimed to seek consensus defined as agreement of 70% or more from a group of experts. Twentytwo (22) dental experts were purposefully selected from the four dental schools in South Africa. An online survey consisting mainly of closed-ended questions was used and analysed quantitatively. In addition, the second part of each question included a comment section to allow participants to elaborate on their answer. Comments were considered as adapted statements for the second round of the Delphi. Results Thirteen (13) participants completed the first round, and 10 participants completed the second round of the Delphi study. Following the first round of the Delphi, consensus (70%) was reached for five of the seven statements. These statements focus on the impact of internal and external stakeholders on the dental schools, leadership support for faculty development, teaching and learning changes, and curriculum mapping. Statements that needed further consideration in round 1 included impact of institutional framework and diversity of academics. Consensus was reached for both the diversity of academics statements in round 2. Conclusion The results suggest that the proposed framework to align and implement the AfriMEDS core competency framework was feasible. However, from the comments of the dental experts there seems to be no common understanding of implementation and alignment of the institutional frameworks and the AfriMEDS core competency framework.

Simulated Placements as Partial Replacement of Clinical Training Time: A Delphi Consensus Study

Article

Full-text available

Jul 2022

Background There is growing interest in utilizing structured extended periods of simulation as partial replacement for clinical placement (PRCP) due to workforce demands and COVID-19 impact. This project aimed to identify the extent to which stakeholders felt that simulated placements could reduce overall clinical placement time requirement for health care students. Method Participants included a range of nurses and Allied Health Professionals including managers, clinicians and lecturers. A three-round Delphi study was conducted with a 75% consensus agreement target. Results Overall, 71 participants engaged with at least one round of the study and identified benefits of PRCP: Safety, learning from mistakes, improved preparation, and reduced placement training burden. Challenges included poor realism, logistics, time, and access to facilities. Conclusions The consensus opinion indicated that between 11% and 30% of clinical training time could be replaced with simulated placement. Ongoing work will need to identify profession-specific consensus opinion and guidance on the use of simulation as PRCP. Cite this article: Bridge, P., et al. (Year, Month). Simulated placements as partial replacement of clinical training time: a Delphi consensus study. Clinical Simulation in Nursing, vol(no.), eXXX-eXXX. doi: 10.1016/j.ecsn.20XX.

Framing Algorithm-Driven Development of Sets of Objectives Using Elementary Interactions

Article

Full-text available

May 2022

Multi-criteria decision analysis (MCDA) is an established methodology to support the decision-making of multi-objective problems. For conducting an MCDA, in most cases, a set of objectives (SOO) is required, which consists of a hierarchical structure comprised of objectives, criteria, and indicators. The development of an SOO is usually based on moderated development processes requiring high organizational and cognitive effort from all stakeholders involved. This article proposes elementary interactions as a key paradigm of an algorithm-driven development process for an SOO that requires little moderation efforts. Elementary interactions are self-contained information requests that may be answered with little cognitive effort. The pairwise comparison of elements in the well-known analytical hierarchical process (AHP) is an example of an elementary interaction. Each elementary interaction in the development process presented contributes to the stepwise development of an SOO. Based on the hypothesis that an SOO may be developed exclusively using elementary interactions (EIs), a concept for a multi-user platform is proposed. Essential components of the platform are a Model Aggregator, an Elementary Interaction Stream Generator, a Participant Manager, and a Discussion Forum. While the latter component serves the professional exchange of the participants, the first three components are intended to be automatable by algorithms. The platform concept proposed has been evaluated partly in an explorative validation study demonstrating the general functionality of the algorithms outlined. In summary, the platform concept suggested demonstrates the potential to ease SOO development processes as the platform concept does not restrict the application domain; it is intended to work with little administration moderation efforts, and it supports the further development of an existing SOO in the event of changes in external conditions. The algorithm-driven development of SOOs proposed in this article may ease the development of MCDA applications and, thus, may have a positive effect on the spread of MCDA applications.

Analysis for the factors of DSpace towards academic library perspective using MCDM

Article

Full-text available

Jun 2022

This study is made to find out the important factors of DSapce relevant to implementation in the academic libraries based on the experts’ opinion view. There is lack of use the MCDM techniques in the library field. In this study authors used the Delphi technique to validate the factors which are important for implementing the software. The different studies related to using MCDM are studied as review of literature. The findings of the study provide the priority of the factors through using the Delphi. The results will be helpful as theoretical and managerial perspective to take decision regarding the implementation of the software.

Bridging the Gap Between Medical and Academic Stakeholders Who Assist University Students Returning to Learn after Concussion

Thesis

Jan 2022

Allyssa K. Memmini

Concussions affect millions of young adults annually, many of whom are enrolled in post-secondary education. To date, concussion research has yet to establish evidence-based protocols to assist university students in returning to the classroom (i.e., return to learn [RTL]) after concussion. Without clear guidelines, students are left to independently navigate the complex medical and university systems. In order to understand and address barriers to receiving and implementing post-concussion academic supports within higher education, this dissertation was conducted using an exploratory-sequential mixed methods approach comprised of three main aims: (1) to collate the experiences of academic stakeholders and previously concussed university students; (2) to develop the Post-Concussion Collegiate RTL Protocol and additional concussion management recommendations for higher education through a multidisciplinary Delphi procedure; and (3) to assess perceptions surrounding implementation of the novel RTL Protocol among clinicians and academic stakeholders across the Power 5 Conferences. The first investigation was conducted using semi-structured interviews among previously concussed undergraduate students (n=21) and university academic stakeholders (n=7). Using a Grounded Theory approach, several emerging themes arose among the two cohorts. Specifically, students recalled empathetic interactions with their instructors; however, academic stakeholders indicated they lacked sufficient information and/or resources on how to appropriately develop academic supports to facilitate the RTL process for students recovering from concussion. The second study sought to mediate the discrepancies in post-concussion RTL support within higher education by developing concussion management guidelines through a modified Delphi procedure. The Delphi panel was comprised of twenty-two multidisciplinary professionals (n=9 clinicians, n=8 researchers, n=5 university academic stakeholders). The iterative process resulted in consensus across four main categories including: recommendations for medical discharge documentation, procedures for developing and maintaining a multidisciplinary RTL approach within post-secondary institutions, processes for university students to obtain academic supports following concussion, as well as the development of the novel Post-Concussion Collegiate RTL Protocol. The third project explored perceptions of prospective utility of the Post-Concussion Collegiate RTL Protocol among clinicians and university faculty/staff within the Power 5 Conferences (Atlantic Coast Conference, Southeastern Conference, Big 10 Conference, Big 12 Conference, and the Pacific 12 Conference). Participants (N=49; n=25 clinicians, n=24 university faculty/staff) completed an electronic survey comprised of four validated implementation outcome measures including the Acceptability of Implementation Measure (AIM), Implementation Appropriateness Measure (IAM), Feasibility of Implementation Measure (FIM), and Organizational Readiness for Implementing Change (ORIC). Mann-Whitney U analyses yielded no significant between-group differences on the protocol’s acceptability, appropriateness, and feasibility in their setting. However, group differences within the change commitment and change efficacy subscales of the ORIC were noted, with university faculty/staff indicating lower organizational readiness for change in contrast to clinicians within the Power 5 Conferences. Collectively, these findings serve as a framework for future university policy to standardize the RTL process for students who require academic support after concussion. Additional research is warranted to investigate university infrastructure to develop implementation pathways to support the Post-Concussion Collegiate RTL Protocol across all institutions of higher learning.

What do I need to know? Essential educational concepts for complex regional pain syndrome

Article

Full-text available

May 2022

Background: Complex Regional Pain Syndrome (CRPS) is a rare but disabling pain condition. Accurate and timely education about CRPS is key to promote optimal clinical outcomes, but it is unclear what the content of that education should be. We aimed to determine the content that both people with CRPS and expert health care professionals (HCPs) reported as important.. Methods: An international three-round e-Delphi was conducted, recruiting adults diagnosed with CRPS and HCPs. In Round 1, participants were asked to list the most important information people with CRPS should know regarding the condition. Data were organised into concepts and allocated to themes. In Rounds 2 and 3, participants rated each concept on a 9-point Likert Scale, categorised as 'not important' (0-3), 'important' (4-6) and 'very important' (7-9). A concept attained consensus when >75% agreement was reached within a category. Results: Sixty-two participants (HCPs: n=7; CRPS: n=55), proposed 193 concepts in Round 1, resulting in 22 themes. Fifteen additional concepts were identified in Round 2, resulting in a total of 208 concepts. From that list, 48 concepts that emphasised understanding and evidence-based management of the disorder, the importance of self-management strategies, pacing and movement, reached joint consensus as 'very important'. One concept: "Advise that movement does not help" reached joint consensus as 'not important'. Conclusion: Forty-eight concepts were jointly considered 'very important' for future CRPS-related educational content. Future research to better understand group differences and to canvas a broader HCP group is warranted.

Improving emergency treatment for patients with acute stroke: the PEARS research programme, including the PASTA cluster RCT

Article

Full-text available

May 2022

Background Intravenous thrombolysis and intra-arterial thrombectomy are proven emergency treatments for acute ischaemic stroke, but they require rapid delivery to selected patients within specialist services. National audit data have shown that treatment provision is suboptimal. Objectives The aims were to (1) determine the content, clinical effectiveness and day 90 cost-effectiveness of an enhanced paramedic assessment designed to facilitate thrombolysis delivery in hospital and (2) model thrombectomy service configuration options with optimal activity and cost-effectiveness informed by expert and public views. Design A mixed-methods approach was employed between 2014 and 2019. Systematic reviews examined enhanced paramedic roles and thrombectomy effectiveness. Professional and service user groups developed a thrombolysis-focused Paramedic Acute Stroke Treatment Assessment, which was evaluated in a pragmatic multicentre cluster randomised controlled trial and parallel process evaluation. Clinicians, patients, carers and the public were surveyed regarding thrombectomy service configuration. A decision tree was constructed from published data to estimate thrombectomy eligibility of the UK stroke population. A matching discrete-event simulation predicted patient benefits and financial consequences from increasing the number of centres. Setting The paramedic assessment trial was hosted by three regional ambulance services (in north-east England, north-west England and Wales) serving 15 hospitals. Participants A total of 103 health-care representatives and 20 public representatives assisted in the development of the paramedic assessment. The trial enrolled 1214 stroke patients within 4 hours of symptom onset. Thrombectomy service provision was informed by a Delphi exercise with 64 stroke specialists and neuroradiologists, and surveys of 147 patients and 105 public respondents. Interventions The paramedic assessment comprised additional pre-hospital information collection, structured hospital handover, practical assistance up to 15 minutes post handover, a pre-departure care checklist and clinician feedback. Main outcome measures The primary outcome was the proportion of patients receiving thrombolysis. Secondary outcomes included day 90 health (poor status was a modified Rankin Scale score of > 2). Economic outputs reported the number of cases treated and cost-effectiveness using quality-adjusted life-years and Great British pounds. Data sources National registry data from the Sentinel Stroke National Audit Programme and the Scottish Stroke Care Audit were used. Review methods Systematic searches of electronic bibliographies were used to identify relevant literature. Study inclusion and data extraction processes were described using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Results The paramedic assessment trial found a clinically important but statistically non-significant reduction in thrombolysis among intervention patients, compared with standard care patients [197/500 (39.4%) vs. 319/714 (44.7%), respectively] (adjusted odds ratio 0.81, 95% confidence interval 0.61 to 1.08; p = 0.15). The rate of poor health outcomes was not significantly different, but was lower in the intervention group than in the standard care group [313/489 (64.0%) vs. 461/690 (66.8%), respectively] (adjusted odds ratio 0.86, 95% confidence interval 0.60 to 1.2; p = 0.39). There was no difference in the quality-adjusted life-years gained between the groups (0.005, 95% confidence interval –0.004 to 0.015), but total costs were significantly lower for patients in the intervention group than for those in the standard care group (–£1086, 95% confidence interval –£2236 to –£13). It has been estimated that, in the UK, 10,140–11,530 patients per year (i.e. 12% of stroke admissions) are eligible for thrombectomy. Meta-analysis of published data confirmed that thrombectomy-treated patients were significantly more likely to be functionally independent than patients receiving standard care (odds ratio 2.39, 95% confidence interval 1.88 to 3.04; n = 1841). Expert consensus and most public survey respondents favoured selective secondary transfer for accessing thrombectomy at regional neuroscience centres. The discrete-event simulation model suggested that six new English centres might generate 190 quality-adjusted life-years (95% confidence interval –6 to 399 quality-adjusted life-years) and a saving of £1,864,000 per year (95% confidence interval –£1,204,000 to £5,017,000 saving per year). The total mean thrombectomy cost up to 72 hours was £12,440, mostly attributable to the consumables. There was no significant cost difference between direct admission and secondary transfer (mean difference –£368, 95% confidence interval –£1016 to £279; p = 0.26). Limitations Evidence for paramedic assessment fidelity was limited and group allocation could not be masked. Thrombectomy surveys represented respondent views only. Simulation models assumed that populations were consistent with published meta-analyses, included limited parameters reflecting underlying data sets and did not consider the capital costs of setting up new services. Conclusions Paramedic assessment did not increase the proportion of patients receiving thrombolysis, but outcomes were consistent with improved cost-effectiveness at day 90, possibly reflecting better informed treatment decisions and/or adherence to clinical guidelines. However, the health difference was non-significant, small and short term. Approximately 12% of stroke patients are suitable for thrombectomy and widespread provision is likely to generate health and resource gains. Clinician and public views support secondary transfer to access treatment. Future work Further evaluation of emergency care pathways will determine whether or not enhanced paramedic assessment improves hospital guideline compliance. Validation of the simulation model post reconfiguration will improve precision and describe wider resource implications. Trial registration This trial is registered as ISRCTN12418919 and the systematic review protocols are registered as PROSPERO CRD42014010785 and PROSPERO CRD42015016649. Funding The project was funded by the National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research ; Vol. 10, No. 4. See the NIHR Journals Library website for further project information.

Delphi Tekniği Kullanılarak Kritik ve Öncelikli Öğrenme Analitiği Göstergelerinin Belirlenmesi

Article

Full-text available

Aug 2021

Öğrenme analitikleri eğitim ortamlarının ve paydaşlarının objektif ölçümlerle anlaşılabilmesine imkân sağlaması, bu ortamlarda yaşanan sorunlara çözüm sunması ve bu ortamların daha verimli hale getirilmesine yönelik yeni yaklaşımlar sunması nedeniyle üzerine dikkatleri çeken bir konu haline gelmiştir. Öğrenme analitiği göstergeleri ise, henüz öğrenme problemleri ile ilişkilendirilmemiş, ancak e-öğrenme ortamlarında kullanılabilecek türde analitiklerdir. Bir bakıma öğrenme analitiği adaylarıdır. Bu çalışmanın amacı gelişim aşamasında görülen öğrenme analitiklerinin temellerine değinmenin yanı sıra, araştırmacı ve uygulamacılara e-öğrenme ortamlarında kullanabilecekleri öğrenme analitiği göstergelerine ilişkin bir liste sunmaktır. Çalışmada, alanyazın taramasından elde edilen göstergelerin yanı sıra Delphi tekniği kullanılarak uzmanların fikir birliğine vardıkları göstergelere de ulaşılmıştır. Çalışma için geliştirilen çevrimiçi platform aracılığıyla yürütülen Delphi panelleri yaklaşık 1 yıl sürmüş, 22 uzman ile başlayan süreç 11 uzman ile 4 tur sonunda tamamlanmıştır. Çalışmada demografik, betimsel ve algoritmik üst başlıklarında toplam 41 maddeden oluşan öğrenme analitiği göstergesine ulaşılmıştır. Bu listenin gerek öğrenme analitiklerini kullanarak öğrenme ortamlarında nesnel kararlar almaya çalışan uygulayıcılara gerekse de öğrenme analitiği ile ilgili konularda çalışmayı düşünen araştırmacılara katkı sağlayacağı düşünülmektedir.

ONE EXAMPLE OF CIVIL ENGINEERING RESEARCH CONDUCTED WITH THE DELPHI METHOD

Article

Full-text available

Dec 2021

In this work one example of civil engineering research conducted with the Delphi method is shown. The Delphi method is one of the most popular scientific methods for research of problems with lack of known data. For this research, researcher must form the team of experts and analyze their answers on questions asked during the research with controlled feedback. In this article researcher wanted to prove hypothesis for making risk assessment model for planning and design processes of wastewater treatment plants. This research was conducted with Classical Delphi method is shown step by step.

Consensus on a standardised treatment pathway algorithm for lumbar spinal stenosis: an international Delphi study

Article

Full-text available

Jun 2022
BMC MUSCULOSKEL DIS

Background Lumbar spinal stenosis (LSS) is a common degenerative spinal condition in older adults associated with disability, diminished quality of life, and substantial healthcare costs. Individual symptoms and needs vary. With sparse and sometimes inconsistent evidence to guide clinical decision-making, variable clinical care may lead to unsatisfactory patient outcomes and inefficient use of healthcare resources. Methods A three-phase modified Delphi study comprising four consensus rounds was conducted on behalf of the International Taskforce for the Diagnosis and Management of LSS to develop a treatment algorithm based on multi-professional international expert consensus. Participants with expertise in the assessment and management of people with LSS were invited using an international distribution process used for two previous Delphi studies led by the Taskforce. Separate treatment pathways for patients with different symptom types and severity were developed and incorporated into a proposed treatment algorithm through consensus rounds 1 to 3. Agreement with the proposed algorithm was evaluated in the final consensus round. Results The final algorithm combines stratified and stepped approaches. When indicated, immediate investigation and surgery is advocated. Otherwise, a stepped approach is suggested when self-directed care is unsatisfactory. This starts with tailored rehabilitation, then more complex multidisciplinary care, investigations and surgery options if needed. Treatment options in each step depend on clinical phenotype and symptom severity. Treatment response guides pathway entrance and exit points. Of 397 study participants, 86% rated their agreement ≥ 4 for the proposed algorithm on a 0–6 scale, of which 22% completely agreed. Only 7% disagreed. Over 70% of participants felt that the algorithm would be useful for clinicians in public healthcare (both primary care and specialist settings) and in private healthcare settings, and that a simplified version would help patients in shared decision-making. Conclusions International and multi-professional agreement was achieved for a proposed LSS treatment algorithm developed through expert consensus . The algorithm advocates different pathway options depending on clinical indications. It is not intended as a treatment protocol and will require evaluation against current care for clinical and cost-effectiveness. It may, however, serve as a clinical guide until evidence is sufficient to inform a fully stratified care model.

Perceived usefulness of trauma audit filters in urban India: a mixed-methods multicentre Delphi study comparing filters from the WHO and low and middle-income countries

Article

Full-text available

Jun 2022

Objective To compare experts’ perceived usefulness of audit filters from Ghana, Cameroon, WHO and those locally developed; generate context-appropriate audit filters for trauma care in selected hospitals in urban India; and explore characteristics of audit filters that correlate to perceived usefulness. Design A mixed-methods approach using a multicentre online Delphi technique. Setting Two large tertiary hospitals in urban India. Methods Filters were rated on a scale from 1 to 10 in terms of perceived usefulness, with the option to add new filters and comments. The filters were categorised into three groups depending on their origin: low and middle-income countries (LMIC), WHO and New (locally developed), and their scores compared. Significance was determined using Kruskal-Wallis test followed by Wilcoxon rank-sum test. We performed a content analysis of the comments. Results 26 predefined and 15 new filter suggestions were evaluated. The filters had high usefulness scores (mean overall score 9.01 of 10), with the LMIC filters having significantly higher scores compared with those from WHO and those newly added. Three themes were identified in the content analysis relating to medical relevance , feasibility and specificity . Conclusions Audit filters from other LMICs were deemed highly useful in the urban India context. This may indicate that the transferability of defined trauma audit filters between similar contexts is high and that these can provide a starting point when implemented as part of trauma quality improvement programmes in low-resource settings.

An Exploration of Employee Behavioural Impact towards Organisational Resilience: A Study among Malaysian MSC Status Companies

Article

Full-text available

May 2022

Shubashini rathina velu

Purpose Globalisation and a significant reliance on technology for business management are on the rise in the twenty-first century. Organisations are growing increasingly exposed to both external and internal issues, making business more difficult. Drawing upon the system theory and self determination theory , this study examines the underlying theoretical mechanism between complex behaviour that emerges as a result of interactions among employees and the orginasations. Design/methodology/approach Employees of the Malaysian Digital Economic Corporation Sdn. Bhd (MDEC) registered .'s organisations were chosen as samples for this study .A total of 171 employees from these companies completed a survey. The data were analysed using PLS-SEM to discover employees collective behaviour capacities positively influenced the organisational capacity of MSC status companies. Findings The findings of the present study have validated the empirical model, showing a significant relationship in between the proposed constructs. Employees are an important commodity for a company, not only in leadership but also in survival. There is a positive set of behavioural capitals which allow the company to develop as a key agent of national economic advancement. The internal social structure influences resilience as a capacity. Thus the theoretical insight explain how organisations develop their internal capabilities as a capacity for resilience in the emerging market context of Malaysian MSCs Originality/value The study is first of its kind which has tried to investigate an exhaustive direct relationship model in the MSC sector. The study has postulated that it is not just the organizational resilience content but the perceptions of employees toward enhancing organizational resilience. The solution is based on an organisation's collective behavioral capabilities and internal organisational processes that connects the employees towards enhancing organizational resilience .

Usability, acceptability, and implementation strategies for the Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm: a Delphi study

Article

Full-text available

May 2022
SUPPORT CARE CANCER

Introduction Oncology guidelines recommend participation in cancer rehabilitation or exercise services (CR/ES) to optimize survivorship. Yet, connecting the right survivor, with the right CR/ES, at the right time remains a challenge. The Exercise in Cancer Evaluation and Decision Support (EXCEEDS) algorithm was developed to enhance CR/ES clinical decision-making and facilitate access to CR/ES. We used Delphi methodology to evaluate usability, acceptability, and determine pragmatic implementation priorities. Methods Participants completed three online questionnaires including (1) simulated case vignettes, (2) 4-item acceptability questionnaire (0–5 pts), and (3) series of items to rank algorithm implementation priorities (potential users, platforms, strategies). To evaluate usability, we used Chi-squared test to compare frequency of accurate pre-exercise medical clearance and CR/ES triage recommendations for case vignettes when using EXCEEDS vs. without. We calculated mean acceptability and inter-rater agreement overall and in 4 domains. We used the Eisenhower Prioritization Method to evaluate implementation priorities. Results Participants (N = 133) mostly represented the fields of rehabilitation (69%), oncology (25%), or exercise science (17%). When using EXCEEDS (vs. without), their recommendations were more likely to be guideline concordant for medical clearance (83.4% vs. 66.5%, X² = 26.61, p < .0001) and CR/ES triage (60.9% vs. 51.1%, X² = 73.79, p < .0001). Mean acceptability was M = 3.90 ± 0.47; inter-rater agreement was high for 3 of 4 domains. Implementation priorities include 1 potential user group, 2 platform types, and 9 implementation strategies. Conclusion This study demonstrates the EXCEEDS algorithm can be a pragmatic and acceptable clinical decision support tool for CR/ES recommendations. Future research is needed to evaluate algorithm usability and acceptability in real-world clinical pathways.

Successful Information Technology Project Process Management Best Practices

Thesis

Full-text available

May 2022

Jude Orazulike

Globally, most information technology projects (ITPs) are reported as unsuccessful. Poor project management practices have consistently been identified as the leading cause of ITP failures. However, ITP practitioners manage project processes in diverse ways without clear guiding principles in terms of what does or does not work in practice for success. Process management practices in projects were explored in this grounded theory qualitative study from a systems theory perspective. The purpose was to understand from project practitioners' experiences what guiding principles potentially influenced ITPs to success. These experiences were then analyzed to develop a theory describing how to best use process management in projects to achieve success. The main research question addressed in this study examined participants’ view of practices in successful ITPs that effectively led to success. The data were collected during in-depth interviews of 14 project participants using semistructured questions and were coded using the grounded theory continuous-comparison approach until theoretical saturation and themes were generated. The finding is an emergent theory, which indicates that practices in ITP process management consisting of continuous learning, regular engagement, constant surveillance, process orchestration, and timely response positively impacts a successful outcome. Leveraging this finding, process management principles are recommended to better understand ITP process management in practice. This study contributes to positive social change by providing a guide for practitioners’ use, potentially resulting in more successful educational and healthcare ITPs, especially in Africa.

Fostering translational research in chronic disease management: a logic model proposal

Article

Full-text available

Dec 2022

Translational research aims at reducing gaps between fundamental scientific discoveries and real-world applications. However, the trajectory of most scientific discoveries along the translation research continuum remains highly complex. Logic models are powerful tools that can help reduce this complexity. They are often used to lay out road maps and depict the relationship between activities and their intended effects. Few if any existing tools have been designed to guide the implementation and evaluation of collaborative models between community-based primary health care and biomedical research. To address this gap, we developed a logic model in two stages: 1) a literature review; and 2) the drafting and revision of the model by experts in the field. We describe its components, including objectives, inputs, activities, target groups, outputs, and results for a collaborative model involving fundamental biomedical research and primary health care practices. Our proposed logic model provides a road map that has the potential to reduce the complexity faced by translational research in chronic diseases by providing guidelines for decision-makers. Future work should attempt to validate the model before its broad-based implementation.

Core competencies for a biomedical laboratory scientist – a Delphi study

Article

Full-text available

Jun 2022
BMC Med Educ

Background After completing university education, biomedical laboratory scientists work in clinical laboratories, in biomedical research laboratories, in biotech, and in pharmaceutical companies. Laboratory diagnostics have undergone rapid development over the recent years, with the pace showing no signs of abatement. This rapid development challenges the competence of the staff and will most certainly influence the education of future staff. This study aimed to examine what was considered the necessary competencies needed to pursue a career as a biomedical laboratory scientist. Methods A modified Delphi technique was used, with the panel of experts expressing their views in a series of three questionnaire. Consensus was defined as the point which 75 % or more of the panel participants agreed that a particular competency was necessary. Results The study highlights the perceived importance of mostly generic competencies that relate to quality, quality assurance, and accuracy, as well as different aspects of safety, respect, trustworthiness (towards patients/clients and colleagues), and communication skills. The results also stress the significance of self-awareness and professionality. Conclusions We identified important competencies for biomedical laboratory scientists. Together with complementary information from other sources, i.e., guidelines, laws, and scientific publications, the competencies identified can be used as learning outcomes in a competency-based education to provide students with all the competencies needed to work as professional biomedical laboratory scientists.

Protocol for the development of a reporting guideline for causal and counterfactual prediction models in biomedicine

Article

Full-text available

Jun 2022

Introduction While there are guidelines for reporting on observational studies (eg, Strengthening the Reporting of Observational Studies in Epidemiology, Reporting of Studies Conducted Using Observational Routinely Collected Health Data Statement), estimation of causal effects from both observational data and randomised experiments (eg, A Guideline for Reporting Mediation Analyses of Randomised Trials and Observational Studies, Consolidated Standards of Reporting Trials, PATH) and on prediction modelling (eg, Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis), none is purposely made for deriving and validating models from observational data to predict counterfactuals for individuals on one or more possible interventions, on the basis of given (or inferred) causal structures. This paper describes methods and processes that will be used to develop a Reporting Guideline for Causal and Counterfactual Prediction Models (PRECOG). Methods and analysis PRECOG will be developed following published guidance from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network and will comprise five stages. Stage 1 will be meetings of a working group every other week with rotating external advisors (active until stage 5). Stage 2 will comprise a systematic review of literature on counterfactual prediction modelling for biomedical sciences (registered in Prospective Register of Systematic Reviews). In stage 3, a computer-based, real-time Delphi survey will be performed to consolidate the PRECOG checklist, involving experts in causal inference, epidemiology, statistics, machine learning, informatics and protocols/standards. Stage 4 will involve the write-up of the PRECOG guideline based on the results from the prior stages. Stage 5 will seek the peer-reviewed publication of the guideline, the scoping/systematic review and dissemination. Ethics and dissemination The study will follow the principles of the Declaration of Helsinki. The study has been registered in EQUATOR and approved by the University of Florida’s Institutional Review Board (#202200495). Informed consent will be obtained from the working groups and the Delphi survey participants. The dissemination of PRECOG and its products will be done through journal publications, conferences, websites and social media.

Assessing the Relationship between Landscape and Management within Marinas: The Managers’ Perception

Article

Full-text available

Jun 2022

Marinas are maritime features related to nautical tourism. The contemplation of pleasant surroundings acquires great importance in achieving this leisure character. The European Landscape Convention undertakes the necessity of integrating landscape into the planning policies. Thus, the marina’s management decision-making processes should reflect this awareness of the landscape. However, the landscape evaluation has not been appropriately considered despite its importance. This research attempts to introduce an initial framework to evaluate this influence, highlighting the strengths and weaknesses of the different subjects. For this purpose, the most significant elements of the marina management related to the landscape were rated, both from management and landscape perspectives. Two expert panels from Spain were used: 23 experts evaluated the above elements following the Delphi method, and 17 weighted the main management activities using DHP. Results show that there is a lack of concern for the landscape. Managers tend to consider physical conditions, whereas subjective conditions are relegated to the background. In this respect, this methodology provides the first stage for the landscape/management relationship, helping managers identify the main topics and prioritize related actions

Towards a signature pedagogy for technology-enhanced task-based language teaching: Defining its design principles

Article

Full-text available

Jun 2022
ReCALL

This paper reports on ongoing research aimed at characterizing a signature pedagogy (Shulman, 2005) of technology-enhanced task-based language teaching (TETBLT). To achieve this goal, we initially identified 15 pedagogical principles and practices distinctive of TETBLT. This initial set of principles and practices were motivated by second language acquisition theories (Doughty & Long, 2003), methodological approaches in foreign language teaching (Kramsch, 2014), and state-of-the-art publications on computer-assisted language learning (Chapelle & Sauro, 2017). During the first phase of the study, we consulted an initial group of 34 experts in the field, using the Delphi technique to achieve gradual consensus about the set of principles. After analyzing the first set of responses (N = 23) to the principles, which attained a degree of agreement averaging 71% and ranging from 48% to 96%, we refined the principles incorporating the feedback received and sent out a second questionnaire, which allowed us to reach a consensus about a set of eight robust pedagogical principles for TETBLT.

Development of a Consensus-Based Cross-Domain Protocol for the Management of Elastic Compression Stocking Therapy in Patients With Deep Venous Thrombosis and Chronic Venous Disease: A Modified Delphi Study

Article

Full-text available

May 2022

Objective Elastic compression stocking (ECS) therapy is commonly used in patients with deep venous thrombosis (DVT) and chronic venous disease (CVD). The provision of ECS therapy is complex, and studies indicate a lack of practical guidance and suboptimal collaboration among health care professionals. We aimed to reach consensus on critical issues of ECS therapy among the involved health care professionals and patients. Methods A three-round modified Delphi analysis was performed in the Netherlands in which 56 health care professionals (internists, dermatologists, general practitioners, emergency room nurses, home care nurses, medical stocking suppliers, and occupational therapists) and seven patients were invited. The 21 statements included in this analysis were based on information collected from a previously conducted Functional Resonance Analysis Method and Realist Evaluation. We used 7-point Likert scale questions and a 75% threshold for consensus. Results Of the 63 persons invited for this study, 59 (94%) agreed to participate and responded in the first questionnaire round; of whom 52 were health care professionals and seven were patients (five DVT and two CVD). The overall response rate for the three questionnaire rounds was 91%. After completion of the rounds, full consensus was achieved on 19 out of 21 statements. No consensus was reached on the need for a follow-up appointment for CVD patients and who should be responsible to determine the ECS type (custom-made or standard). Conclusion We identified 19 consensus-driven recommendations on treatment decisions and collaboration in ECS therapy among an interdisciplinary panel of health care professionals and patients. These recommendations form a basis for consensus-driven optimization of ECS therapy and should ideally be incorporated in a general cross-domain protocol for ECS therapy in patients with DVT and CVD.

Cross-Cultural Translation and Application of the Lieberman–Brian Inclusion Rating Scale for PE in German-Speaking Countries

Article

Full-text available

Jun 2022
Int J Environ Res Publ Health

Overcoming participation barriers of students with disabilities in physical education is of great importance and an internationally recognized goal. Research highlights that students with disabilities have mixed feelings about their inclusion experiences in physical education. Physical education teachers often do not feel prepared to appropriately support all students. In German-speaking countries in particular, there is a strong tradition of segregation, with varying interpretations of inclusion. In this light, an instrument to reliably assess the inclusive potential of physical education is needed, thereby providing data on the efficacy of teachers’ practices. Such an assessment scale would be important to identify barriers to inclusive physical education while providing teachers with data that could potentially enhance the learning environment. The purpose of this study was to outline initial insights into the cross-cultural translation process of the Lieberman/Brian Inclusion Rating Scale for PE in German-speaking countries. The translation process followed suggestions for transcultural validation. Expert review was provided to check content and face validity. Major item challenges centered around paraeducators, gym management, and conceptual differences regarding physical education.

Evaluating the Performance of Emergency Centers during Coronavirus Epidemic Using Multi-Criteria Decision-Making Methods (Case Study: Sari City)

Article

Full-text available

Jul 2022
DISCRETE DYN NAT SOC

In this study, due to the importance of emergency centers and patient transport vehicles in epidemic conditions, the performance of emergency centers has been evaluated based on health protocols. e criteria were rst divided into preventive and operational sections by collecting opinions, health experts, standard criteria, and the Delphi method. Preventive criteria for evaluating emergency centers and operating criteria for assessing vehicles in these centers are considered. e weighting of the determined criteria was done by the triangular fuzzy aggregation method. According to the standard criteria, the emergency centers have been evaluated for a 30-day period. e results have been assessed as a qualitative and quantitative matrix using the PROMETHEE method. e results showed better performance of Center A (63%) due to proper performance and better compliance with protocols in both criteria (preventive and operational). e reason for the superiority of this center over Center B can be considered the better performance of this center in terms of prevention indicators and better performance of the center's vehicles (Ambulance A-1 and Ambulance A-2) in the performance index by observing the standards.

An overview of the Delphi technique in social pharmacy and health services research

Chapter

Jan 2022

The Delphi technique is a group judgment method which is typically used to reach agreement from a group of people with expertise in a particular area. It is an iterative process where panel members complete questionnaires over several rounds, often rating their agreement/disagreement against a statement, with changes made in later rounds based on the feedback received. It has been used widely in pharmacy-related studies relevant to education, research, and practice. This chapter provides a critical analysis of the various design choices which researchers may consider when planning a Delphi, namely, the panel of participants, the use of the Likert scale, the effect of feedback, what constitutes consensus, and the number of rounds. It also gives an overview of the development and origins of the Delphi, and discusses the advantages and disadvantages of the technique. Advantages include that the Delphi can be conducted with panel members in different geographical locations in their own time; however, the technique can therefore take longer to conduct and lacks face-to-face discussion. This chapter shows the importance of careful planning of the design choices to ensure the reliability and validity of the Delphi.

Design and application of the simulated patient method in pharmacy and health services research

Chapter

Jan 2022

The simulated patient method is becoming an increasingly popular observational method to measure practice behavior in pharmacy practice and health services research. The simulated patient method involves sending a trained individual (simulated patient among other names), who is indistinguishable from a regular consumer, into a healthcare setting with a standardized scripted request. This method has come to be accepted as being well-suited for observing practice in the naturalistic setting and has also been used as an intervention when combined with feedback and coaching. This chapter presents an overview of the method, a brief history of its use, considerations for designing, implementing, and evaluating simulated patient studies, including ethical considerations, as well as methods of analysis and mixed-methods designs.

Consensus evidence-based clinical practice recommendations for the management of fibromyalgia

Article

Dec 2022

Background Because of the subjective character of symptoms, absence of a diagnostic test, modest response to treatments, and, at times, patient reports of important functional disability, fibromyalgia remains a challenge for the treating health care professionals in the standard clinical practice. The aim of this study was to develop an up-to-date consensus and evidence-based clinical practice guidelines for a treat-to-target management of fibromyalgia. Fifteen key clinical questions were identified by a scientific committee according to the Patient/Population, Intervention, Comparison, and Outcomes (PICO) approach. A literature review team performed a systematic review to summarize the evidence advocating the benefits and harms of available pharmacologic and nonpharmacologic therapies for fibromyalgia. Subsequently, recommendations were formulated. The level of evidence was determined for each section using the Oxford Centre for Evidence-based Medicine (CEBM) system. A 3-round Delphi process was conducted with 16 experts. All rounds were conducted online. A consensus was achieved on the direction and the strength of the recommendations. Results An online questionnaire was sent to an expert panel who participated in the three rounds (response rate 100%). At the end of round 3, a total of fifteen recommendation items, categorized into 10 sections to address the main fibromyalgia categories, were obtained. Agreement with the recommendations (ranks 7–9) ranged from 85 to 100%. Consensus was reached (i.e., ≥ 80% of respondents strongly agreed or agreed) on the wording of all the 15 clinical standards identified by the scientific committee. An algorithm for the management of fibromyalgia has been suggested. Conclusions These recommendations provide an updated consensus on both the non-pharmacological and the pharmacological treatments of fibromyalgia. The provided strategies to reach optimal treat-to-target outcomes in common clinical scenarios are based on a combination of evidence and expert opinions. Best treatment decisions should be tailored to each individual patient situation.

Cybersecurity service level agreements: understanding government data confidentiality requirements

Article

May 2022

Cybersecurity requirements, such as data security, are often used as evidence for the Government's relationship with external service providers to process, store and transmit sensitive government data. However, cybersecurity researchers have not profoundly studied the practical application of government data security requirements (e.g. data confidentiality) in service level agreements (SLAs) in the context of an outsourced scenario. The relationships with external service providers are usually established through SLAs as trust-enhancing instruments. However, there is a concern that existing SLAs mainly focus on the system availability and performance aspects but overlook cybersecurity requirements (e.g. data security) in SLAs. Such an understanding is essential to develop government SLA data confidentiality requirements into the formulation of security-related SLAs. We seek to provide insights by developing and conducting a grounded adaptive Delphi method (GADM) with 35 government participants through group discussions and individual sessions. The work on the Indonesian Government's data confidentiality requirements was used as a case study. This paper provides insights into three understandings of the increasing considerations of the Government's data confidentiality requirements in SLA definitions. The three perceptions of security-related SLAs are the target of protection, the data confidentiality risks and the government SLA data confidentiality requirements. Our findings play important implications for a better understanding of how to incorporate data confidentiality requirements according to perceived threats for government data classification in security-SLAs. Based on these findings, we recommend that the Government and service providers improve existing security-related SLAs and future research lines.

Predictive models for human–AI nexus in group decision making

Article

May 2022

Machine learning (ML) and artificial intelligence (AI) have had a profound impact on our lives. Domains like health and learning are naturally helped by human–AI interactions and decision making. In these areas, as ML algorithms prove their value in making important decisions, humans add their distinctive expertise and judgment on social and interpersonal issues that need to be considered in tandem with algorithmic inputs of information. Some questions naturally arise. What rules and regulations should be invoked on the employment of AI, and what protocols should be in place to evaluate available AI resources? What are the forms of effective communication and coordination with AI that best promote effective human–AI teamwork? In this review, we highlight factors that we believe are especially important in assembling and managing human–AI decision making in a group setting. Our simplified representation in Figure 1 is now augmented to include (1) the presence of AI agents in the group, and (2) the adoption of ML models to predict the behavior of a human+AI group. In this review, we confine our ML modeling efforts to the specific tasks of predicting the emergence of influence systems (the left big arrow) and the prediction of decisions under multiple risk/reward scenarios (the right big arrow). Note that ML techniques can be utilized broadly in other tasks in the group environment, such as deciding optimal group composition and the design of interventions.

Nurses' Priorities for Perioperative Research in Africa

Article

May 2022

Background: Mortality rates among surgical patients in Africa are double those of surgical patients in high-income countries. Internationally, there is a call to improve access to and safety of surgical and perioperative care. Perioperative research needs to be coordinated across Africa to positively impact perioperative mortality. Methods: The aim of this study was to determine the top 10 perioperative research priorities for perioperative nurses in Africa, using a research priority-setting process. A Delphi technique with 4 rounds was used to establish consensus on the top 10 perioperative research priorities. In the first round, respondents submitted research priorities. Similar research priorities were amalgamated into single priorities when possible. In round 2, respondents ranked the priorities using a scale from 1 to 10 (of which 1 is the first/highest priority, and 10 is the last/lowest priority). The top 20 (of 31) were determined after round 2. In round 3, respondents ranked their top 10 priorities. The final round was an online discussion to reach consensus on the top 10 perioperative research priorities. Results: A total of 17 perioperative nurses representing 12 African countries determined the top research priorities, which were: (1) strategies to translate and implement perioperative research into clinical practice in Africa, (2) creating a perioperative research culture and the tools, resources, and funding needed to conduct perioperative nursing research in Africa, (3) optimizing nurse-led postoperative pain management, (4) survey of operating theater and critical care resources, (5) perception of, and adherence to sterile field and aseptic techniques among surgeons in Africa (6) surgical staff burnout, (7) broad principles of infection control in surgical wards, (8) the role of interprofessional communication to promote clinical teamwork when caring for surgical patients, (9) effective implementation of the surgical safety checklist and measures of its impact, and (10) constituents of quality nursing care. Conclusions: These research priorities provide the structure for an intermediate-term research agenda for perioperative research in Africa.

Framework for Measuring Process Innovation Performance at Indonesian State-Owned Companies

Article

May 2022

Several state-owned companies (SOEs) have successfully implemented process innovation. This paper examines the success factors that influence the successful implementation of process innovation in Indonesian state-owned companies. The present study used the three stages of an online questionnaire in the Delphi method to obtain consensus from experts. The consensus was measured based on the mean and standard deviation of the assessment answers provided by our respondents. Based on the development of process innovation implementation frameworks shown in previous studies, we obtained 28 factors that influence the successful implementation of process innovation. Our panelists believed that human resources and organizational factors, such as leadership, problem understanding, strategy, and culture, affected the success of process innovation in SOEs, even more than the research and development budget, support, and commitment factor of a given company. Meanwhile, cost efficiency was the main factor driving a company’s motivation to implement process innovation. The success factors in implementing process innovation are an essential consideration for the management of other SOEs eager to improve their company’s innovation performance, especially process innovation. Success factors were carried out comprehensively for all implementation indicators of process innovation, including inputs, process (idea generation, idea selection, and idea implementation), outputs and outcomes, diffusion, culture, strategies, and push and pull factors.

Preservice Music Teachers’ World Music Preference for Taught Pieces and Its Transfer to Untaught Pieces

Article

Jul 2022
J Res Music Educ

We investigated whether preference transfer occurred from 18 taught world music pieces to 18 untaught pieces from the same regions among preservice music teachers. Eighty-three preservice music teachers participated in an interactive, culturally diverse music curriculum over the course of a semester. Participants learned 18 African, Asian, and Latin American folk songs in small groups from culture bearers via video conferencing. They then team-taught the same songs to their peers. Before and after the curriculum intervention with taught pieces only, we tested participants’ preferences for the taught and untaught pieces. Wilcoxon signed-rank test results indicated that the participants’ preference scores increased from the pretest to posttest for both taught and untaught world music pieces in all three regions. This result suggests the existence of possible preference transfer from taught to untaught pieces. Through follow-up interviews and instructor’s reflective notes analyses, we identified four interconnected themes across cognitive, cultural, social, and pedagogical dimensions that explained reasons for preference transfer: applying principles from taught pieces, appreciating cultures with authenticity and legitimacy, learning safely via intra- and intergroup interactions, and experiencing reciprocal course structure. We discuss implications for the profession that bridge formal and informal endeavors of teaching and learning music from diverse cultures.

Sector Perspectives on the Attributes of System Approaches to Water, Sanitation, and Hygiene Service Delivery

Article

Jun 2022

This study sought to coalesce sector knowledge on the use of systems approaches for sustainable water, sanitation, and hygiene (WASH) service delivery in low-income countries. To accomplish this objective, we remotely convened a panel of WASH sector experts within a multiround survey to identify, characterize, and prioritize the necessary attributes of WASH systems approaches. The first survey round asked experts to comment on the conceptual differences between traditional approaches and systems approaches to WASH. Emerging within responses from the first survey round were attributes of WASH systems (i.e., factors, actors) and attributes of WASH systems approaches (i.e., flexible, scalable). A three-round Delphi survey was then administered to reach consensus on these emerging attributes. By the final round of the Delphi, consensus was reached on every attribute of WASH systems, indicating alignment between the experts on the fundamental characteristics and implications of WASH systems. Consensus was also reached on the majority of attributes of WASH systems approaches, including the inherent interconnected and complex nature of WASH systems. However, consensus was not achieved on attributes related to mapping WASH system interconnections, convening stakeholders in collective action, and the appropriateness and feasibility of applying systems approaches at scale. This indicates a need for future research that explores practical and scalable tools and techniques to map and evaluate WASH system interactions, and ways to engage relevant actors in these approaches to collectively apply systems knowledge. © 2022 This work is made available under the terms of the Creative Commons Attribution 4.0 International license,.

Artificial Intelligence and Machine Learning: Exploring drivers, barriers, and future developments in marketing management

Article

Oct 2022
J BUS RES

Companies neither fully exploit the potential of Artificial Intelligence (AI), nor that of Machine Learning (ML), its most prominent method. This is true in particular of marketing, where its possible use extends beyond mere segmentation, personalization, and decision-making. We explore the drivers of and barriers to AI and ML in marketing by adopting a dual strategic and behavioral focus, which provides both an inward (AI and ML for marketers) and an outward (AI and ML for customers) perspective. From our mixed-method approach (a Delphi study, a survey, and two focus groups), we derive several research propositions that address the challenges facing marketing managers and organizations in three distinct domains: (1) Culture, Strategy, and Implementation; (2) Decision-Making and Ethics; (3) Customer Management. Our findings contribute to better understanding the human factor behind AI and ML, and aim to stimulate interdisciplinary inquiry across marketing, organizational behavior, psychology, and ethics.

Closing the communication gap in neonatal inter-hospital transfer: a neonatal referral form for resource-limited settings - a modified e-Delphi-consensus study

Article

Jun 2022

Background: Standardised neonatal referral forms (NRFs) facilitate effective communication between healthcare providers and ensure continuity of care between facilities, which are essential for patient safety. We sought to determine the essential data items, or core clinical information (CCI), that should be conveyed for neonatal inter-hospital transfer in resource-limited settings (Rounds 1 to 3) and to create an NRF suitable for our setting (Round 4). Methods: We conducted an international, four-round, modified Delphi-consensus study. Round-1 was a literature and internet search to identify existing NRFs. In Round-2 and -3, participants were Rwandan clinicians and international paediatric healthcare practitioners who had worked in Rwanda in the five years before the study. These participants evaluated the draft items and proposed additional items to be included in an NRF. Round-4 focused on creating the NRF and used five focus groups of Rwandan general practitioners at district hospitals. Results: We identified 16 pre-existing NRFs containing 125 individual items. Of these, 91 items met the pre-defined consensus criteria for inclusion in Round-2. Only 33 items were present in more than 50% of the 16 NRFs, confirming the need for this consensus study. In Round-2, participants proposed 12 new items, six of which met the pre-defined consensus criteria. In Round-3, participants scored items for importance, and 57 items met the final consensus criteria. In Round-4, 29 general practitioners took part in five focus groups; a total of 16 modifications were utilised to finalise the NRF. Conclusions: We generated a novel, robust, NRF that may be readily employed in resource-limited settings to communicate the essential clinical information to accompany a neonate requiring inter-hospital transfer.

Development of a core measurement set for research in degenerative cervical myelopathy: a study protocol (AO Spine RECODE-DCM CMS)

Article

Full-text available

Jun 2022

Introduction Progress in degenerative cervical myelopathy (DCM) is hindered by inconsistent measurement and reporting. This impedes data aggregation and outcome comparison across studies. This limitation can be reversed by developing a core measurement set (CMS) for DCM research. Previously, the AO Spine Research Objectives and Common Data Elements for DCM (AO Spine RECODE-DCM) defined ‘what’ should be measured in DCM: the next step of this initiative is to determine ‘how’ to measure these features. This protocol outlines the steps necessary for the development of a CMS for DCM research and audit. Methods and analysis The CMS will be developed in accordance with the guidance developed by the Core Outcome Measures in Effectiveness Trials and the Consensus-based Standards for the selection of health Measurement Instruments. The process involves five phases. In phase 1, the steering committee agreed on the constructs to be measured by sourcing consensus definitions from patients, professionals and the literature. In phases 2 and 3, systematic reviews were conducted to identify tools for each construct and aggregate their evidence. Constructs with and without tools were identified, and scoping reviews were conducted for constructs without tools. Evidence on measurement properties, as well as on timing of assessments, are currently being aggregated. These will be presented in phase 4: a consensus meeting where a multi-disciplinary panel of experts will select the instruments that will form the CMS. Following selection, guidance on the implementation of the CMS will be developed and disseminated (phase 5). A preliminary CMS review scheduled at 4 years from release. Ethics and dissemination Ethical approval was obtained from the University of Cambridge (HBREC2019.14). Dissemination strategies will include peer-reviewed scientific publications; conference presentations; podcasts; the identification of AO Spine RECODE-DCM ambassadors; and engagement with relevant journals, funders and the DCM community.

Developing of Multidimensional Perspectives Checklist of Professionalism for Undergraduate Occupational Therapy Students in Assistive Technology Service: Delphi Study

Article

Jun 2022
Int J Environ Res Publ Health

Professionalism is a critical attribute that occupational therapy students must establish throughout education, especially in the context of assistive technology (AT). This study aimed to construct a multidimensional perspectives checklist (MPC) from the stakeholders to reflect professionalism development in the AT service courses. Fifteen experts in AT clinical practice and educators (N = 15) consented to and completed three rounds of the Delphi survey. Data were qualitatively analyzed to form a preliminary checklist in the first round. In rounds two and three, participants rated their level of agreement with the MPC items. A descriptive analysis of consensus was performed. Thirty items were classified into five subscales (teachers, therapists, patients, peers, learners) and fit into a framework with three dimensions and seven categories. After the Delphi survey, the MPC achieved high consensus, convergence, and stability. Two additional categories of professionalism emerged in the study, namely aesthetic and personal characteristics and reflection. The MPC developed in this study reflects the perspectives of various stakeholders in occupational therapy practice, providing helpful information for students to prepare themselves. Therefore, the MPC could contribute to expanding and developing the scope of professionalism in occupational therapy, especially in AT service.

Communication support needs assessment in dementia (CoSNAT‐D): An international content validation study

Article

Jun 2022

The decline of language and communication abilities is common among people living with dementia and impacts on many areas of everyday life, including active participation in social activities and decision‐making. Despite a growing body of supporting evidence for approaches that address language and communication decline in dementia, the concept of communication rehabilitation is largely neglected in this population. This paper reports on the content validation of a novel tool, the Communication Support Needs Assessment Tool for Dementia (CoSNAT‐D). The tool has been developed to assist in the initial identification of communication difficulties and related support needs of people living with dementia. Importantly, the CoSNAT‐D is the only available tool that takes a three‐way informed approach, considering the view of the person living with dementia, their carer and an administering healthcare professional. Content validity was established between September and December 2018 using a modified Delphi approach. An international expert panel rated 32 items of a face‐validated item pool regarding their importance and relevance through an iterative feedback process. Consensus was pre‐determined at 70% of agreement for both importance and relevance of an item. Data were analysed using descriptive statistics and qualitative content analysis of comments provided in each round. Twenty‐eight experts working in dementia, language and communication participated in the Delphi survey. Qualitative analysis resulted in the addition of five items, of which three reached the required consensus in Round 3. Consensus was established for 35/37 items in three rounds. The pilot version of the CoSNAT‐D demonstrates adequate content validity and face validity. The use of the CoSNAT‐D may assist a range of healthcare professionals in the decision‐making process about appropriate next management steps, and thereby improve the care path for people with dementia and language and communication impairment. The establishment of further psychometric properties is warranted.

Expert consensus for a national essential antidote list: E-Delphi method

Article

Jun 2022
PLOS ONE

Antidote stocking represents a major challenge to hospitals all over the world, including Kuwait. In order to assist hospitals to reduce costs and improve patient care, an essential antidote list can be used as an initial foundation for securing sufficient antidote availability at healthcare institutions. The aim of our study is to generate a nationally relevant essential antidote list for emergency care hospitals in Kuwait using the e-Delphi method by establishing consensus through a multidisciplinary expert group of healthcare providers. An electronic survey with 47 essential antidotes was developed. The e-Delphi method was used, with three rounds of voting, to determine expert consensus on an essential antidote list for hospitals in Kuwait. A purposive sample of healthcare professionals from governmental and private hospitals were selected for this study (n = 30). Consensus was gained if ≥75% of the expert panel agreed on the inclusion of the antidote, without any strong disagreements. Round 1 of the e-Delphi resulted in 41 antidotes reaching consensus and seven new antidotes suggested by the expert panel. Round 2 had two antidotes (out of seven newly suggested ones) reaching consensus. Round 3 was a confirmatory round, where the expert group agreed on their previous rounds’ opinions. This resulted in the development of an essential antidote list with 43 antidotes. The optimal approach for ensuring adequate availability of antidotes is continuous monitoring of local poisoning incidence and antidote requirements through collaborations between academic researchers and emergency care clinicians. The development of an essential antidote list, with expert consensus, is one of the initial steps in securing a foundation for appropriate provision of antidotes at all healthcare institutions. This is the first study that the authors are aware of that demonstrates that the e-Delphi technique can consolidate recommendations of experts in emergency medicine to provide a list of essential antidotes.

Predictors of expert providers’ familiarity with intervention practices for school- and transition-age youth with autism spectrum disorder

Article

Jun 2022
AUTISM

Lay abstract: School-age children, adolescents, and young adults with autism spectrum disorder encounter many different types of providers in their pursuit of treatment for anxiety, behavior problems, and social difficulties. These providers may all be familiar with different types of intervention practices. However, research has not yet investigated patterns in expert providers' familiarity with different practices nor how these patterns are related to the characteristics of providers (years in practice, academic discipline, setting) and the youth (age and intellectual disability) they typically support. A panel of 53 expert transdisciplinary providers rated their familiarity with 55 intervention practices (derived from research and expert nominations) via an online Delphi poll. Advanced statistical methods were used to identify types of intervention practices with which providers were familiar, which included two approaches (cognitive and behavioral) and two strategies (engagement and accessibility). Providers who practiced outside a school setting or treated clients without intellectual disability were more familiar with cognitive approaches. Clinical psychologists, behavior analysts, and school-based providers were more familiar with behavioral approaches. Providers practicing outside school settings were also more familiar with engagement strategies, and providers with more years in practice were more familiar with accessibility strategies. These results may help families and researchers to better anticipate how services may vary depending on the types of autism spectrum disorder providers seen and work to reduce disparities in care that may result.

Developing an Entrepreneurial Competence Framework for Tertiary Arts Education Through a Delphi Method – A Chinese Perspective

Chapter

Jun 2022

In recent years, artists who characterise themselves as freelancers with short-term contracts or running their own businesses are known as “arts entrepreneurs” rather than “artists”. In China, arts entrepreneurship is a new discipline while arts-specific entrepreneurial competencies have not yet been comprehensively identified. The purpose of this study was validating an entrepreneurial competence framework for tertiary arts education in China. A modified Delphi method was used, which involved a consensus approach through three surveys of an expert multidisciplinary panel with more than 10 years’ experience in the field. Results from 53 experts who completed all 3 rounds of survey indicated a wide range of entrepreneurial competences that could be taught at tertiary level to support artists to successfully navigate their current and future careers. Consensus centered 37 essential competencies of the 3 domains: general competence, artistic competence and entrepreneurial competence. The results of this study offer implications and directions for future research. The validated competencies can inform analyses of the extent to which these competencies are actually attained and pursued in arts entrepreneurship education programmes in China.

Strategizing national health in the 21st century: a handbook

Book

Full-text available

Nov 2016

This handbook is designed as a resource for providing up-to-date and practical guidance on national health planning and strategizing for health. It establishes a set of best practices to support strategic plans for health and represents the wealth of experience accumulated by WHO on national health policies, strategies and plans (NHPSPs). WHO has been one of the leading organizations to support countries in the development of NHPSPs. The focus on improving plans has grown in recent years, in recognition of the benefits of anchoring a strong national health sector in a written vision based on participation, analysis, and evidence.

Publishing clinical prActice GuidelinEs (PAGE): Recommendations from editors and reviewers

Article

Jun 2022

Transparency Ecosystem for Research and Journals in Medicine (TERM) Working Group summarized the essential recommendations that should be considered to review and publish a high‐quality guideline. These recommendations from editors and reviewers included the ten components of essential requirements: systematic review of existing relevant guidelines, guideline registration, guideline protocol, stakeholders, conflicts of interest, clinical questions, systematic reviews, recommendation consensus, guideline reporting, and external review. TERM Working Group abbreviates them as PAGE (essential requirements for Publishing clinical prActice GuidelinEs), recommends guideline authors, editors, and peer reviewers use them for high‐quality guidelines.

Container vessel selection for maritime shipping companies by using an extended version of the Grey Relation Analysis (GRA) with the help of Type-2 neutrosophic fuzzy sets (T2NFN)

Article

Jul 2022
COMPUT IND ENG

Contrary to expectations, based on occurring changes in customer behaviours related to consuming and shopping triggered by the pandemic, the global container shipping market has continued to grow during COVID 19. Experts estimate that these increases will continue in the future due to changing consumption habits. However, container shipping companies (CSCs) may soon encounter many troubles and challenging situations. They have an extremely fragile structure and may be entirely unprotected when they encounter unexpected situations sourced from external factors. (i.e., due to grounding of a ship, complete blockage of the Suez channel for three weeks can be given as a clear example of that). Hence, selecting an appropriate container vessel type can help construct a healthier container shipping system less influenced by adverse conditions for decision-makers and practitioners. Besides, it can provide a more effective and productive maritime transportation environment for all stakeholders. However, selecting a proper container vessel type is a complicated decision-making problem since many conflicting criteria and complex ambiguities exist. The current paper proposes an extended version of the GRA technique with the help of type-2 neutrosophic fuzzy sets (T2NFN) for capturing and processing uncertainties better than the traditional MCDM frameworks. According to the obtained results, C6, container carrying capacity, is the most influential criterion and the type of post-Suezmax container vessel is the best option for the CSCs, as it provides advantages at a satisfactory level for almost all criteria than others. After the proposed model was applied, a comprehensive sensitivity analysis (SA) was performed to test the validity of the T2NFN GRA approach. The results of SA approve the applicability, effectivity, and robustness of the model.

EUCAPA 2022 Book of Abstracts

Article

Jun 2022

European Conference of Adapted Physical Activity

Group techniques for program planning

Article

Full-text available

Jan 1975

The Delphi Method: Techniques and Applications

Book

Full-text available

Jan 1975

Research guidelines for the Delphi Survey Technique

Article

Full-text available

Nov 2000

Research guidelines for the Delphi survey technique Consensus methods such as the Delphi survey technique are being employed to help enhance effective decision-making in health and social care. The Delphi survey is a group facilitation technique, which is an iterative multistage process, designed to transform opinion into group consensus. It is a flexible approach, that is used commonly within the health and social sciences, yet little guidance exists to help researchers undertake this method of data collection. This paper aims to provide an understanding of the preparation, action steps and difficulties that are inherent within the Delphi. Used systematically and rigorously, the Delphi can contribute significantly to broadening knowledge within the nursing profession. However, careful thought must be given before using the method; there are key issues surrounding problem identification, researcher skills and data presentation that must be addressed. The paper does not claim to be definitive; it purports to act as a guide for those researchers who wish to exploit the Delphi methodology.

EASE-scale (Examination of Anomalous Self-Experience)

Article

Full-text available

Sep 2005

The Examination of Anomalous Self-Experience ( EASE) is a symptom checklist for semi-structured, phenomenological exploration of experiential or subjective anomalies that may be considered as disorders of basic or 'minimal' self-awareness. The EASE is developed on the basis of self-descriptions obtained from patients suffering from schizophrenia spectrum disorders. The scale has a strong descriptive, diagnostic, and differential diagnostic relevance for disorders within the schizophrenia spectrum. This version contains interview-specific issues and psychopathological item descriptions ( Manual), a scoring sheet ( Appendix A), a reminder list of items for use during the interview ( Appendix B) and an EASE/BSABS ('Bonner Skala fur die Beurteilung von Basis-symptomen') item comparison list (Appendix C).

Forecasting Technology for Planning Decisions

Book

Jan 1978

A Delphi evaluation of agreement between organizations

Article

Jan 1975

C.G. Jones

Forecasting risks in the tourism industry using the Delphi technique

Article

Jan 2002

S. Cunliffe

This paper describes a research design aimed at examining the risks and impacts of natural and anthropogenic catastrophic events on the future of the tourism industry. The focus is on the selection of a survey method, the Delphi technique, and the reasons behind selecting this qualitative forecasting tool. The Delphi technique relies primarily on the intuitive judgment of experts. This paper argues that with thorough pre-planning and rigorous management of the survey, the Delphi technique can provide futures information that other conventional extrapolative methods cannot reliably forecast.

Projecting the next decade in safety management: A Delphi technique study

Article

Jan 2001
Prof Saf

S.J. Adams

Foundations of Behavioral Research

Article

Jan 1973

F. N. Kerlinger

Experimental assessment of Delphi procedures with group value judgments

Article

Jan 1972

Delphi inquiries and knowledge utilization

Article

Jan 1975

J. Ludlow

The Modified Delphi Technique - A Rotational Modification

Article

May 1999

The Delphi Forecasting Method

Article

Jan 1971
PHI DELTA KAPPAN

W.T. Weaver

Delivery methodology of the Delphi: A comparison of two approaches

Article

Jan 2001

An experimental study of group opinion

Article

Jan 1969

NC Dalkey

An Experimental Application of the DELPHI Method to the Use of Experts

Article

Apr 1963

Olaf Dalkey Norman; Helmer

This paper gives an account of an experiment in the use of the so-called DELPHI method, which was devised in order to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in depth interspersed with controlled opinion feedback.

The methodological worth of Delphi forecasting technique

Article

Dec 1975
TECHNOL FORECAST SOC

Focusing on the general issues of reliability and validity, several specific problems evident in reported applications of the Delphi method to forecasting are discussed. Reliability threats, which are noted in a number of procedural aspects of the method, arise from ill-considered procedural variations and lack of standardization. While validity threats are also found in several procedural aspects of the tool, they arise principally from pressures for convergence of predictions. This feature of the method, along with certain structural characteristics, is found to undermine critically its forecasting ability. Having discussed in some detail the nature of these difficulties, the paper closes with consideration of the reasons for the continued use of Delphi in spite of its shortcomings and with comments on alternative approaches.

Assessing Needs in Educational and Social Programs

Article

Jan 1986
EDUC EVAL POLICY AN

Considerations in the Application of a Modified Scree Test for Delphi Survey Data

Article

Apr 1991
EVALUATION REV

Recently, Keith Zoski and Stephen Jurs proposed a modified version of the scree test for analyzing Delphi survey data. Although the test has merit, three considerations or problems affecting its use are examined and examples of data related to those problems are given. Discussion is also provided regarding appropriate and inappropriate applications of the modified scree test to Delphi data.

Book Review: Planning and Conducting Needs Assessments: A Practical Guide

Article

Jun 1997
AM J EVAL

Paula Brook

Use of Delphi methods in higher education

Article

Dec 1972
TECHNOL FORECAST SOC

Robert C. Judd

This paper is concerned with describing the principal uses of the Delphi method in higher education. These areas of use are found to be (1) searching out goals and objectives, (2) campus and curriculum planning, and (3) development of evaluation criteria. Important to understanding the use of Delphi in a higher education setting are the methodological adaptations that have been employed. Delphi as used in technological forecasting requires modification when used in the generation and assessment of goals. Three principal problems in methodology are examined here. They are (1) selection of the panel, (2) the character of round one and (3) the consequences of feedback.

Technological Forecasting for Decision Making

Article

Jan 1972

Joseph P. Martino

The Delphi method: an experimental study of group opinion

Article

Sep 1969
FUTURES

Dalkey NC

This report describes the results of an extensive set of experiments, conducted at RAND, concerned with evaluating the effectiveness of the Delphi procedures for formulating group judgements. The study represents a small beginning in the field of research that could be called ‘opinion technology’, and is of direct relevance for the use of experts as advisers in decision making, especially in areas of broad or long-range policy formulation.

On the Epistemology of the Inexact Science

Article

Jan 1960

Delphi Technique: Expanding Applications.

Article

Jan 1979

Kenneth W. Brooks

Describes the Delphi Technique, an approach to elicit and refine the opinions of a group of people. Outlines the steps in the process and provides illustrations of the applications of the technique for schools and colleges in determining future research needs, assessing staff development needs, and identifying problems confronting administrators. (MB)

Instructional Technology: Past, Present, and Future. Second Edition

Article

Jan 1995

Anglin, Gary J., Ed

Leading professionals in the field discuss instructional design and systems, computer applications in education and training, research and evaluation in instructional technology, future prospects for instructional technology, and professional development. The book contains six parts. A brief outline of instructional development and educational technology is given in part 1. This is followed by consideration in part 2 of the current state of the art, trends and issues, and challenges in linking the cognitive sciences with the instructional design process. Instructional development, including needs assessment and task analysis procedures are covered in part 3, and part 4 reviews various applications of instructional technology, such as the Internet and distance education. Parts 4 and 5 present an overview of current research and cover certification and professional development in the field. New to this edition are seven chapters that address such current topics as educational systems development and instructional systems development, postmodernism and instructional technology, interactive technologies, the Internet and higher education, qualitative research, and instructional technology and attitude change. (Author/AEF)

The Delphi Technique: A Case Study

Article

Jan 1971
PHI DELTA KAPPAN

Delphi technique as used by the University of Virginia School of Education. (LR)

The Content of a College-Level Outdoor Leadership Course

Article

Jan 1982

Paul Green

This research study used the Delphi technique to determine the ideal content of a college-level outdoor leadership course for land-based outdoor pursuits in the Pacific Northwest. Topics were generated and value-rated by 61 Pacific Northwest outdoor leaders using a Likert-type scale in three separate questionnaires. Thirty-five topics were retained in the ideal content, of which the top 10 were: (1) risk management plans; (2) judgement; (3) wilderness ethics; (4) first aid; (5) analyzing risks; (6) minimum impact practices; (7) outdoor leadership objectives; (8) hazard analysis for hypothermia; (9) backcountry first aid; and (10) minimum impact philosophy. An outdoor leadership course outline constructed from research results included philosophy of outdoor leadership, risk management, decision making, outdoor leadership methods, teaching principles, small group dynamics in the outdoors, environmental awareness skills, pre-trip planning, hazards analysis, and map reading and interpretation. Tables list final rankings and mean scores of the 35 consensus topics and the 23 topics rated least important. An appendix provides a copy of the third and final questionnaire with ratings for 176 topics. (LFL)

Predicting Future Trends in Adult Fitness Using the Delphi Approach

Article

Jun 1987

This study determined how experts see the future of adult fitness by using the Delphi Technique as a measurement tool. Experts (N = 50), all Fellows of the American College of Sports Medicine, were selected on the basis of their reputations in adult fitness. Three questionnaires were submitted to the panel. In the first questionnaire each expert listed five or more events, trends, or developments they thought would take place and five or more events they would like to see take place in the next 25 years. From the events and trends listed on this questionnaire, 100 items were generated that formed the basis for the last two questionnaires. Responses to the items on the second questionnaire were evaluated on a 5-point Likert scale for probability and desirability. In the third questionnaire, group means and standard deviations were calculated and recorded for each item. Only those items which were significantly (p < .01) higher than 3.0, the average probability or desirability response on the Likert scale, were included in the analysis. The items were divided into seven categories: fitness in sport and recreation, fitness and wellness, fitness in industry, fitness in education, fitness in business, research in fitness, and fitness equipment and facilities. The highest single mean for any item for both probability (M = 4.31) and desirability (M = 4.83) was increased awareness of health and fitness among the elderly.

The utilization of expert opinion in decision‐making

Article

Nov 1972
AICHE J

Albert J. Klee

This is a review of the state-of-the-art of rational and primarily quantitative approaches to the utilization of expert opinion in the inexact sciences, among which the engineering sciences must be included. Of primary concern is the decision-maker's utilization of one or more experts, and the criteria and scoring rules used to evaluate their performance. Also reviewed are certain game-theoretic strategies used by experts to maximize their own goal satisfaction. With this review, it is hoped that chemical engineers, when acting as decision-makers, will be able to utilize expert opinion more effectively and will better understand the function of experts when it is necessary for them to adopt such roles themselves.

Methodological problems associated with the use of the Delphi technique — some comments

Article

Feb 1988

E. W. Marchant

The development of indicators for sustainable tourism: Results of a Delphi survey of tourism researchers

Article

Aug 2001
TOURISM MANAGE

Graham Miller

This paper presents the results of a two round Delphi survey conducted into expert opinion on the development of indicators to measure the movement of the tourism product at a company/resort level towards a position of greater or lesser sustainability. This research forms part of a wider project to develop indicators that consumers can use in the selection of their holidays and promote a more sustainable form of tourism. The results of this expert survey show considerable disagreement over “sustainability” and where the borders of the concept exist. In addition, the research identified contrasting views over the use of qualitative versus quantitative indicators and the role that consumer pressure can play. The use of the Delphi technique to address complex and uncertain issues is also explored.

The Delphi method: Substance, context, a critique and an annotated bibliography

Article

Feb 1971
Soc Econ Plann Sci

Juri Pill

This paper is concerned with a methodology for efficiently obtaining concensus from a panel of evaluators on questions which are shrouded in uncertainty and can not be measured or evaluated in the classical sense. The methodology and its historical development are described, it is placed in context vis-a-vis other methods of subjective scaling, and a critique is provided. An extensive and current annotated bibliography is presented.

Internationalizing extension : an exploration of the characteristics evident in a state university extension system that achieves internationalization /

Article

Thesis (Ph. D.)--Ohio State University, 1994. Includes bibliographical references (leaves 163-170). Advisor: R. Kirby Barrick, Dept. of Agricultural Education. System requirements: World Wide Web browser and PDF viewer.

Forecasting through hierarchical Delphi.

Article

Thesis--Ohio State University. Bibliography: leaves 171-179.

The use of the Delphi method in organizational communication : a case study /

Article

Lois Merle. Kaplan

Thesis (M.A.)--Ohio State University, 1971. Includes bibliographical references (leaves 88-91).

Development of an instrument to evaluate intercollegiate athletic coaches [microform] : a modified Delphi study /

Article

Carol Ann. Eckman

Thesis (Ed. D.)--West Virginia University, 1983. Vita. Includes bibliographical references (leaves 254-259). Microfilm. s

Delphi technique. Lecture, Applied evaluation design

Jan 2003

J W Altschuld

Altschuld, J. W. (2003). Delphi technique. Lecture, Applied evaluation design. The Ohio State University.

Using the Delphi process to identify significant recreation research-based innovations

Jan 1993
25-36

D H Anderson
I E Schneider

Anderson, D. H., & Schneider, I. E. (1993). Using the Delphi process to identify significant recreation research-based innovations. Journal of Park and Recreation Administration, 11 (1), 25-36.

A comparative study of the effectiveness of decision making processes which utilize the Delphi and leaderless group methodologies. Unpublished doctoral dissertation

Jan 1983

D C Douglas

Douglas, D. C. (1983). A comparative study of the effectiveness of decision making processes which utilize the Delphi and leaderless group methodologies. Unpublished doctoral dissertation, The Ohio State University, Columbus.

Essential assessment criteria for physical education teacher education programs: A Delphi study

Jan 1996

J M Jacobs

Jacobs, J. M. (1996). Essential assessment criteria for physical education teacher education programs: A Delphi study. Unpublished doctoral dissertation, West Virginia University, Morgantown.

Priorities within the health care system: A Delphi survey

Jan 1981

C A Lindeman

Lindeman, C. A. (1981). Priorities within the health care system: A Delphi survey. Kansas City, MO: American Nurses' Association.

Rethinking the outlook of colleges whose roots have been in agriculture

Jan 1992

J H Meyer

Meyer, J. H. (1992). Rethinking the outlook of colleges whose roots have been in agriculture. Davis, CA: University of California.

Determining what could/should be: The Delphi technique and its application

Oct 2006

L E Miller

Miller, L. E. (2006, October). Determining what could/should be: The Delphi technique and its application. Paper presented at the meeting of the 2006 annual meeting of the Mid-Western Educational Research Association, Columbus, Ohio.

The Delphi method: Techniques and applications

Jan 1975
262-287

M Scheibe
M Skutsch
J Schofer

Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linstone, & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 262-287). Reading, MA: Addison-Wesley Publishing Company.

Tourism marketing and management handbook

Jan 1989

R E Taylor
L L Judd

Taylor, R. E., & Judd, L. L. (1989). Delphi method applied to tourism. In S. Witt, & L. Moutinho, (Eds.). Tourism marketing and management handbook. New York: Prentice Hall.

Gazing into the oracle: The Delphi method and its application to social policy and public health

Jan 1996
56-88

M Turoff
S R Hiltz

Turoff, M., & Hiltz, S. R. (1996). Computer based Delphi process. In M. Adler, & E. Ziglio (Eds.). Gazing into the oracle: The Delphi method and its application to social policy and public health (pp. 56-88). London, UK: Jessica Kingsley Publishers.

Human resource development: The theory and practice of need assessment

Jan 1983

F L Ulschak

Ulschak, F. L. (1983). Human resource development: The theory and practice of need assessment. Reston, VA: Reston Publishing Company, Inc.

Planning and conducting needs assessment: A practical guide

Jan 1995

B R Witkin
J W Altschuld

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage Publications, Inc.

The Delphi Technique: Making Sense Of Consensus

Abstract

No full-text available

Recommendations

Organizational Mindfulness in Higher Education Information Technology: A Delphi Study

Educational Resources Needed by College Faculty to Facilitate Effective Case-Based Teaching in the Life Sciences

Contribuições da ergonomia no desenvolvimento de projetos de formação profissional

CONSENSUS IN DECISION MAKING BY THE DELPHI TECHNIQUE