No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Delphi-based approach to developing expert systems with the cooperation of multiple experts
Abstract
Knowledge acquisition has been a critical bottleneck in building knowledge-based systems. In past decades, several methods and systems have been proposed to cope with this problem. Most of these methods and systems were proposed to deal with the acquisition of domain knowledge from single expert. However, as multiple experts may have different experiences and knowledge on the same application domain, it is necessary to elicit and integrate knowledge from multiple experts in building an effective expert system. Moreover, the recent literature has depicted that “time” is an important parameter that might significantly affect the accuracy of inference results of an expert system; therefore, while discussing the elicitation of domain expertise from multiple experts, it becomes an challenging and important issue to take the “time” factor into consideration. To cope with these problems, in this study, we propose a Delphi-based approach to eliciting knowledge from multiple experts. An application on the diagnosis of Severe Acute Respiratory Syndrome has depicted the superiority of the novel approach.
... Delphi method is a systematic and interactive technique which involves the collection of knowledge from a panel of experts through a series of questionnaires interspersed with controlled feedback [11] [12]. It was developed by Dalkey and Helmer in the 1950s. ...
... It consists of at least two rounds of data collection. The cycle of data collection and feedback will be terminated when a certain level of agreement has been reached or expert opinion has been established [13,11]. ...
... There are seventeen sustainability components that meet these 3 combinations. They include (1) life cycle cost analysis, (2) road safety planning, (3) long life pavement design, (4) environmental management system, (5) waterpermeable pavement, (6) reuse and recycle material, (7) implementing a quality management system, (8) conducting environmental training, (9) reducing the use of fossil fuels, (10) Reducing emissions of construction equipment, (11) using recycled materials, (12) using materials near the project site, (13) maintaining drainage channels, (14) constructing retaining wall to prevent soil sliding, (15) pavement management system, (16) road infrastructure maintenance, and (17) With regard to the relevance of components, the experts were agreed that road safety planning, long-life pavement design, environmental management system, water-permeable pavement, and use and recycle material, should be applied for indicators for sustainability at the design phase of local road. Road safety is a paramount indicator. ...
To ensure our sustainable future, the whole lifecycle of our current and newly built infrastructure must satisfy sustainable standard, focusing Sustainable Development Goals (SDG) 13. It should meet functional requirements of reducing negative impacts to the environment while at the same time support economic growth and societal development. Due to its vast network, local roads may contribute to sustainable living assuming that it is designed, constructed, operated, and maintained in such a way that satisfy sustainable standard. Despite the fact that sustainable road rating systems have been developed in many countries, there is no agreement on the components for defining and measuring local road sustainability, especially in developing countries. In facilitating the development of the future local road sustainability index, this paper attempts to identify the components by finding insights and agreement from experts. In this case Delphi technique was employed. Seventeen components were specified consists of eleven environmental components, three economic components, and two social components. This means that the achievement of local roads sustainability integrates the three aspects; environmental stewardship, economic growth, and social development.
... The FDM analysis was conducted in two rounds: validation of the main constructs as well as validation of items in each of the constructs. Threshold values less than 0.2 [13] and expert agreement percentages greater than 75% are found for all the main constructs against the FDM criteria [14]. Accordingly, these results suggest that the constructs are relevant and appropriate to be part of the MoGHS. ...
... We also look at the consensus by calculating the overall percentage of the expert agreement on construct and item levels. If such a percentage of agreement is 75% or above, there is expert consensus [14]. Assuming expert agreement at a certain percentage and assuming the condition of a certain threshold, the construct and item set are 'accepted' or 'rejected'. ...
... Of which 44 items were validated using the consensus criteria for a threshold value (d) ≤ 0.2 and an agreement percentage ≥ 75%. The findings are consistent with the previous studies in stressing that FDM is a perfect tool to deal with uncertainty and improve accuracy in module development [13,14]. This is significant since three items were rejected (reading lesson notes, projectbased strategies, materialbased strategies). ...
Background and study aim. The purpose of this study is to expand the Healthy Lifestyle Module (MoGHS) using the Fuzzy Delphi Method (FDM) for use as a co-curricular study related to the matriculation program. Expert consensus was obtained on the module’s constructs and items, including physical fitness, stress management, and healthy nutrition. Material and methods. The study employed the Fuzzy Delphi Method, a systematic approach for achieving expert consensus on the validity and reliability of the Healthy Lifestyle Module. Results. Based on the FDM analysis, six main constructs were validated at the threshold value of less than 0.2 and an agreement percentage above 75%. At the same time, 44 out of 47 proposed items were accepted, thus meeting educational requirements. Conclusions. The module is holistically tailored to go beyond traditional health education, fostering soft skills such as discipline, teamwork, and time management. This module serves as a comprehensive tool for promoting healthy lifestyle practices and fostering soft skills among students, contributing to their holistic development in co-curricular education. Its adaptability underscores the module’s potential as a valuable resource for promoting healthy lifestyle practices in diverse populations.
... This systematic approach involved structured interviews across three rounds, where experts provided anonymous feedback to achieve consensus on the topic. In each round, feedback was aggregated and shared to allow experts to refine their responses, continuing iteratively until a consensus was reached (Chu and Hwang 2008;Kermanshachi et al. 2020). The KAMET rules ensured that the knowledge gathered was relevant and valuable, guiding the Delphi process from round t to round t þ 1 based on specific criteria. ...
... (3). Source: Chu and Hwang (2008). values are considered acceptable as follows: According to Hair et al. (2019), scales with a coefficient from 0.8 to near 1 are considered perfect, those with a coefficient from 0.7 to near 0.8 are considered well-used, and those with a coefficient of 0.6 or higher are considered eligible. ...
This study evaluates the attributes affecting contractors’ sustainable productivity management in developing countries, utilizing the Balanced Scorecard (BSC) framework. By synthesizing existing knowledge from previous research, the study developed a portfolio of twenty-nine attributes organized into four groups aligned with the BSC perspectives. The feasibility of these attributes was tested through semi-structured interviews with fifteen experts over three rounds using the Delphi technique based on the Knowledge Acquisition for Multiple Experts with Time Scales (KAMET) rules. Subsequently, a survey with 325 respondents from the construction sector in Vietnam was conducted. Reliability was assessed using Cronbach’s Alpha, and Exploratory Factor Analysis was performed to build the model. Multiple regression analysis tested the research hypotheses. The study identified twenty-nine key attributes distributed across financial, customer, internal processes, and learning-growth perspectives, finding significant correlations with sustainable productivity management. It offers practical recommendations for stakeholders, bridging theory and practice and emphasizing balanced economic performance, environmental stewardship, and social responsibility. This study provides actionable insights through the BSC framework to enhance contractor productivity sustainably, addressing a critical need for guidance in developing countries’ construction sectors and fostering long-term industry growth.
... After receiving the experts' responses, the research team synthesized and analyzed the results based on the Knowledge Acquisition for Multiple Experts with Time scales (KAMET) principle (see Table 2). The KAMET principle evaluates the importance of each factor (qi) at different stages by considering a combination of statistical values, including the mean value (Mqi), the quartile deviation (Qqi), and the percentage of experts who changed their assessment Rating Variant (Vqi) [39,44]. ...
... The KAMET rule analyzes assessments from experts using the Delphi method[44] ...
The application of information and communication technology in general and educational scientific research is developing rapidly, especially in Vietnam, where integrating Information and Communication Technology (ICT) in education is gaining momentum as part of the national digital transformation agenda. However, there remains a gap in understanding how pedagogical students, who are future educators, apply ICT in their scientific research activities. This study investigates the factors influencing the competency of pedagogical students to apply ICT in Educational Scientific Research (ESR), grounded in the theoretical frameworks of the Technology Acceptance Model, Social Learning Theory, and Theory of Planned Behavior. The authors identify potential factors by synthesizing existing research about factors affecting students’ scientific research competencies and ICT competencies. This study uses the Delphi method because of the complexity of the factors and the need to gather expert consensus in refining and validating these elements, which was achieved through three rounds of expert consultation. The results revealed 18 key factors significantly impacting students’ competence in effectively integrating ICT into their educational research activities. These factors span various domains, including institutional policies, financial support, ICT infrastructure, curriculum design, and individual student characteristics such as ICT skills, foreign language proficiency, and self-directed learning capabilities. Among these, institutional policies supporting ICT integration, financial support, ICT infrastructure, students’ ICT skills, and self-directed learning capabilities were the most significant and impactful. The findings offer valuable insights for educational institutions seeking to improve students’ research competencies by implementing targeted interventions that harness the potential of ICT in scientific research.
... b. Peratus kesepakatan pakar mestilah melebihi atau sama dengan 75% (Chu & Hwang, 2008;Murry & Hammons, 1995) c. Nilai defuzzification (alpha cut) hendaklah melebihi atau sama 0.5 (Tang & Wu, 2010;Bodjanova, 2006). ...
... Dalam FDM, sesuatu data hendaklah memenuhi tiga syarat utama FDM iaitu (i) nilai threshold (d) tidak melebihi atau sama dengan 0.2, (Cheng & Lin, 2002) (ii) peratus kesepakatan pakar mestilah melebihi atau sama dengan 75% (Chu & Hwang, 2008;Murry & Hammons, 1995) dan (ii) nilai defuzzification (alpha cut) hendaklah melebihi atau sama 0.5 (Tang & Wu, 2010;Bodjanova, 2006).Sekiranya sesuatu data tidak memenuhi salah satu daripada ketiga-tiga kriteria ini, maka ia ditolak. ...
Kajian ini bertujuan untuk mendapatkan konsensus pakar terhadap kandungan yang diperlukan bagi mereka bentuk pembangunan Model Bahasa Arab Untuk Perundangan Islam Melalui Integrasi Ilmu Naqli Dan Aqli. Kajian ini menggunakan kaedah kuantitatif iaitu kaedah Fuzzy Delphi dengan soal selidik tujuh skala likert. Pengkaji mendapatkan konsensus 1 orang pakar dari bidang bahasa Arab dan 2 orang pakar dari bidang Syariah dan Undang- Undang. Borang selidik mengandungi tiga elemen; (i) Kesepakatan pakar secara umum berkenaan tahap pengetahuan bahasa Arab yang diperlukan oleh pelajar Jurusan Syariah & Undang-Undang, (ii) Kesepakatan pakar mengenai keperluan penguasaan pelajar terhadap tajuk-tajuk Nahu dan Fungsi Dalalah Nahwiyyah dalam pembinaan Hukum, (iii) Kesepakatan pakar mengenai keperluan penguasaan pelajar terhadap tajuk-tajuk Balaghah dan Fungsi Dalalah Balaghiyyah dalam pembinaan Hukum. Dapatan kajian berdasarkan konsensus pakar terhadap hampir kesemua item adalah diterima dengan dan melepasi 3 syarat kriteria Fuzzy Delphi iaitu nilai threshold (d) tidak melebihi atau sama dengan 0.2, peratus kesepakatan pakar melebihi atau sama dengan 75% dan nilai Defuzzication (alpha cut) melebihi atau sama dengan 0.5. Namun terdapat 2 item ditolak kerana mencatat nilai Defuzzication kurang dari 0.5. Kajian ini diharap dapat memberi idea baharu kepada institusi pendidikan dalam melahirkan pelajar yang menguasai bahasa Arab dan fungsinya dalam memahami hukum syarak.
... The knowledge acquisition bottleneck in expert systems was a significant challenge, as the process was very labor intensive -it required manual input and curation provided by human experts. Building, updating, and expanding the knowledge base was a time-consuming and costly process, making it difficult to keep the systems up-to-date and relevant (Chu & Hwang, 2008;Hubert & Dreyfus, 1997). Performance was also a challenge in early expert systems, often attributed to the use of tools that interpreted code expressions without prior compilation. ...
This paper introduces Generative Artificial Experts (GAEs) - a concept of a new type of generative AI agents designed for human-AI collaboration in knowledge work. GAEs have specialized domain expertise, perform tasks within bounded autonomy, include a synthetic persona and possess multimodal generative AI capabilities, among other features. We provide a definition of GAEs which includes seven defining traits, offering a taxonomy which sets them apart from other generative AI systems. We use literature-review based conceptual analysis with abductive reasoning to propose the new concept that addresses identified limitations in existing systems. The paper explores the emergence of GAEs as a leap from expert systems. We name two enablers for GAEs - ongoing development of a research field of human-AI collaboration and growing capabilities of generative artificial intelligence systems. We discuss existing generative AI agents, noting that GAEs as such do not exist yet, but are starting to emerge. Due conceptual nature of this paper we do not explore the technical aspects of GAEs development. Instead, we use illustrative examples to present possible applications of GAEs and their potential role in the future of knowledge work. This article appears in the AI & Society track.
... The agreement of the expert group is referred to as the consensus group. A consensus of 75% would be required to show an agreement among the experts [24]. If the consensus is below 75%, the second round should be implemented to ensure the experts' consensus is at least 75%. ...
Mobile learning, also known as m-learning, has been popular recently due to the portability, ease of use, and affordability of mobile devices. M-learning can take place anywhere and at any time. Furthermore, adult learners face challenges due to the increasing use of technology and the ageing population. Current design guidelines for smartphones for older adults can be challenging to implement and interpret. Little research has been done on mobile learning elements before application development. Different frameworks of the mobile learning model apply specific principles and endorse the theory of demographics and other related variables. This study addresses the gap in pre-application research on mobile learning elements, focusing on adult learners. This study uses the Fuzzy Delphi Method (FDM) with ten experts and the result from the analysis threshold score was (d) ≤ 0.2, the percentage consensus of the expert was more than 75%, and the average fuzzy score (A) for all the elements is 0.5 and above. Results indicate that seven proposed elements (usability, navigation, touch gestures, andragogy, scaffolding, content, and layout) are important for adult learners. It is suggested that these FDM-identified components be included in a mobile learning design model to improve the educational experience for this particular group of learners. This study provides insightful information on particular elements that are necessary to successfully create mobile learning experiences for adult learners.
... To get experts' consensus on every item, the threshold value (d) should not be more than 0.2 69 , the percentage of experts' consensus needs to be more than 75% 70 whereas, the defuzzification score (A) for every item must be more than the α-cut value, which is, 0.5 69 . The defuzzification process also determines the position or rank of every item. ...
An industrial heritage site, whether functioning or abandoned, has impact both on the industrial workers and on the population living in an industrial neighborhood, comprising the urban industrial community. However, limited research addresses the role of public perception in heritage management, often excluding the perspective of the urban community from discussions of urban well-being. This study examines the riverine industrial landscape of West Bengal, India, exploring the alignment and contradiction between expert and community views on industrial heritage value. Utilizing a mixed-methods approach of content analysis, Fuzzy Delphi survey, and Principal Component Analysis, this study reduces the ambiguity of subjective responses, identifying 19 consensus-based indicators categorized into six heritage values. Comparative analysis of the rankings reveals significant disparities in 12 of these indicators, underscoring a contrast in how experts and industrial community prioritize value indicators of industrial heritage. Indicators with notable disparities were further analyzed to address their implications for 5 urban well-being dimensions, including employment, living environment, cultural diversity, citizen services, and education. As urban industrial areas navigate the dual pressures of industrial heritage and modernization, understanding these perceptual differences is essential for inclusive heritage management. The findings also advance methodological approaches for assessing urban heritage perceptions.
... Percentage of expert agreement ≥ 75% (equal to or greater than 75%) Accepted (Chu and Hwang, 2008;Murry and Hammons, 1995) 3 ...
Creating a high-quality learning environment where students can solve real-world problems and be receptive is essential for fostering students’ innovation competencies. Using appropriate pedagogical strategies and classroom activities is a crucial aspect of Malaysian education. This article uses the Fuzzy Delphi Method (FDM) to design chemistry classroom teaching strategies based on the design thinking paradigm. This research involves 12 experts in purposive sampling to form a diverse panel encompassing expertise in Chemistry Education, Curriculum, Module Development, Research, and Innovation. Using the Fuzzy Delphi method (FDM), the data were analyzed. Four elements for exploratory constructs, two elements for construct interpretation, four elements for ideation, two elements for execution, and three elements for construct evolution met the FDM requirements, according to the findings. Its threshold value is less than 0.2, the expert consensus is less than 75%, and the average score of the fuzzy number is over 0.5. Encouraging design thinking in chemistry classes and thereby enhancing students’ innovation skills, this research unquestionably induces a paradigm shift in teaching practice.
... 2,(Cheng & Lin, 2002), then all experts have reached a consensus. Subsequently(2) The percentage of expert consensus for the value of the percentage of expert consensus is equal to or above ≥ 75%(Chu & Hwang, 2008; Murry & Hammons, 1995) shows that the consensus of the expert panel is adhered to. And finally (3) Defuzzification Process, which is a fuzzy evaluation process to determine the value of the Fuzzy score and the priority of the position of each element in each main component is based on the value of the fuzzy score, α-cut. ...
The integration of digital pedagogical approaches has emerged as a fundamental imperative in 21st-century educational transformation, aligning with both the Sustainable Development Goals (SDGs) and Digital Education Policy frameworks, particularly within the domain of Visual Arts Education (VAE) at the primary education level. This research endeavors to examine the systematic development of digitalized pedagogical frameworks for Visual Arts Education in primary institutions, utilizing the Fuzzy Delphi methodology to enhance instructional efficacy and learning outcomes. The implementation of the Fuzzy Delphi method facilitates the synthesis of expert consensus in establishing critical components for pedagogical digitalization that address both educator and learner requirements. The empirical findings demonstrate the efficacy of the Fuzzy Delphi approach in ensuring the pedagogical digitalization framework maintains contextual relevance and appropriateness within the primary school teaching environment.
... This smaller size allows for more focused discussions and facilitates consensusbuilding among participants. 74 On the other hand, if the participants in the Delphi panel are from a similar field (15 to 30) or from different fields (5 to 10) for each specialised group, 75 this study will include primary groups of policymakers and specialists from the fields of medicine and nursing, comprising a minimum of 15 participants. ...
Introduction
The primary caregivers of patients with cancer, especially in advanced stages, experience significant physical and emotional strain and require comprehensive support. Respite care has been acknowledged as an effective strategy to aid primary caregivers, and different models have been established in Western nations. However, implementing these models poses challenges due to notable cultural, social and healthcare system variances between Western and non-Western regions like Iran and other Middle Eastern countries. It is crucial to develop a model tailored to the country’s specific needs, culture and resources to systematically support caregivers and ensure care continuity. Hence, this study aims to develop and validate a respite care model for adult patients with cancer in Iran.
Methods and analysis
This study will use a multimethod approach across three stages. In the first phase, a scoping review will be conducted to identify, describe, conceptualise and implement the respite care system based on available documents and evidence. Then, a directed content analysis will be carried out to clarify care requirements and system establishment needs from stakeholders’ perspectives. Data from the scoping review and directed content analysis will be integrated to identify the crucial components of the model using the Delphi method in the second phase. In the third phase, another Delphi study will be conducted to assess the importance, scientific credibility and feasibility of the model, with the aim of finalising it.
Ethics and dissemination
This research project has received the code of ethics (IR.TBZMED.REC.1402.720) from Ethics Committee of Tabriz University of Medical Sciences. The findings of this study will be published in reputable journals and disseminated in accessible formats through various media channels.
... A review of 34 Delphi studies since 2019 found stability was almost never used as a closing criterion and was often not reported (Nasa et al., 2021). In practice, consensus and not stability is often used as the criterion for stopping further inquiry (Chu and Hwang, 2008;Aziz et al., 2015;Chen et al., 2017); this study adopts the same approach. ...
A comprehensive set of regionally relevant indicators and metrics is crucial for tracking progress in transforming food systems to meet the UN Sustainable Development Goals (SDGs). Household food security, foundational to sustainable, equitable, and resilient food systems, aligns with SDG 2 (Zero Hunger). Policymakers require accurate data to guide decisions, yet a major challenge is developing scientifically sound, participatory, and reproducible approaches for sub-national food system indicators and metrics. This study addresses this need by using the Delphi research method to create a multi-indicator system of both stand-alone and complementary food security metrics specific to Hawaiʻi. Engaging 24 food security experts in Hawaiʻi, with a 58% response rate in the second round, consensus was achieved on 55 of 71 metrics (78%). The U.S. Household Food Security Survey Module (HFSSM) received the highest rating among stand-alone metrics (3.5, SD = 0.8). Complementary metrics were organized within a framework encompassing the four dimensions of food security: availability, access, utilization, and stability. Access metrics received top ratings, including the ‘rate of households below the ALICE threshold’ (4.4, SD = 0.7). ALICE stands for Asset Limited, Income Constrained, and Employed, and refers to households that earn enough to be ineligible for many public assistance programs but not enough to afford basic necessities. Results highlight the need to clarify terminology, address food (in)security misconceptions, develop new metrics for data gaps, and prioritize initiatives like the Supplementary Nutrition Assistance Program (SNAP) that address Hawaiʻi’s high living costs and enhance food access. The participatory, multidimensional metrics model presented here is adaptable to other regions, extending its impact beyond Hawaiʻi.
... [40]. For the second requirement, the percentage value achieving group consensus must be greater than or equal to 75% [41] . ...
Innovative and creative techniques and materials for teaching and learning are required to stay up with the advancements of globalization. The purpose of this study is to reach expert consensus on the precise and relevant components that should be used while designing and developing an acid-base Et-PBL module to improve problem-solving skills.This study's design is quantitative and employs the Fuzzy Delphi method.Thirty-three items made up the questionnaire instrument used to gather research data. Twelve experts from 10 different colleges who specialize in chemistry education and Islamic education participated in the study.Based on the expert consensus value over 75%, the fuzzy score (A) α - cut value = 0.5, and the threshold value (d) 0.2, the data analysis results indicated that the experts accepted all these items. Consequently, it demonstrates that these components have been agreed upon by experts and are required for the design and development of the acid-based Et-PBL module
... The Delphi expert method is a systematic consultation process where experts accumulate information and express their knowledge through rounds of participation to achieve consensus on the issues under discussion. A novel approach is the Knowledge Acquisition for Multiple Experts with Time scales (KAMET) proposed by Hui-Chun Chu, Gwo-Jen Hwang (2007). When all the questionnaires from the Delphi rounds are collected, the ranking analysis rules are applied using the Delphi approach with KAMET by Hui-Chun Chu, Gwo-Jen Hwang (2007, pp. ...
This paper aims to introduce the process of using the Delphi method to construct the University Brand Management Index Table - a crucial tool in evaluating and managing university brand development. The Delphi method facilitates the collection of opinions from 38 experts with knowledge and experience in brand management and education from 7 private universities in Hanoi, Vietnam. The panel includes 4 representatives from employers, 4 representatives from the media, and 1 cooperation partner. The consensus of this expert panel, following the KAMET rules, combined with careful design of sequential questionnaires, enhances the reliability and quality of the final University Brand Management Index Table. The research results, after conducting 3 Delphi rounds, propose a table of brand management indices for universities, comprising 61 explanatory elements across 14 key indices. These indices include: University branding identity, University Brand Essence and Characteristics, Brand association, Competitive capabilities vividly demonstrate the university brand's characteristics, Brand Perceived Quality, Learner's Willingness to Experience Learning, Satisfaction, Brand loyalty, Brand Assets from Alumni, Website metrics, Social media marketing metrics, Communication Metrics, The Value Synergy of Self with University Brand Equity, University Brand Performance reflects the outcomes and achievements. The combination of a rigorous process and multidimensional contributions from experts enhances the integrity and effectiveness of this method, bringing significant benefits in affirming the applicability of the Delphi method in constructing the University Brand Management Index Table as appropriate for brand management in higher education institutions.
... First, for the Triangular Fuzzy Number (TFN), the threshold (d) must be ≤ 0.2 to signify expert agreement 32,33 . Traditionally, if the expert group agreement exceeds 75%, it is considered accepted 28,34 . The second condition involves the Defuzzification Process, where the fuzzy (A) score value is determined based on the α-cut value of 0.5. ...
Cardiovascular disease (CVD) is a major global cause of premature mortality. While multiple studies propose CVD mortality prediction models based on regression frameworks, incorporating causal understanding through causal inference approaches can enhance accuracy. This paper demonstrates a methodology combining evidence synthesis and expert knowledge to construct a causal model for premature CVD mortality using Directed Acyclic Graphs (DAGs). The process involves three phases: (1) initial DAG development based on the Evidence Synthesis for Constructing Directed Acyclic Graphs (ESC-DAGs) framework, (2) validation and consensus-building with 12 experts using the Fuzzy Delphi method (FDM), and (3) application to data analysis using population-based survey data linked with death records. Expert input refined the initial DAG model, achieving consensus on 45 causal paths. The revised model guided selection of confounding variables for adjustment. For example, to estimate the total effect of diabetes on premature CVD mortality, the suggested adjustment set included age, dietary pattern, genetic/family history, sex hormones, and physical activity. Testing different DAG models showed agreement between expert ratings and data accuracy from regression models. This systematic approach contributes to DAG methodology, offering a transparent process for constructing causal pathways for premature CVD mortality.
... If the overall agreement for the entire construct is 75% or more, it indicates a significant consensus among specialists. The 75% threshold for individual items ensures that each component achieves at least 75% agreement (Chu and Hwang, 2008;Murry Jr and Hammons, 1995). ...
The well-being of a person includes different parts of life, such as mental and emotional health, physical health, and social relationships. Emotional well-being is especially important and refers to how people feel psychologically, including finding meaning in life, feeling positive emotions, and being satisfied with life. For students with special educational needs, focusing on emotional well-being is especially important after the COVID-19 pandemic to help them grow and succeed. This study aimed to create a plan for supporting the emotional well-being of these students in special education after the pandemic. The study had two main goals: (i) to find out if experts agree on the key components needed to create a model for emotional well-being for students with special needs after COVID-19, and (ii) to identify if there is agreement among experts on the important factors needed to build such a model. The Fuzzy Delphi Method (FDM) was used, gathering input from nine experts through questionnaires. The data were analyzed using triangular fuzzy numbers and the defuzzification process to rank each variable. The study found more than 75% agreement among experts, threshold values (d) below 0.2, and α-cut values above 0.5. The FDM analysis identified six main elements for an emotional well-being model for students with special educational needs after COVID-19. These results offer important guidance for developing emotional well-being models for these students, providing key support for both teachers and students in managing emotions in the post-pandemic period.
... In this phase, the information obtained from the first phase (qualitative study and literature review) will be combined and prioritized for feasibility through two rounds of surveys with the panel of experts. Since the validity of Delphi relies not on the number of participants but on the scientific credibility of the experts involved, Delphi study participants are usually fewer than 50, most commonly between 12 and 20 [41]. In the present study, 20 people will be selected as expert panel. ...
Background
Unintended pregnancy, a pregnancy that have been either unwanted or mistimed, is a serious public health issue. The present study aims to design and implement interventions to improve unplanned pregnancy experiences.
Methods/design
This exploratory sequential mixed method study will be conducted in three phases: qualitative, intermediate, and quantitative. The qualitative phase will use qualitative conventional content analysis with in-depth and semi structured individual interviews to explain and define the components and elements of pregnancy experiences of unplanned pregnancies, which include mothers with unplanned pregnancies, their spouses, and prenatal care providers, who will be selected purposefully. Additionally, in the initial phase, the study will employ literature reviews alongside qualitative findings to elucidate the components and elements of pregnancy experiences and their improving interventions. In the second phase, appropriate interventions (prioritized and feasible) will be determined through an expert panel using the Delphi technique. In the third phase, the intervention program agreed upon in the previous phase will be implemented in the form of a randomized controlled clinical trial.
Discussion
The implementation of the interventions could be beneficial in changing attitudes and achieving positive experiences in unplanned pregnancies. It is anticipated that the design and implementation of the intervention program aimed at improving the experiences of unplanned pregnancies will be effective in minimizing adverse maternal and neonatal outcomes.
Trial registration Iranian Registry of Clinical Trials (IRCT): (IRCT20170506033834N12/Date of registration: 2024‑02‑12)
... The Fuzzy Delphi technique also involves the process of determining whether the expert group's agreement is greater than or equal to 75% for the entire construct or for each item. Each item is assumed to reach expert consensus if the percentage of expert agreement for the item is equal to or greater than 75% (Chu & Hwang, 2008, Murry Jr & Hammons, 1995. ...
The aim of this study is to obtain expert agreement on the requirements of the main components, the elements within the main components, and the order of priority of the main components for each construct in the design and development of the Comdyslexic model of inclusive education for dyslexic students. In this study, the Fuzzy Delphi method, using seven Likert scales, was employed to gather 19 experts in the field of special education as respondents. The research questionnaire comprises 42 items related to the design and development of the Comdyslexic model, which is divided into three components: the level of teacher knowledge, the teacher's skills and experience, and the level of teacher understanding. Data were analyzed using fuzzy triangle numbering (Triangular Fuzzy Numbers), and the ranking of each variable was determined using the Defuzzification Process. The findings of the study indicate that expert feedback and consensus regarding the elements in the main components of the Comdyslexic model are at a good level. The overall results of the study, through expert consensus, are over 75%, with a threshold value (d) < 0.2 and α-cut over 0.5. Overall, this study is expected to provide benefits and guidance to special education teachers. The presence of the main elements and components in the model is anticipated to have a positive impact on special education teachers in terms of the level of mastery, knowledge, and awareness of the Inclusive Education Program (IEP). The IEP is considered a good long-term strategy to improve the education level of special needs students in our country.
... In the final stage, ranking, and defuzzification (the process to determine the weights), the researchers decided or agreed on a group of experts known as a consensus group. To ensure that there was at least 75% consensus among the experts, as recommended by 52 if the consensus group was less than 75%, the researchers repeated the procedure. The researchers then transformed the final agreement of the answer statement for each question, as shown in Table 2. ...
This study aims to investigate the key features necessary for the development of a mobile application for human resource management (HRM). It addresses the shortage of empirical studies on digital HRM in the Middle East and emphasizes the increasing importance of HRM mobile applications in the region. The research utilizes qualitative research methods, including the Delphi method, in-depth interviews, literature review, historical analysis, and critical discussion. A qualitative research strategy is employed, analyzing data from 43 HR experts and mobile app development team members from international and local businesses in the UAE and the Kingdom of Bahrain. NVivo software is used for data analysis, encompassing over 120 transcriptions, notes, and academic papers. The study identifies and explores the logical and practical features required for an HRM mobile app. It uncovers challenges related to the Delphi survey experiences, such as managing heterogeneous information and ensuring compatibility with iOS and Android platforms. This research fills a gap in the literature by investigating the digital HRM landscape and HRM mobile applications in the Middle East. It provides valuable insights from HR and technology experts, contributing to the advancement of digital HRM, mobile applications, and the application of the Delphi method methodology. The study highlights implications for industries to enhance HR operations and productivity through the adoption of mobile HR applications. Theoretical contributions include new insights into the digital HRM literature, specifically regarding the identification and discussion of essential features in HRM mobile applications. Practical implications involve the benefits of transitioning from traditional HRM to digitalized HRM in the professional era, as identified through the analyzed features of the HR app. The study offers practical insights for the development and implementation of mobile HR apps. Organizations can leverage these insights to create effective mobile HR apps that enhance HR operations and employee productivity.
... The first condition is the threshold value (d), while the second condition is the expert agreement percentage for each of the items. The threshold value (d) for each item measured should be less than or equal to 0.2 (Chen, 2000;Cheng & Lin, 2002), and the expert agreement percentage must be greater than or equal to 75% (Chu & Hwang, 2008). ...
Pro-environmental behaviour is imperative to promote sustainable management and consumption of energy in the fight against climate change. The Theory of Planned Behaviour (TPB) has been widely used to explain and predict behaviour in a multitude of behavioural domains including pro-environmental behaviour. However, the TPB does not prioritise the influence of knowledge and habits on pro-environmental behaviour. Past studies also confirmed the influence of cultural factors in predicting environmental intent and behaviour, which were not covered by TPB. Considering that Malaysia is a multicultural country, on top of the environmental education factors, cognitive competencies, and affective factors, the influence of values that contribute to the pro-environmental intention behaviour on energy conservation has also to be considered. Hence, this study employed Fuzzy Delphi Method to determine the indicators explaining the effects of educational and psychological factors on pro-environmental behaviour. It was hypot
... In the selection of experts, satisfactory results can be obtained from small panels of 10-15 individuals from a homogenous group of experts (7). ...
Background: Emergency departments (EDs) have had to cope with various pandemics, such as HIN1, Ebola and the currently ongoing COVID-19. This study aimed to determine the elements of the new norm that has introduced changes into healthcare systems, particularly EDs, during the COVID-19 pandemic and to obtain consensus from the experts, the Emergency physicians in EDs across Malaysia. No previous study has been conducted on this topic. Methods: This study used the fuzzy Delphi method (FDM) to acquire expert consensus. There were two phases in this study. The first involved extracting the elements to be assessed by the selected experts from a literature review. Three major domains were considered: infrastructure, governance and human resources. A total of 35 items were identified and divided into the three domains. In the second phase, the selected items were sent to a group of 15 ED physicians, who were asked to rate the items on a Likert-type scale. The data were then analysed using FDM. Results: A total of 35 items were identified as possible new norms from a literature search for the three domains (governance, infrastructure and human resources). The first step of the FDM analysis showed that 9 out of the 35 items did not fulfil the initial requirement of the FDM, since the threshold value (d) must be lower than 0.2 (d </ 0.2). In meeting the second condition for the FDM, 25 out of the 35 items had an expert consensus of > 75%. Regarding the third requirement for FDM, only 1 out of the 35 items did not meet the criteria of an average fuzzy number (A value) of > 0.5. Finally, 25 items fulfilled all three requirements of FDM, so these were retained and the remaining 10 items were discarded. Conclusion: The FDM used in this study had identified 25 items achieved the required level of agreement by the chosen experts. The results of this study can be used to guide EDs in Malaysia to utilise the new norms items in mitigating major outbreak affecting the ED services.
... One of the key challenges was the knowledge acquisition bottleneck, which refers to the difficulty in eliciting and incorporating domain expertise into the system. The process of acquiring and representing expert knowledge can be time-consuming and complex, leading to delays in system development and deployment (Chu & Hwang, 2008). These systems were rule-based, relying heavily on predefined knowledge bases, which made them rigid and unable to learn from new data. ...
This thesis explores the innovative application of Large Language Models (LLMs) in video game quest design, with a specific focus on Assassin's Creed Odyssey. The study aims to develop a methodology for fine-tuning LLMs using synthetic data derived
from structured quest information and to evaluate their effectiveness in generating contextually appropriate quests compared to non-fine-tuned models. Leveraging community-driven content from the Assassin's Creed Fandom Wiki, the research constructs a robust dataset to fine-tune models like ChatGPT 3.5 Turbo and GPT-4o-mini. The findings reveal that while fine-tuning offers potential for specialized content
generation, however base models often outperform fine-tuned variants across multiple evaluation criteria. The research highlights the importance of structured data and nuanced fine-tuning strategies in achieving high-quality, AI-generated content. This work contributes to the growing body of knowledge on the intersection of AI and game design, offering insights into the future of narrative generation and the role of AI in
creative industries.
... The determination of the acceptability of constructs and their respective items necessitated the fulfillment of three prerequisites. These prerequisites encompassed: (i) the adherence to a threshold value (d-construct) with a stipulation of d-construct ≤0.2 [114]; (ii) the attainment of a minimum 75 % consensus among experts regarding the evaluated items [115]; and (iii) the assignment of ranking to each item based on the fuzzy score (A max ) provided that its value is greater than or equal to α-cut value (0.5) [108]. The threshold value, d-construct, serves as a pivotal criterion for the selection of specific constructs based on expert consensus for each construct. ...
This study delves into the critical barriers hindering sustainable manufacturing practices in medium-sized ready-made garments manufacturing enterprises. Initially, barriers were identified through a literature review and expert opinions. Then, the fuzzy Delphi Method and expert consensus were used to prioritize the critical barriers. Finally, based on the insights from a focus group discussion and the supported literature, strategies to mitigate the identified critical barriers were formulated. In this study, the initial barriers identified encompassed various dimensions. The top three critical barriers were identified as higher prices of sustainable products, limited supplier commitment, and lack of access to information. Strategies to address these critical barriers include adopting collaborative business approaches, incorporating sustainability standards, investing in technology, implementing sustainable manufacturing techniques, and integrating green supply chain management practices. Overall, by focusing on these strategies, medium-sized garment manufacturing enterprises can enhance their sustainability efforts and gain economic, environmental, and social benefits.
... Expert consensus is obtained if the value of d ≤ 0.2, the percentage is ≥75%, and the alpha-cut (DV) > 0.5 (Murry & Hammons, 1995;Cheng & Lin, 2002;Chu & Hwang, 2008). ...
Data-Based Educational Decisions (DDED) is an important part of the learning process where DDED is used to improve the quality of learning. This research purpose to develop Data Driven Educational Decision Implementation Guidelines (DDED) supporting formative tests in IPA learning. DDED is an adaptation of Data Drive Decision Making (DDDM) in learning. DDED is based on Data Drive Decision Making (DDDM), which is a decision-making process that has been widely applied in the world of management. The DDED directs teachers to be adaptive and flexible in implementing various methods, models, teaching materials and learning media based on student access. The development design used is 4D which consists of defining, designing, developing and deploying. The content of the guidelines has been agreed through expert consensus through the fuzzy delphi technique. Fuzzy Delphi Method to create a broad and detailed understanding, especially in education. The consensus indicated that formative tests should include relevant questions and quizzes, comprising both multiple-choice and essay formats, aligning with the principles of DDED. The resulting guideline tests the level of validity and user response. Users respond well to the guidelines developed.
... Step 6: Determination of expert consensus percentage The second condition to determine the percentage value of expert consensus is that the overall group consensus must exceed 75% for each item. If not, a second round needs to be conducted (Chu & Hwang, 2008;Murray et al., 1985). ...
... Langkah 4: Jika jarak di antara purata dengan data penilaian pakar adalah kurang daripada nilai threshold 0.2, maka semua pakar-pakar dianggap telah mencapai konsensus. Jika peratusan mencapai konsensus kumpulan adalah lebih daripada 75% (Chu & Hwang, 2008), seterusnya pergi ke Langkah 5. ...
Abstrak Amalan kepimpinan lestari mampu membina satu persekitaran pendidikan yang menggalakkan percambahan dan gabungan idea dan amalan melalui proses perkongsian kuasa dan proses pembelajaran secara kekal dalam sekolah. Justeru, kajian ini bertujuan untuk melihat elemen-elemen Kepimpinan tahap lestari yang diperlukan dalam Inventori Amalan Kepimpinan (IAK). Kajian ini menggunakan kaedah Design and Development Research (DDR) Fasa 3, iaitu pengujian Jenis 1. Penyelidikan penilaian modul latihan merupakan satu proses empirikal yang memperlihatkan kepada keberkesanan penggunaan modul latihan untuk menunjukkan bukti dan sokongan empirikal terhadap keberkesanan modul latihan NPQEL 2.0 terhadap bakal pengetua dan guru besar. Teknik pensampelan yang digunakan adalah pensampelan bertujuan (purposive sampling) bagi pemilihan pakar, iaitu seramai 17 orang. Kesemua pakar ini pernah terlibat dengan Program Kelayakan Profesional Pemimpin Pendidikan Kebangsaan (NPQEL). Set soal selidik terdiri daripada 15 item yang dibina khusus bagi domain Lestari. Kaedah analisis data yang digunakan dalam kajian ini melibatkan Teknik Fuzzy Delphi. Dapatan kajian menunjukkan terdapat dua puluh tujuh elemen tahap lestari yang telah disepakati oleh pakar dengan peratus konsensus melebihi 75% dengan nilai ambang (d) ≤0.2. Hasil kajian ini membolehkan dua puluh tujuh item daripada Inventori Amalan Kepimpinan dapat digunakan untuk mengukur tahap lestari seorang pemimpin organisasi. Kata Kunci: Lestari, Inventori Amalan Kepimpinan, Fuzzi Delphi Pendahuluan Program Kelayakan Profesional Pemimpin Pendidikan Kebangsaan atau dikenali dengan NPQEL yang mengambil singkatan daripada perkataan National Professional Qualification for Educational Leaders merupakan satu program latihan kepimpinan pendidikan Kementerian Pendidikan Malaysia (KPM) peringkat nasional. NPQEL bertujuan melatih bakal pemimpin sekolah. Program ini dilaksanakan mulai tahun 1999 sehingga kini di bawah kelolaan Institut Aminuddin Baki (IAB). Mulai 1 Julai 2014, KPM telah meluluskan bahawa sijil NPQEL sebagai syarat mandatori bagi pelantikan Pengetua dan Guru Besar sekolah (KPPM, 2013). Perubahan pendekatan latihan dalam NPQEL versi 2018 atau singkatannya NPQEL 2.0 menekankan kaedah latihan masa hadapan dan berfokuskan kompetensi. Kerangka NPQEL versi 2018 dibangunkan daripada definisi operasional pemimpin berprestasi tinggi. Elemen kompetensi yang diberi fokus adalah pemimpin berwawasan, pemimpin pembelajaran dan pemimpin perubahan yang memberi impak kepada kecemerlangan sekolah dan kemenjadian murid. Kaedah penyampaian NPQEL versi 2018 mengadaptasikan pendekatan 70:20:10, iaitu 70% pembelajaran berasaskan pengalaman, 20% pembelajaran sosial dan 10% pembelajaran formal. Semua aktiviti dalam sesi latihan berteraskan prinsip andragogi dan berpusatkan peserta bertujuan membangunkan penguasaan 10 kompetensi kepimpinan pendidikan. Kompetensi-kompetensi berkenaan adalah berfikiran strategik, membuat keputusan, memimpin perubahan, menyelesaikan masalah, memimpin pembelajaran, membangun kapasiti, membina jaringan dan jalinan, berkomunikasi berkesan, berkepemimpinan dan berkeperibadian tinggi.
... Step 4: As stated in Cheng and Lin (2002), consensus among experts is considered achieved if the value of d is less than 0.2. In the case of a group consisting of m × n experts, if the consensus percentage exceeds 75% (Chu & Hwang, 2008) step 5 can be proceeded with. However, if the consensus percentage falls below this threshold, it is necessary to repeat round two of the FDM. ...
In the contemporary landscape characterized by volatility, uncertainty, complexity, and ambiguity (VUCA), crises can arise, placing
organizations at risk. To prevent such failures, organizations must
have capable leaders who possess the necessary crisis management
competencies to handle such situations. Effective crisis management involves four crucial competencies: improvisation skills, analytical thinking skills, strategic planning skills, and change
management skills. This study purposes to evaluate the essential
leadership competencies required for effective crisis management,
with a specific focus on conceptual competencies among middle
leaders in Malaysia’s Ministry of Education (MOE). Utilizing the
Fuzzy Delphi Method (FDM), this research seeks to obtain expert
consensus on the most critical competencies required for effective
crisis management. An instrument containing a list of conceptual
competencies was presented to a diverse panel of 15 experts,
including those with expertise in crisis management and education.
The four elements tested surpassed the required conditions set in
FDM. The defuzzification process also rearranged all the elements.
In conclusion, this study successfully identified four crisis management competencies that are crucial for middle leaders in educational organizations. The study’s findings have benefits for
theoretical evolution, policy development for organizational education, and practice, especially for middle leaders in educational
organizations towards sustainable education.
... FDM also evaluates whether the overall construct or each item achieves at least 75% consensus from the experts. An item is considered to have reached expert consensus if the percentage is equal to or exceeds 75% (Chu & Hwang, 2008;Murray & Hammons, 1995). ...
The Quran is a vital guide for Muslims, requiring proper recitation with tajwid. However, many converts struggle with accurate Quranic recitation even after years of embracing Islam. This study aims to develop Total Physical Response (TPR)-based teaching activities to enhance Quranic recitation proficiency among converts by integrating physical movements with learning, aiding in recognizing Hijaiyah letters and tajwid rules. Utilizing the Fuzzy Delphi Method (FDM), this study achieved expert consensus on proposed TPR-based activities. Ten experts in technical and vocational education, educational technology, and curriculum development participated, selected through purposive sampling. Data collection involved a two-stage process: a workshop where experts rated each activity on a 5-point Likert scale, followed by FDM analysis, transforming responses into triangular fuzzy numbers. This rigorous evaluation required at least 75% consensus and a threshold value (d) not exceeding 0.2. Findings validated all 25 TPR-based activity elements, demonstrating their suitability for teaching Quranic recitation to converts. TPR’s integration of physical activities significantly enhances memory retention and understanding, making it effective for converts with diverse backgrounds and minimal prior Quranic knowledge. The study recommends developing structured modules, providing expert training, creating interactive learning environments, and implementing continuous assessments. Incorporating Higher Order Thinking Skills (HOTS) and varied teaching methods aims to improve engagement and motivation, ultimately making Quranic learning more effective and enjoyable for converts.
... Fuzzy Delphi was used to analyse the data and determine the value of each component. Using this approach, researchers must have expert consensus and ensure that the percentage value is at least 75% (Chu & Hwang, 2008). The Fuzzy Delphi procedure must be repeated for the second round until it reaches the predetermined percentage if this percentage figure is not obtained. ...
... If the distance between the average and the expert's rating data is less than the threshold value of 0.2, it is considered that all experts have reached an agreement (Cheng & Lin, 2002). Moreover, if the percentage of group consensus among m x n experts exceeds 75% (Chu & Hwang, 2008;Murry Jr & Hammons, 1995), then move to step 6. Otherwise, a second round of the Fuzzy Delphi Method is needed, or the item is discarded. ...
The topic of Arabic grammar learning has detailed debates and requires more focus to be understood. The content of textbooks explained only by teachers causes students to feel bored and unattractive, diminishing their interest in learning Arabic grammar. Utilizing technology is now essential for creating a relaxed and effective learning environment, especially for complex course content such as Arabic grammar. Therefore, this study aims to achieve a consensus of experts on the main components and elements of each component as a basis for developing an interactive infographic module for learning Arabic grammar. The design phase is to obtain expert agreement on these components and elements by using the Fuzzy Delphi Method (FDM) with the use of a seven-point Likert scale. Ten experts from various fields of expertise were involved as respondents. Data obtained were analysed using Triangular Fuzzy Numbering, while ranking was determined through the Defuzzification process. The findings show that experts have reached a consensus with a threshold value of less than 0.2 (d<0.2), a consensus rate exceeding 75 per cent (%) and a fuzzy score value (A) exceeding 0.5. Overall, the main components and elements agreed upon by experts related to interactive infographic modules in learning Arabic grammar can produce a module that can provide fun not only for students but also for teachers because the module is interactive and contains full multimedia elements.
... The main component has obtained an expert agreement of 93%, surpassing the threshold value (d) of 0.9. Chu and Hwang (2008) suggested that items receiving as low as 20% expert agreement should not be considered suitable for research purposes. However, in this context, all items have garnered expert agreement levels exceeding 75%, and their defuzzification values also surpass the α-cut value of 0.5. ...
Incorporating experiential learning methods into classroom activities is
crucial for creating an immersive on-campus experience. Front Office
students require proficiency in communication, practical skills, knowledge
application, and dispositional skills. This study aims to identify the essential
elements necessary to integrate scenarios and resources for designing an
online scenario-based learning module named the e-SBL FOIM-ATG for
front-office courses in a Community College. The research design follows a
quantitative approach, utilizing the Fuzzy Delphi technique. Research data
was collected through a questionnaire administered to fifteen experts
specializing in hospitality (Front Office), educational technology, scenario-
based learning, and curriculum development. Analysis of the data revealed
that the experts reached a consensus on all elements, with consensus values
exceeding 75%, threshold value (d) ≤ 0.2, and fuzzy score (A) ≥ α-cut value
= 0.5. This consensus confirms the importance of these elements in
developing the Front Office Instructional Module – Advise the Guest at the
Community College. This research holds significant implications for
lecturers, students, and the broader education system in Malaysia, promoting
the adoption of scenario-based active learning as a transformative educational
approach.
... The Fuzzy Delphi technique also entailed figuring out if the expert consensus surpassed or was equal to 75 percentage for the overall construct or for each individual item (Abdul Ghani et al., 2021). If the proportion of expert consensus for an item was equal to or more than 75 percent, it was assumed that the item had reached expert consensus (Chu & Hwang, 2008;Murray et al., 1985). ...
Nowadays, one of the most prominent leadership philosophies is digital leadership. However, there are still few tools available to assess digital leadership. Unfortunately, the discussion about expert consensus items is limited, especially involving fuzzy considerations. Therefore, this study will assess the content validity of an instrument called situational judgement test (SJT) by using Fuzzy Delphi Method (FDM). There are 13 panels chosen to serve as the FDM's experts. The item should satisfy three conditions of FDM, namely the threshold value (d), the percentage of expert consensus, and the fuzzy score (Amax). The questionnaire uses a seven-point Likert scale based on appropriateness. There are nine constructs, namely (1) Student Engagement, Learning, and Outcomes; (2) Learning Environment and Spaces; (3) Professional Growth and Learning; (4) Communication; (5) Public relations; (6) Branding; (7) Opportunities; (8) Empowered Professionals; and (9) Learning Catalyst. Each construct has 10 items at the first level. According to the results of the study, only 45 of 90 items are appropriate for inclusion in the SJT to assess teachers' digital leadership roles, i.e., five items for each construct. The findings have important implications to recommend any other empirical ways of assessing SJT items to increase the validity aspect of the items.
... The rule base for RB-DEWES currently contains 33 natural indicators (behavioural observation, astronomical, meteorological), with the capability of adding additional indicators in the future. Each indicator has its corresponding certainty factor (CF), which is a measure of the indicator's relevance to natural occurrences, as determined by the focus group based on years of experience ( Table 6-2) (Chu, Hwang, 2008). Table 6 Table 6-3. ...
Drought is a complex environmental phenomenon that affects millions of people and communities all over the globe and is too elusive to be accurately predicted. This is mostly due to the scalability and variability of the web of environmental parameters that directly/indirectly causes the onset of different categories of drought. Since the dawn of man, efforts have been made to uniquely understand the natural indicators that provide signs of likely environmental events. These indicators/signs in the form of indigenous knowledge system have been used for generations. The intricate complexity of drought has, however, always been a major stumbling block for accurate drought prediction and forecasting systems. Recently, scientists in the field of agriculture and environmental monitoring have been discussing the integration of indigenous knowledge and scientific knowledge for a more accurate environmental forecasting system in order to incorporate diverse environmental information for a reliable drought forecast. Hence, in this research, the core objective is the development of a semantics-based data integration middleware that encompasses and integrates heterogeneous data models of local indigenous knowledge and sensor data towards an accurate drought forecasting system for the study areas. The local indigenous knowledge on drought gathered from the domain experts is transformed into rules to be used for performing deductive inference in conjunction with sensors data for determining the onset of drought through an automated inference generation module of the middleware. The semantic middleware incorporates, inter alia, a distributed architecture that consists of a streaming data processing engine based on Apache Kafka for real-time stream processing; a rule-based reasoning module; an ontology module for semantic representation of the knowledge bases.
... Requirement 1 : using the value of Threshold, d [26], [27]. The Threshold value, d determined by using this formula : ...
This study aims to develop and assess the effectiveness of the HOTS Mathematical Problem Solving Framework (HMPSF) with the Bar Model Strategy on the achievement of Year Five Students. This study is a design and development study involving three phases. The first phase is a needs analysis study administered to 65 National School Mathematics teachers and 102 Year 5 students. The second phase is the design and development phase using the Fuzzy Delphi method involving 15 experts in Mathematics Education. Phase three is the implementation and testing phase of HMPSF with the Bar Model Strategy. In this phase, the study was conducted through a quasi-experimental design of a non-equivalent group of pre-post analysis involving 63 Year Five students from two schools in Kinta Utara district, Perak, Malaysia. The sampling process is done randomly. Data were analyzed through ANOVA and MANOVA. The needs analysis conducted show that 92.2% of teachers and 98.04% of students stated that there is a need to develop the HMPSF. Fuzzy Delphi analysis showed that the three main conditions were met (i) Threshold value, D ≤ 0.2, (ii) Percentage of expert agreements,> 75% and (iii) α-cut value (Fuzzy score), ≥ 0.5. This finding shows that the HMPSF with the Bar Model Strategy has been successfully developed through the agreement of 15 experts. The findings in phase three indicate that there is a significant difference in the mean of the post-group test for student achievement. The findings of the Multivariate Pillai’s Trace test also show that there is a significant effect between the pre and post-test. In conclusion, the HMPSF with the Bar Model Strategy developed is proven can enhance student achievement. Conclusively, the HMPSF with the Bar Model Strategy is suitable as a main guide for teachers and students in strengthening problem-solving skills in the field of Numbers and Mathematical Operations.
... The traditional Delphi technique states that it is acceptable if the expert group agreement exceeds 75%. 53 On the other hand, the Defuzzification Process is the determination of the fuzzy (A) score value based on the α-cut value of 0.5. 54,55 If the fuzzy score value (A) is equal to or greater than 0.5, then the measured item is accepted. ...
Objective
This study aimed at developing and validating a web application on hypertension management called the D-PATH website.
Methods
The website development involved three stages: content analysis, web development, and validation. The model of Internet Intervention was used to guide the development of the website, in addition to other learning and multimedia theories. The content was developed based on literature reviews and clinical guidelines on hypertension. Then, thirteen experts evaluated the website using Fuzzy Delphi Technique.
Results
The website was successfully developed and contains six learning units. Thirteen experts rated the website based on content themes, presentation, interactivity, and instructional strategies. All experts reached a consensus that the web is acceptable to be used for nutrition education intervention.
Conclusion
D-PATH is a valid web-based educational tool ready to be used to help disseminate information on dietary and physical activity to manage hypertension. This web application was suitable for sharing information on dietary and physical activity recommendations for hypertension patients.
Introduction: The emergence of complex issues leads to the use of consensus or collective agreement methods; however, many scholars may wrongly use these methods interchangeably. Familiarity with collective decision-making methods can be helpful. Therefore, this article aims to review the conventional methods of collective decision-making methods and identify their characteristics. Information Sources or Data: This is a review study. The related studies in Farsi or En-glish were searched without time limit in Persian databases including Mag Iran, SID, Iran Medex and English databases including PubMed, Web of Science, and Science Direct using the keywords Focus Group technique, Nominal group technique, Delphi technique, Expert panel, and their Persian equivalents. Selection Methods for Study: More than 560 articles were found. After removing the duplicates , checking the relevance by reading titles, and checking the accessibility to the articles ,18 articles remained for the review. Combine Content and Results: Collective decision-making methods in various scientific fields including Delphi technique, expert panel technique, focus group technique and nominal group technique are very helpful for making the most appropriate decision when faced with the complexity of issues and the lack of sufficient information. Discussion: Collective decision-making methods are very helpful for making the most appropriate decisions regarding ambiguous issues. It is recommended that experts design appropriate guides to help young scholars in familiarizing with these collective agreement methods and choosing the appropriate technique.
Once the oil spills have occurred it has introduced acute effects on the environment, marine life, human life, society, communities, economics, aquatic ecosystems, and shoreline societies. This research analyses the ranking of the significant elements of collaboration that are essential in oil spill preparation and responses. The oil spill statistical incidents reported by the Department of Environment (DOE) from 2014 to 2023 have been analysed. Malaysia has encountered 140 oil spill cases. This research has utilized the questionnaire survey from the Modified Delphi Survey Method which involved the purposive expert respondents from 11 related agencies in Malaysia. The agencies are the Department of Environment (DOE), Marine Department, MMEA, Fishery Department, Meteorology Department, Royal Malaysian Navy, Air Operation Force, PIMMAG, PETRONAS, Fire and Rescue Department, and Marine Park. The data has been examined by Statistical Package for the Social Science (SPSS) version 26.0 analysis tools such as Frequency, Mean Analysis, Standard Deviation, and Cronbach’s Alpha gathered from 59 expert respondents. The ranking of the significant elements has been examined utilizing the Fuzzy Delphi Method (FDM). The results show the top five significant elements are: i) Coordinated decision making at 0.914, ii) Adequate communication among agencies at 0.913, iii) Good response time at 0.912, iv) Good leadership link at 0.908, and v) Large teamwork cooperation at 0.907. Nevertheless, as recommended by the experts the other significant element is efficient budget allocation and suitable equipment which are also essentials to be anticipated.
Energy production of a power plant is significantly affected by the availability of spare parts for machinery operation and maintenance (O&M). The availability of spare parts prevents prolonged breakdown and maintenance. However, even though some quantitative methods exist to determine the number of spare parts, to avoid the events occurring, generally companies follow manufacturer recommendation that leads to depositing a large number of spare parts. Retaining excessive stocks increases inventory costs. This paper presents a novel methodology to determine the type and number of spare parts of a power plant. The methodology, which is a blending of quantitative (modified multi-criteria classification method) and qualitative (semi-Delphi approach) methods, offers a practical way that even can be exercised with paper and pencil and a simple calculator, but sound. The methodology provides a significant inventory saving. A case study of dual-fuel power plants showed that the number of spare parts recommended was less than the number currently stocked by the company. The percentage is reduced to ~ 70% with cost saving on average 6.1%. The proposed methods suggested stocking parts categorized into H (highly required) and M (recommended) types of spare parts only.
This study aims to obtain expert agreement on the components and tests for the design and development of health-related physical fitness assessment framework in Malaysian preschool children. The development of this evaluation framework uses a Fuzzy Delphi method. Questionnaires were distributed to a total of 17 samples who were involved as field experts in the design and development process. The fitness component questionnaire consists of six section, namely; (1) demographic information; (2) body composition components; (3) cardiovascular endurance components; (4) hand muscle endurance components; (5) leg muscle strength components; and (6) flexibility components. Meanwhile, the fitness test questionnaire also has eight sections, namely; (1) demographic information; (2) body mass index; (3) body fat percentage measurement; (4) 20 meter shuttle run; (5) 1200 meter run; (6) hand grip; (7) standing long jump; and (8) sit-and-reach test. Data was analysed using the Fuzzy Delphi analysis template, showed that body composition components, cardiovascular endurance components, hand muscle endurance components, leg muscle strength components, and flexibility components were accepted as components of the assessment framework. Body mass index, shuttle run, hand grip, standing long jump, and sit-and-reach tests were accepted for each component, respectively. This is because the components and tests have met the requirements of the triangular fuzzy number, namely, the threshold value (d) ≤ 0.2 and the percentage of expert agreement ≥ 75%. This study has produced a health-related physical fitness assessment framework in Malaysian preschool children that can be used by various stakeholders, especially the Ministry of Education Malaysia (MOE) in improving the physical fitness of preschool children.
Penanganan penyakit tanaman padi saat ini terkendala oleh waktu dan banyaknya maslaah pada petani. sebagai solusi masalah ini, dibuat Sistem pakar diagnosis penyakit tanaman padi menggunakan bahasa Delphi. Sistem ini dirancang untuk menganalisa jenis penyakit pada tanaman padi berdasarkan gejala yang ada, disertai solusi yang tepat bagi penyakit tersebut. Hasil dari pengujian didapat sistem pakar ini dapat menentuan penyakit tanaman padi beserta solusinya dari gejala yang diinput.
The increasing complexity of educational demands in the 21st century has emphasised the need for innovative pedagogical frameworks, particularly in enhancing writing skills among ESL students. This study explores the application of the fuzzy Delphi method (FDM) to identify and validate the critical constructs necessary for designing and developing a multimodal learning framework aimed at improving writing skills in academic settings.The design of this study is a quantitative study using the fuzzy Delphi technique. A questionnaire instrument was used to collect research data. Eleven experts in TESL, educational technology, multimodal learning, and curriculum participated in the study. Data analysis results showed that the experts accepted all these elements through the expert consensus value above 75%, the threshold value (d) ≤ 0.2, and the fuzzy score (A) ≥ α - cut value = 0.5. Therefore, it shows that these elements have gained expert consensus and are needed to design and develop the multimodal learning framework for writing skills. The findings not only contribute to the theoretical understanding of multimodal learning in writing instruction but also offer practical guidelines for educators aiming to implement this framework in foundation-level classrooms. This research underscores the value of FDM in educational design, particularly in areas where expert judgement is critical for addressing complex instructional challenges.
This study aims to obtain expert consensus on the appropriate elements of open badge learning guidelines. The Fuzzy Delphi Method (FDM) was used to reach a consensus from 20 experts with different backgrounds. These guidelines were designed and developed based on the Fuzzy Delphi Method (FDM) as well as the integration of the ASSURE instructional design model by Heinich, Molenda, and Russell (1993). The guidelines were evaluated by selected experts using several criteria, and data were collected through questionnaires. Based on the results of the study, all items meet the requirements in the number of three fuzzy triangles and pass the de-fuzzing process (more than Alpha α – cut-off value 0.5). Therefore, the findings of the study show that the experts agree that this module is suitable to be implemented according to the six themes set. The findings highlight key constructs for OBL guideline development included several key constructs related to OBL guideline content, state guideline objectives, method, media/tool, and material selection, as well as recommendations for use and participation based on the ASSURE instructional design model. Further research can incorporate insights from fields such as psychology, sociology, and educational policy. The focus should be on technical aspects and use qualitative and quantitative methods to ensure that all students, regardless of background, receive fair qualifications and recognition.
Purpose
Stock investing choices of individual investors are predominantly influenced by heuristic biases, leading to sub-optimal choices. Accordingly, this study aims to identify, categorize, validate, prioritize, and find causality among the heuristic biases shaping stock investment decisions of individual investors.
Design/methodology/approach
This research offers original contribution by employing a hybrid approach combining fuzzy DELPHI method (FDM), fuzzy analytical hierarchy process (FAHP), and fuzzy decision-making trial and evaluation laboratory (F-DEMATEL) techniques to validate, prioritize, and find causality among the heuristic biases.
Findings
Twenty sub-heuristic biases were identified under five main heuristic bias categories. Out of which, 17 were validated using FDM. Further, availability and representativeness within main heuristic categories, and availability cascade and retrievability within sub-heuristic biases were prioritized using FAHP. Overconfidence and availability were identified as the causes among the five main biases by F-DEMATEL.
Practical implications
This study offers the stock investors a deeper understanding of heuristic biases and empowers them to make rational investment decisions.
Originality/value
This paper is the inaugural effort to identify, categorize, validate, prioritize and examine the cause-and-effect relationship among the heuristic biases.
Cardiovascular disease (CVD) is a major global cause of premature mortality. While multiple studies propose CVD mortality prediction models based on regression frameworks, incorporating causal understanding through causal inference approaches can enhance accuracy. This paper demonstrates a methodology combining evidence synthesis and expert knowledge to construct a causal model for premature CVD mortality using Directed Acyclic Graphs (DAGs). The process involves three phases: (1) initial DAG development based on the Evidence Synthesis for Constructing Directed Acyclic Graphs (ESC-DAGs) framework, (2) validation and consensus-building with 12 experts using the Fuzzy Delphi method (FDM), and (3) application to data analysis using population-based survey data linked with death records. Expert input refined the initial DAG model, achieving consensus on 45 causal paths. The revised model guided selection of confounding variables for adjustment. For example, to estimate the total effect of diabetes on premature CVD mortality, the suggested adjustment set included age, dietary pattern, genetic/family history, sex hormones, and physical activity. Testing different DAG models showed agreement between expert ratings and data accuracy from regression models. This systematic approach contributes to DAG methodology, offering a transparent process for constructing causal pathways for premature CVD mortality.
The language disability aphasia has several sub-diagnoses such as Amnestic, Broca, Global, and Wernicke. Data concerning 265
patients is available in the form of test scores and diagnoses, made by physicians according to the Aachen Aphasia Test. A
neural network model has been built, which is available for consultation on the World Wide Web. The neural network model is
in this paper compared with a fuzzy model. Rather than concluding which method provides the best approximation, the paper
acts as an example solution useful for other benchmark studies.
Key wordsamnestic–anomic–Broca aphasia–Wernicke aphasia–AAT–perceptron–MLP–FCM
Knowledge acquisition is known to be a critical bottleneck in building expert systems. In past decades, various methods and systems have been proposed to efficiently elicit expertise from domain experts. However, in building a medical expert system, disease symptoms are usually treated as time-irrelevant attributes, such that much important information is abandoned and hence, the performance of the constructed expert systems is significantly affected. To cope with this problem, in this paper, we propose a time scale-oriented approach to eliciting medical knowledge from domain experts. The novel approach takes the time scale into consideration, such that the variant of disease symptoms in different time scales can be precisely expressed. An application to the development of a medical expert system has depicted the superiority of our approach.
Over the past ten years, significant progress has been made in understanding how the knowledge acquisition process for classification systems can be automated. But during this period little attention has been paid to the problem of how to automate the knowledge acquisition process for systems that solve problems by constructing solutions. This paper describes SALT, a tool designed to assist with knowledge acquisition for configuration tasks. SALT1 assumes a problem-solvin g strategy involving stages of generate, test, backup, modify, and re generate. It exploits this problem-solving strategy to guide its interrogation of domain experts and to represent the knowledge they provide in a way that insures it will be brought to bear whenever relevant.2
MORE is a tool that assists in eliciting knowledge from domain experts. Acquired information is added to a domain model of qualitative causal relations that may hold among hypotheses, symptoms, and background conditions. After generating diagnostic rules from the domain model, MORE prompts for additional information that would allow a stronger set of diagnostic rules to be generated, MORE'S primary value lies in its understanding of what kinds of knowledge are likely to be diagnostically significant. By formulating its questions in a way that focuses on such knowledge, it makes the most effective use of the domain experts' time.1
Full-text available online at http://www.shortliffe.net/Buchanan-Shortliffe-1984/MYCIN Book.htm or at http://aitopics.org/publication/rule-based-expert-systems-mycin-experiments-stanford-heuristic-programming-project
Artificial intelligence, or AI, is largely an experimental science—at least as much progress has been made by building and analyzing programs as by examining theoretical questions. MYCIN is one of several well-known programs that embody some intelligence and provide data on the extent to which intelligent behavior can be programmed. As with other AI programs, its development was slow and not always in a forward direction. But we feel we learned some useful lessons in the course of nearly a decade of work on MYCIN and related programs. In this book we share the results of many experiments performed in that time, and we try to paint a coherent picture of the work. The book is intended to be a critical analysis of several pieces of related research, performed by a large number of scientists. We believe that the whole field of AI will benefit from such attempts to take a detailed retrospective look at experiments, for in this way the scientific foundations of the field will gradually be defined. It is for all these reasons that we have prepared this analysis of the MYCIN experiments.
Our OntoLearn system is an infrastructure for automated ontology learning from domain text. It is the only system, as far as we know, that uses natural language processing and machine learning techniques, and is part of a more general ontology engineering architecture. We describe the system and an experiment in which we used a machine-learned tourism ontology to automatically translate multiword terms from English to Italian. The method can apply to other domains without manual adaptation.
This article describes the Delphi method, how it is used, its underlying assumptions, its strengths and limitations, its potential benefits to higher education research, and some key considerations in its use. The authors illustrate the Delphi method by discussing a recent national study to develop a set of management audit assessment criteria. Their focus is on often overlooked and unique aspects of this versatile qualitative research methodology.
This research focuses on the problem of acquiring strategic knowledge-knowledge used by an agent to decide what action to perform next. Strategic knowledge is especially difficult to acquire from experts by conventional methods, and it is typically implemented with low-level primitives by a knowledge engineer. This dissertation presents a method for partially automating the acquisition of strategic knowledge from experts. The method consists of a representation for strategic knowledge, a technique for eliciting strategy from experts, and a learning procedure for transforming the information elicited from experts into operational and general form. The knowledge representation is formulated as strategy rules that associate strategic situations with equivalence classes of appropriate actions. The elicitation technique is based on a language of justifications with which the expert explains why a knowledge system should have chosen a particular action in a specific strategic situation. The learning procedure generates strategy rules from expert justifications in training cases, and generalizes newly-formed rules using syntactic induction operators. The knowledge acquisition procedure is embodied in an interactive program called ASK, which actively elicits justifications and new terms from the expert and generates operational strategy rules. ASK has been used by physicians to extend the strategic knowledge for a chest pain diagnosis application and by knowledge engineers to build a general strategy for the task of prospective diagnosis. A major conclusion is that expressive power can be traded for acquirability. By restricting the form of the representation of strategic knowledge, ASK can present a comprehensible knowledge elicitation medium to the expert and employ well-understood syntactic generalization operations to learn from the expert's explanations.
In this paper, we propose a knowledge acquisition method EMCUD which can elicitembedded meanings of the initial knowledge provided by domain experts. EMCUD also helps experts to decide uncertainty of the embedded meanings according to the relationships of the embedded meanings and the initial knowledge. The strategy of EMCUD could easily be added to the repertory grid-oriented methods or systems to enhance the knowledge in the prototype. We present a realistic example to show how EMCUD enriches the knowledge base constructed by a repertory grid-oriented method and hence ease the refinement processes.
A technique for use in any phase of planning where convergence of opinion seems desirable is described, along with its limitations and the methodological problems that may arise with its use.
Retrieving problem-solving information from a human expert is a major problem when building an expert system. Methods from George Kelly's personal construct psychology have been incorporated into a computer program, the Expertise Transfer System, which interviews experts, and helps them construct, analyse, test and refine knowledge bases. Conflicts in the problem-solving methods of the expert may be enumerated and explored, and knowledge bases from several experts may be combined into one consultation system. Fast (one to two hour) expert system prototyping is possible with the use of the system, and knowledge bases may be constructed for various expert system tools.
Acquiring knowledge from a human expert is a major problem when building a knowledge-based system. Aquinas, an expanded version of the Expertise Transfer System (ETS), is a knowledge-acquisition workbench that combines ideas from psychology and knowledge-based systems research to support knowledge-acquisition tasks. These tasks include eliciting distinctions, decomposing problems, combining uncertain information, incremental testing, integration of data types, automatic expansion and refinement of the knowledge base, use of multiple sources of knowledge and providing process guidance. Aquinas interviews experts and helps them analyse, test, and refine the knowledge base. Expertise from multiple experts or other knowledge sources can be represented and used separately or combined. Results from user consultations are derived from information propagated through hierarchies. Aquinas delivers knowledge by creating knowledge bases for several different expert-system shells. Help is given to the expert by a dialog manager that embodies knowledge-acquisition heuristics.Aquinas contains many techniques and tools for knowledge acquisition; the techniques combine to make it a powerful testbed for rapidly prototyping portions of many kinds of complex knowledge-based systems.
One of the most difficult and time-consuming activities in constructing an expert system is the process of knowledge acquisition. Our objectives are to identify a set of heuristics for expertise transfer and modeling based on our experience in knowledge acquisition for expert systems, and to formalize this knowledge as rules. Aquinas, a knowledge-acquisition workbench, contains tools that interview experts; analyse, test, and refine knowledge; and generate knowledge bases for expert system shells. A set of heuristics for knowledge acquisition has been defined and incorporated in the Dialog Manager sybsystem of Aquinas to provide guidance in the knowledge acquisition process to domain experts and knowledge engineers.The implementation of the Dialog Manager is described, and an example transcript shows the interaction of Aquinas, the Dialog Manager, and the expert. A preliminary classification for knowledge acquisition heuristics is proposed. Finally, issues in formalizing strategies for knowledge acquisition and a plan for future research are presented.
This paper gives a state-of-the-art report on the use of techniques based on personal construct psychology to automate knowledge engineering for expert systems. It presents the concept of knowledge support systems as interactive knowledge engineering tools, states the design criteria for such systems, and outlines the structure and key components of the KITTEN implementation. KITTEN includes tools for interactive repertory grid elicitation and entailment analysis that have been widely used for rapid prototyping of industrial expert systems. It also includes tools for text analysis, behavioral analysis and schema analysis, that offer complementary and alternative approaches to knowledge acquisition. The KITTEN implementation integrates these tools around a common database with utilities designed to give multiple perspectives on the knowledge base.
Knowledge-based knowledge acquisition means restricting the domain of knowledge that can be acquired and developing a conceptual model of the domain. We have built a prototype knowledge-based knowledge acquisition system for the domain of data analysis. A critique of the prototype has led to a design for a possibly practical data analysis knowledge acquisition system.
SALT is a knowledge-acquisition tool for generating expert systems that can use a propose-and-revise problem-solving strategy. The SALT-assumed method incrementally constructs an initial design by proposing values for design parameters, identifying constraints on design parameters as the design develops and revising design decisions in response to detection of constraint violations in the proposal. This problem-solving strategy provides the basis for SALT’s knowledge representation. SALT guides its interrogation of the domain expert using its identification of weaknesses in the knowledge base.
A general problem when modifying knowledge bases is that changes may degrade system performance. This is especially a problem when the knowledge base is large; it may be unclear how changing one item in a knowledge base containing thousands of items will affect overall system performance. Aquinas, a knowledge acquisition tool, uses knowledge elicitation and representation techniques and consultation review mechanisms to help alleviate this problem. The consultation review mechanisms are discussed here. We are experimenting with ways to use consultations and test cases to refine the information in an Aquinas knowledge base. The domain expert can use interactive graphics to specify the expected results. Modifications to the knowledge base may be tested against previous consultations; adjustments are suggested that make the results of all previous consultations as well as the current consultation correlate better with the expert's expectations. New traits are synthesized that would improve the performance of all previous consultations. New test cases are suggested that cover aspects missed by previous test cases. While we are just beginning to experiment with these techniques, they promise to provide help in improving problem-solving performance and gaining problem-solving insight.
In this paper, we employ the fuzzy repertory table technique to acquire the necessary domain knowledge for software agents to act as sellers and buyers using a bilateral, multi-issue negotiation model that can achieve optimal results in semi-competitive environments. In this context, the seller's domain knowledge that needs to be acquired is the rewards associated with the products and restrictions attached to their purchase. The buyer's domain knowledge that is acquired is their requirements and preferences on the desired products. The knowledge acquisition methods we develop involve constructing three fuzzy repertory tables and their associated distinctions matrixes. The first two are employed to acquire the seller agent's domain knowledge; and the third one is used, together with an inductive machine learning algorithm, to acquire the domain knowledge for the buyer agent.
A distributed anticipatory system formulation of knowledge acquisition and transfer processes is presented which provides scientific foundations for knowledge engineering. The formulation gives an operational model of the notion of expertise and the role it plays in our society. It suggests that the basic cognitive system that should be considered is a social organization, rather than an individual. Computational models of inductive inference already developed can be applied directly to the social model. One practical consequence of the model is a hierarchy of knowledge transfer methodologies which defines the areas of application of the knowledge-engineering techniques already in use. This analysis clarifies some of the problems of expertise transfer noted in the literature, in particular, what forms of knowledge are accessible through what methodologies. The model is being used as a framework within which to extend and develop a family of knowledge-support systems to expedite the development of expert-system applications.
MOLE can help domain experts build a heuristic problem-solver by working with them to generate an initial knowledge base and then detect and remedy deficiencies in it. The problem-solving method presupposed by MOLE makes several heuristic assumptions about the world, which MOLE is able to exploit when acquiring knowledge. In particular, by distinguishing between covering and differentiating knowledge and by allowing covering knowledge to drive the knowledge-acquisition process, MOLE is able to disambiguate an under-specified knowledge base and to interactively refine an incomplete knowledge base.
teiresias is a program designed to provide assistance on the task of building knowledge-based systems. It facilitates the interactive transfer of knowledge from a human expert to the system, in a high level dialog conducted in a restricted subset of natural language. This paper explores an example of teiresias in operation and demonstrates how it guides the acquisition of new inference rules. The concept of meta-level knowledge is described and illustrations given of its utility in knowledge acquisition and its contribution to the more general issues of creating an intelligent program.
This paper discusses the KADs approach to knowledge engineering. In KADS, the development of a knowledge-based system (KBS) is viewed as a modelling activity. A KBS is not a container filled with knowledge extracted from an expert, but an operational model that exhibits some desired behaviour that can be observed in terms of real-world phenomena. Five basic principles underlying the KADS approach are discussed, namely (i) the introduction of partial models as a means to cope with the complexity of the knowledge engineering process, (ii) the KADS four-layer framework for modelling the required expertise, (iii) the re-usability of generic model components as templates supporting top-down knowledge acquisition, (iv) the process of differentiating simple models into more complex ones and (v) the importance of structure—preserving transformation of models of expertise into design and implementation. The actual activities that a knowledge engineer has to undertake are briefly discussed. We compare the KADS approach to related approaches and discuss experiences and future developments. The approach is illustrated throughout the paper with examples in the domain of troubleshooting audio equipment.
This paper describes a knowledge-acquisition tool that builds expert systems for evaluating designs of electro-mechanical systems. The tool elicits from experts (1) knowledge in the form of a skeletal report, (2) knowledge about a large collection of report fragments, only some of which will be relevant to any specific report, and (3) knowledge of how to customize the report fragments for a particular application. The tool derives its power from exploiting its understanding of two problem-solving methods and of the different roles that knowledge plays in those two methods.†
The bottleneck in the process of building exnert svstems is retrieving the appropriate problem- solving knowledge from tKe hum-in expert. -Methods of knowledge elicitation and analysis from psychotherapy based on enhancements to George Kelly's Personal Construct Theory are applied to this process. The Expertise Transfer System is described which interviews a human expert and then constructs and analyzes the knowledge that the expert uses to solve his particular problem. The first version of the system elicits the initial knowledge needed to solve analysis problems without the intervention of a knowledge entineering: team. Fast (two hour) initial prototypyng auf expercsystems which run on KS- 300*,* (an extended version of EMYCIN) and OPS5 is also performed. Conflicts in the problem-solving methods of the expert may also be enumerated and explored.
A hybrid system for automatic knowledge acquisition for expert systems is presented. The system integrates artificial intelligence and cognitive science methods to construct knowledge bases employing different knowledge representation formalisms. For the elicitation of human declarative knowledge, the tool contains automated interview methods. The acquisition of human procedural knowledge is achieved by protocol analysis techniques. Textbook knowledge is captured by incremental text analysis. The goal structure of the knowledge elicitation methods is an intermediate knowledge-representation language on which frame, rule and constraint generators operate to build up the final knowledge bases. The intermediate knowledge representation level regulates and restricts the employment of the knowledge elicitation methods. Incomplete knowledge is laid open by pattern-directed invocation methods (the intermediate knowledge base watcher) triggering the elicitation methods to supplement the necessary knowledge.
Acquiring knowledge from a human expert is a major problem when building a knowledge-based system. Aquinas, an expanded version of the Expertise Transfer System (ETS), is a knowledge-acquisition workbench that combines ideas from psychology and knowledge-based systems research to support knowledge-acquisition tasks. These tasks include eliciting distinctions, decomposing problems, combining uncertain information, incremental testing, integration of data types, automatic expansion and refinement of the knowledge base, use of multiple sources of knowledge and providing process guidance. Aquinas interviews experts and helps them analyze, test, and refine the knowledge base. Expertise from multiple experts or other knowledge sources can be represented and used separately or combined. Results from user consultations are derived from information propagated through hierarchies. Aquinas delivers knowledge by creating knowledge bases for several different expert-system shells. Help is given to the expert by a dialog manager that embodies knowledge-acquisition heuristics. Aquinas contains many techniques and tools for knowledge acquisition; the techniques combine to make it a powerful testbed for rapidly prototyping portions of many kinds of complex knowledge-based systems.
A detailed look at MYCIN, an expert system for diagnosing bacterial infections and prescribing treatment for them. Issues covered include detailed examinations of knowledge acquisition, reasoning, explanation, tutoring, performance evaluation, and human interface
To evaluate the performance of CADIAG-II/RHEUMA as consultant in the primary evaluation of patients visiting a rheumatological outpatient clinic, a CADIAG-II/RHEUMA consultation was done for 54 patients and the list of generated diagnostic hypotheses was compared to each clinical discharge diagnosis. For 26 of a total of 126 rheumatological discharge diagnoses, no matching CADIAG-II/RHEUMA diagnosis was available. 94% of all other discharge diagnoses were found in the list of CADIAG-II/RHEUMA hypotheses, 82% among the first third of the list of hypotheses and 48% among the first five hypotheses. We identified the following factors limiting the ability of CADIAG-II/RHEUMA to generate a comprehensive and correctly ranked list of diagnostic hypotheses: (1) a large percentage of patients with early stages of not clearly identified rheumatological conditions; (2) the limited number of CADIAG-II/RHEUMA diagnoses compared to the large number of known rheumatological conditions; (3) the fact that rheumatological diseases are rarely characterized by a single pathognomonic feature but are usually diagnosed by combinations of rather unspecific findings.
An intelligent aid to assist knowledge engineers with interviewing experts KADS: A principle approach to knowledge-based system development
- O Bannon
- R M Schreiber
- G Wielinga
- B Breuker
O'Bannon, R. M. (1987). An intelligent aid to assist knowledge engineers with interviewing experts. In IEEE Western Conference on Expert Systems. Schreiber, G., Wielinga, B., & Breuker, J. (1993). KADS: A principle approach to knowledge-based system development. London: Academic Press.
Heuristics for expertise transfer: The automatic management of complex knowledge acquisition dialogs
- C M Kitto
- J H Boose
Kitto, C. M., & Boose, J. H. (1986). Heuristics for expertise transfer: The
automatic management of complex knowledge acquisition dialogs.
IEEE Expert System in Government Symposium.
ISODEPOR: System for the Interpretation of Isoquinetics. Res umenes de Comunicaciones del IV congreso internacional de entrenamiento deportivo
- J M Barreiro
- J P Caraca
- A Fernández
- A Ló Pez-Illescas
- C Montes
- J Olmo
Barreiro, J. M., Caraca, J. P., Fernández, A., Ló pez-Illescas, A., Montes,
C., & Olmo, J. (1997). ISODEPOR: System for the Interpretation
of Isoquinetics. Res umenes de Comunicaciones del IV congreso internacional de entrenamiento deportivo. Tecnolog ıa y Entrenamiento. Leó n.
The system architecture of an integrated medical consultation system and its implementation based on fuzzy technology
- G Kolousek
Kolousek, G. (1997). The system architecture of an integrated medical
consultation system and its implementation based on fuzzy technology.
Doctoral thesis, Technical University of Vienna, Austria.
Design and implementation of a web-based knowledge acquisition toolkit for medical expert consultation systems, Doctoral thesis
- K Boegl
Boegl, K. (1997). Design and implementation of a web-based knowledge
acquisition toolkit for medical expert consultation systems, Doctoral
thesis, Technical University of Vienna, Austria.
NeoETS: Capturing expert system knowledge in hierarchical rating grids
- J H Boose
- J M Bradshaw
Boose, J. H., & Bradshaw, J. M. (1986). NeoETS: Capturing expert system
knowledge in hierarchical rating grids. In IEEE expert system in
government symposium.
An intelligent aid to assist knowledge engineers with interviewing experts
- R M O'bannon
O'Bannon, R. M. (1987). An intelligent aid to assist knowledge engineers
with interviewing experts. In IEEE Western Conference on Expert
Systems.
Using the Delphi technology in institutional planning. New Directions for Institutional Research
- Uhl
The system architecture of an integrated medical consultation system and its implementation based on fuzzy technology
- G Kolousek
- Austria
- H Leitich
- H P Kiener
- G Kolarz
- C Schuh
- W Graninger
- K.-P Adlassnig
Kolousek, G. (1997). The system architecture of an integrated medical
consultation system and its implementation based on fuzzy technology.
Doctoral thesis, Technical University of Vienna, Austria.
Leitich, H., Kiener, H. P., Kolarz, G., Schuh, C., Graninger, W., &
Adlassnig, K.-P. (2001). A prospective evaluation of the medical
consultation system CADIAG-II/RHEUMA in a rheumatological
outpatient clinic. Methods of Information in Medicine, 40, 213-220.