Chapter

Advanced Learning Algorithm to Create FCM Models From Quantitative Data

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

This chapter describes an FCM model for decision-making and prediction problems where concepts are split into inputs and outputs. A key property of this model relies on its hybrid nature, where experts are expected to define the weights of some relationships while others are learned from the data. The learning procedure does not alter the weights defined by the experts, thus enabling hybrid reasoning. The learning algorithm computes the learnable weights from historical data using the Moore-Penrose inverse, a mathematically solid and fast operation. In this regard, we develop a toy example that illustrates how to use this algorithm to solve a prediction problem using an existing Python implementation. Supplementary aspects addressed in this chapter revolve around optimizing the network topology. As such, we describe two methods to identify and eliminate redundant relationships connecting inputs with outputs without significantly harming the simulation results. The chapter also conducts a parameter sensitivity analysis and compares the Moore-Penrose inverse-based learning method with metaheuristic-based methods. After reading this chapter, we expect the reader can master the foundations of these algorithms and apply them to datasets describing multi-output prediction problems.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

Article
Full-text available
This paper presents a Fuzzy Cognitive Map model to quantify implicit bias in structured datasets where features can be numeric or discrete. In our proposal, problem features are mapped to neural concepts that are initially activated by experts when running what-if simulations, whereas weights connecting the neural concepts represent absolute correlation/association patterns between features. In addition, we introduce a new reasoning mechanism equipped with a normalization-like transfer function that prevents neurons from saturating. Another advantage of this new reasoning mechanism is that it can easily be controlled by regulating nonlinearity when updating neurons’ activation values in each iteration. Finally, we study the convergence of our model and derive analytical conditions concerning the existence and unicity of fixed-point attractors.
Article
Full-text available
In the process of developing interventions targeting health behaviors (e.g., smoking cessation, safe sex, healthy nutrition), intervention planners interact with multiple types of information coming from various sources (e.g. theory, previous research, input from stakeholders). Intervention planners then need to meaningfully integrate this information into an analytical framework that guides the construction of the intervention components. Commonly used frameworks often oversimplify human behaviors despite acknowledging their inherent complexity. For instance, frameworks include chains of lists (in logic models) instead of richer structures considering loops and the dynamicity of relations between determinants. There is thus a lack of systematic approaches to effectively form such rich structures in health behavior interventions. While integrating many aspects of complexity in logic models may be possible, it also runs the risk of producing artifacts that become equally complex instead of equipping intervention planners and health researchers with decision-making tools to estimate the effects of interventions. Consequently, effective intervention development requires (i) techniques of systematic integration of information of different types and from various sources, which can account for (ii) the complex structure and functioning of interactions between the identified behavioral determinants, and (iii) will support the testing of intervention scenarios. In this paper, we show how a hybrid approach of Fuzzy Cognitive Maps and machine learning may support various aspects of complexity in intervention design by meeting all three requirements.
Article
Full-text available
The problem of inferring nonlinear and complex dynamical systems from available data is prominent in many fields including engineering, biological, social, physical, and computer sciences. Many evolutionary algorithm (EA) based network reconstruction methods have been proposed to address this problem, but they ignore several useful information of network structure, such as community structure, which widely exists in various complex networks. Inspired by the community structure, this paper develops a community-based evolutionary multi-objective network reconstruction framework to promote the reconstruction performance of EA-based network reconstruction methods due to their good performance; we refer this framework as CEMO-NR. CEMO-NR is a generic framework and any population-based multi-objective metaheuristic algorithm can be employed as the base optimizer. CEMO-NR employs the community structure of networks to divide the original decision space into multiple small decision spaces, and then any multi-objective evolutionary algorithm (MOEA) can be used to search for improved solutions in the reduced decision space. To verify the performance of CEMO-NR, this paper also designs a test suite for complex network reconstruction problems. Three representative MOEAs are embedded into CEMO-NR and compared with their original versions, respectively. The experimental results have demonstrated the significant improvement benefiting from the proposed CEMO-NR in 30 multi-objective network reconstruction problems.
Article
Full-text available
Fuzzy cognitive map (FCM) is an effective tool for modeling and simulating complex dynamic systems. Research on the problem of learning FCM from available time series is outstanding. Many batch FCM learning methods have been proposed to address this issue and the performance of these methods is satisfactory. However, these batch-learning methods are difficult to cope with large-scale datasets (for example, the memory in computers is not enough to store all instances) and real-time streaming data, leading to the failure of real-time and online analysis of complex systems. In this paper, unlike the existing batch learning methods, such as evolutionary-based and regression-based methods, we first extend FCM learning to an online setting, and then develop an effective algorithm based on a follow-the-regularized-leader (FTRL)-Proximal style learning algorithm to address the online FCM learning problem, termed as OFCM. The performance of OFCM is validated on constructed benchmark datasets, including synthetic datasets and gene regulatory network reconstruction datasets. The experimental results demonstrate the merits of OFCM, which can effectively solve online FCM learning problems. Index Terms-Fuzzy cognitive maps, online learning, large-scale optimization, data stream, big data.
Article
Full-text available
Hybrid artificial intelligence deals with the construction of intelligent systems by relying on both human knowledge and historical data records. In this paper, we approach this problem from a neural perspective, particularly when modeling and simulating dynamic systems. Firstly, we propose a Fuzzy Cognitive Map architecture in which experts are requested to define the interaction among the input neurons. As a second contribution, we introduce a fast and deterministic learning rule to compute the weights among input and output neurons. This parameterless learning method is based on the Moore-Penrose inverse and it can be performed in a single step. In addition, we discuss a model to determine the relevance of weights, which allows us to better understand the system. Last but not least, we introduce two calibration methods to adjust the model after the removal of potentially superfluous weights.
Article
Full-text available
This paper presents a new method based on fuzzy cognitive map (FCM) and possibilistic fuzzy c-means (PFCM) clustering algorithm for categorizing celiac disease (CD). CD is a complex disorder whose development is affected by genetics (HLA alleles) and gluten ingestion. The celiac patients who are not treated are at a high risk of cancer, malignant lymphoma, and small bowel neoplasia. Therefore, CD diagnosis and grading are of paramount importance. The proposed FCM models human thinking for the purpose of classifying patients suffering from CD. We used the latest grading method where three grades A, B1, and B2 are used. To improve FCM efficiency and classification capability, a nonlinear Hebbian learning algorithm is applied for adjusting the FCM weights. To this end, 89 cases are studied. Three experts extracted seven main determinant characteristics of CD which were considered as FCM concepts. The mutual effects of these concepts on one another and on the final concept were expressed in the form of fuzzy rules and linguistic variables. Using the center of gravity defuzzifier, we obtained the numerical values of these weights and obtained the total weight matrix. Ultimately, combining the FCM model with PFCM algorithm, we obtained the grades A, B1, and B2 accuracies as 88, 90, and 91%, respectively. The main advantage of the proposed FCM is the good transparency and interpretability in the decision-making procedure, which make it a suitable tool for daily usage in the clinical practice.
Article
Full-text available
In this paper, a new automated Fuzzy Cognitive Maps (FCMs) learning algorithm is developed to generate FCMs from historical data. Automated FCM learning algorithms are used to model and analyze systems which are very complex and cannot be handled by experts’ knowledge. The algorithm developed in this paper is based on the Imperialist Competitive Algorithm for global optimization and is called the Imperialist Competitive Learning Algorithm (ICLA). The ICLA divides the search space into several sections. It extracts the best knowledge from each section and follows a procedure to avoid local optima alongside rapid learning. Experiments have been conducted to compare the ICLA with other well-known FCM learning algorithms. The results show that in most cases, the ICLA performs better for learning FCMs in terms of solution accuracy and execution time. The testing results show clearly that the ICLA is a robust, fast and accurate FCM learning algorithm.
Article
Full-text available
A new heuristic approach for minimizing possiblynonlinear and non-differentiable continuous spacefunctions is presented. By means of an extensivetestbed it is demonstrated that the new methodconverges faster and with more certainty than manyother acclaimed global optimization methods. The newmethod requires few control variables, is robust, easyto use, and lends itself very well to parallelcomputation.
Article
Full-text available
The particle swarm is an algorithm for finding optimal regions of complex search spaces through the interaction of individuals in a population of particles. This paper analyzes a particle's trajectory as it moves in discrete time (the algebraic view), then progresses to the view of it in continuous time (the analytical view). A five-dimensional depiction is developed, which describes the system completely. These analyses lead to a generalized model of the algorithm, containing a set of coefficients to control the system's convergence tendencies. Some results of the particle swarm optimizer, implementing modifications derived from the analysis, suggest methods for altering the original algorithm in ways that eliminate problems and increase the ability of the particle swarm to find optima of some well-studied test functions
Article
Machine-learning solutions for pattern classification problems are nowadays widely deployed in society and industry. However, the lack of transparency and accountability of most accurate models often hinders their safe use. Thus, there is a clear need for developing explainable artificial intelligence mechanisms. There exist model-agnostic methods that summarize feature contributions, but their interpretability is limited to predictions made by black-box models. An open challenge is to develop models that have intrinsic interpretability and produce their own explanations, even for classes of models that are traditionally considered black boxes like (recurrent) neural networks. In this article, we propose a long-term cognitive network (LTCN) for interpretable pattern classification of structured data. Our method brings its own mechanism for providing explanations by quantifying the relevance of each feature in the decision process. For supporting the interpretability without affecting the performance, the model incorporates more flexibility through a quasi-nonlinear reasoning rule that allows controlling nonlinearity. Besides, we propose a recurrence-aware decision model that evades the issues posed by the unique fixed point while introducing a deterministic learning algorithm to compute the tunable parameters. The simulations show that our interpretable model obtains competitive results when compared to state-of-the-art white and black-box models.
Article
Many real-life situations require ranking alternative decisions with respect to multiple criteria. The problem becomes more complicated when the knowledge of the considered criteria is vague and unreliable. In order to cope with the vagueness, the values of criteria have to be represented in an approximated way. To overcome the lack of reliability, many experts can be involved in the decision process and thus cooperatively elaborate more credible decisions. However, as it turns out, experts are usually unable to provide plausible information on interactions between vague decision criteria. Since those interactions substantially affect the ranking of alternative decisions, they should be taken into account in the decision process. To effectively cope with that issue, we propose a novel multi-criteria group decision-making method that integrates TOPSIS (technique for order of preference by similarity to ideal solution) with IVIFCMs (interval-valued intuitionistic fuzzy cognitive maps), a tool that is able to model interactions among highly imprecise criteria. We illustrate the application of the proposed IVIFCM-TOPSIS to the supplier selection task. Finally, we present the advantages derived from the use of our method compared to competitive approaches known from the literature. In particular, we show that our method is more consistent than the existing state-of-the-art methods used to solve the addressed decision problem.
Article
In the last decade, increasing interest has been paid to human factor for systems analysis and decision-making process. In particular, researchers have directed their attention to the human reasoning understanding and the individual perception of a specific problem. Several applications allow users to model decision-making support system starting from the experts’ knowledge and judgments. This paper examines two specific approaches such as Decision-Making Trial & Evaluation Laboratory and Fuzzy Cognitive Maps; it points out on their similarity and differences, with the aim to propose a hybrid modelling decision support system. This is the first time of proposing this approach, and the receiving outcomes are promising that the considered approaches can be used jointly showing the ability to model dynamical behaviour and create an advanced decision support system.
Article
This paper is focused on an innovative fuzzy cognitive maps extension called fuzzy grey cognitive maps (FGCMs). FGCMs are a mixture of fuzzy cognitive maps and grey systems theory. These have become a useful framework for facing problems with high uncertainty, under discrete small and incomplete datasets. This paper deals with the problem of uncertainty propagation in FGCM dynamics with Hebbian learning. In addition, this paper applies differential Hebbian learning (DHL) and balanced DHL to FGCMs for the first time. We analyze the uncertainty propagation in eight different scenarios in a classical chemical control problem. The results give insight into the propagation of the uncertainty or greyness in the iterations of the FGCMs. The results show that the nonlinear Hebbian learning is the choice with less uncertainty in steady final grey states for Hebbian learning algorithms. IEEE
Article
Learning large-scale sparse fuzzy cognitive maps (FCMs) from observed data automatically without any prior knowledge remains an outstanding problem. Most existing methods are slow and have difficulty in dealing with large-scale FCMs, because of the large searching space. We develop a framework based on compressed sensing (CS), a convex optimization method, to learn large-scale sparse FCMs, called CS-FCM. Combining with the sparsity of FCMs, the task of learning FCMs is first decomposed into sparse signal reconstruction problems. The ability of CS to exactly recover the sparse signals provides CS-FCM the probability to exactly learn FCMs. In the experiments, CS-FCM is applied to learn both synthetic data with varying sizes and densities and real-life data. The results show that CS-FCM obtains good performance by just learning from a small amount of data. CS-FCM can effectively learn sparse FCMs with 1000 nodes and even more, which have one million weights to be determined. CS-FCM is also applied to reconstruct gene regulatory networks (GRNs), and the well-known benchmark datasets DREAM3 and DREAM4 are tested. The results show that CS-FCM also obtains high accuracy in reconstructing GRNs. CS-FCM establishes a paradigm for learning large-scale sparse FCMs with high accuracy. IEEE
Article
Fuzzy cognitive maps (FCMs) are distributed computation systems used for qualitative modelling and behaviour simulation. Constructing an FCM is a time-consuming process and the quality of the resulting map is difficult to assess. In this paper we propose an extension to FCMs that self-adjusts the FCM based on real data from the modelled system. The self-adjusting FCM (SAFCM) changes the cause–effect relationships and concept inferences for each system data point with the goal of reducing the error between real data and values produced by the map. In this way, the burden of map construction imposed on the map builder is reduced and the initially constructed map can be evaluated by examining the degree of change caused by the self-adjustment. We tested the SAFCM on two case studies where we measured the degree of change to the initial map structure set up by an expert. The experiments showed that the self-adjusted maps produced results that were closer to real data than the maps that were initially set up by the expert. We also compared the SAFCM to a basic FCM and to an FCM that used a standard learning algorithm. The results showed that our algorithm had higher accuracy.
Article
Fuzzy cognitive maps (FCMs) have been used to describe and model the behavior of complex systems. Learning large-scale FCMs from a small amount of data without any a priori knowledge remains an outstanding problem. In particular, a significant challenge arises when limited amounts of data are accompanied by noise. Here, we develop a framework based on the least absolute shrinkage and selection operator (lasso), a convex optimization method, to robustly learn FCMs from noisy data, which is termed LASSOFCM. In LASSOFCM, the task of learning FCMs is decomposed into sparse signal reconstruction problems owing to the sparseness of FCMs. In the experiments, LASSOFCM is applied to learn synthetic data with varying sizes and densities. The results show that LASSOFCM obtains good performance in learning FCMs from time series with or without noise and outperforms the existing methods. Moreover, we apply LASSOFCM to reconstruct gene regulatory networks (GRNs) using the benchmark dataset DREAM3 and DREAM4, and LASSOFCM achieves good performance. LASSOFCM establishes a paradigm for learning large-scale FCMs with high accuracy and has potential applications in a wide range of fields.
Conference Paper
The article is focused on the issue of complexity of Fuzzy Cognitive Maps designed to model time series. Large Fuzzy Cognitive Maps are impractical to use. Since Fuzzy Cognitive Maps are graph-based models, when we increase the number of nodes, the number of connections grows quadratically. Therefore, we posed a question how to simplify trained FCM without substantial loss in map’s quality. We proposed evaluation of nodes’ and weights’ relevance based on their influence in the map. The article presents the method first on synthetic time series of different complexity, next on several real-world time series. We illustrate how simplification procedure influences MSE. It turned out that with just a small increase of MSE we can remove up to 13\frac{1}{3} of nodes and up to 16\frac{1}{6} of weights for real-world time series. For regular data sets, like the synthetic time series, FCM-based models can be simplified even more.
Article
In this paper we propose a new approach to learning fuzzy cognitive maps (FCMs) as a predictive model for time series forecasting. The first contribution of this paper is the dynamic optimization of the FCM structure, i.e., we propose to select concepts involved in the FCM model before every prediction is made. In addition, the FCM transformation function together with the corresponding parameters are proposed to be optimized dynamically. Finally, the FCM weights are learned. In this way, the entire FCM model is learned in a completely new manner, i.e., it is continuously adapted to the current local characteristics of the forecasted time series. To optimize all of the aforementioned elements, we apply and compare 5 different population-based algorithms: genetic, particle swarm optimization, simulated annealing, artificial bee colony and differential evolution. For the evaluation of the proposed approach we use 11 publicly available data sets. The results of comparative experiments provide evidence that our approach offers a competitive forecasting method that outperforms many state-of-the-art forecasting models. We recommend to use our FCM-based approach for the forecasting of time series that are linear and tend to be trend stationary.
Article
Fuzzy cognitive maps (FCMs) are cognition fuzzy influence graphs, which are based on fuzzy logic and neural networks. Many automated learning algorithms have been proposed to reconstruct FCMs from data, but most learned maps using such methods are much denser than those constructed by human experts. To this end, we first model the FCM learning problem as a multiobjective optimization problem, and then propose a multiobjective evolutionary algorithm, labeled as MOEA-FCM, to learn FCM models. MOEA-FCM is able to learn FCMs with varying densities at the same time from input historical data, which can provide candidate solutions with different properties for decision makers. In the experiments, the performance of MOEA-FCM is validated on both synthetic and real data with varying sizes and densities. The experimental results demonstrate the efficiency of MOEA-FCM and show that MOEA-FCM can not only reconstruct FCMs with high accuracy without expert knowledge, but also create a diverse Pareto optimal front which consists of FCMs with varying densities. The significance of this study is that decision makers can choose different FCM models provided by MOEA-FCM based on their practical requirements.
Article
If a rubber fender, whose purpose is to absorb the shock energy that is generated when a ship is berthing, fails because of deterioration and vulcanization, the ship and the pier structure may incur serious damage. In this paper, we introduce a fuzzy rule-based decision-making system to predict the maintenance timing of rubber fenders by diagnosing their faults and evaluating their lifetimes on the basis of the knowledge and experience of experts. First, we built a case library containing information concerning fender lifetimes and points in these lifetimes when faults occurred. This was gathered on the basis of the results of testing the lifetime of rubber materials under accelerated conditions and a knowledge base of the physical properties of fenders in order to evaluate the deterioration in a rubber fender. Moreover, after representing the key factors affecting the deterioration of rubber fenders as a fuzzy cognitive map that reflects the knowledge and experience of domain experts, we drew inferences regarding the heuristic factors of the lifetime of rubber, the conceptual centrality of which is relatively high according to the cause-and-effect relationship between key factors. Finally, we propose a diagnostic process for the performance deterioration of rubber fenders based on the various factors that affect the lifetime of rubber and its multiple physical properties in the form of a decision-making tree. We then applied it to rule-based and case-based reasoning. Copyright © 2014 John Wiley & Sons, Ltd.
Article
A detailed comparative analysis of the Hebbian-like learning algorithms applied to train Fuzzy Cognitive Maps (FCMs) operating as pattern classifiers, is presented in this paper. These algorithms aim to find appropriate weights between the concepts of the FCM classifier so it equilibrates to a desired state (class mapping). For these purposes, six different types of Hebbian learning algorithms from the literature have been selected and studied in this work. Along with the theoretical description of these algorithms and the analysis of their performance in classifying known patterns, a sensitivity analysis of the applied classification scheme, regarding some configuration parameters have taken place. It is worth noting that the algorithms are studied in a comparative fashion, under common configurations for several benchmark pattern classification datasets, by resulting to useful conclusions about their training capabilities.
Conference Paper
Fuzzy cognitive maps (FCMs) are a convenient tool for modeling of dynamic systems by means of concepts connected by cause-effect relationships. The FCM models can be developed either manually (by the experts) or using an automated learning method (from data). Some of the methods from the latter group, including recently proposed Nonlinear Hebbian Learning (NHL) algorithm, use Hebbian law and a set of conditions imposed on output concepts. In this paper, we propose a novel approach named data-driven NHL (DD-NHL) that extends NHL method by using historical data of the input concepts to provide improved quality of the learned FCMs. DD-NHL is tested on both synthetic and real-life data, and the experiments show that if historical data are available, then the proposed method produces better FCM models when compared with those formed by the generic NHL method.
Article
Fuzzy cognitive map is a soft computing technique for modeling systems, which combines synergistically the theories of neural networks and fuzzy logic. Developing of fuzzy cognitive map (FCM) relies on human experience and knowledge, but still exhibits weaknesses in utilization of learning methods. The critical dependence on experts and the potential uncontrollable convergence to undesired steady-states are important deficiencies to manage FCMs. Overcoming these deficiencies will improve the efficiency and robustness of the FCM methodology. Learning and convergence algorithms constitute the mean to improve these characteristics of FCMs, by modifying the values of cause–effect weights among concepts. In this paper a new learning algorithm that alleviates the problem of the potential convergence to a steady-state, named Active Hebbian Learning (AHL) is presented, validated and implemented. This proposed learning procedure is a promising approach for exploiting experts’ involvement with their subjective reasoning and at the same time improving the effectiveness of the FCM operation mode and thus it broadens the applicability of FCMs modeling for complex systems.
Article
This note discusses some aspects of the estimation of the density function of a univariate probability distribution. All estimates of the density function satisfying relatively mild conditions are shown to be biased. The asymptotic mean square error of a particular class of estimates is evaluated.