No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Necessity and Difficulty of Navigating Uncertainty to Develop an Individual-Level Computational Model
Abstract
The design of an individual-level computational model requires modelers to deal with uncertainty by making assumptions on causal mechanisms (when they are insufficiently characterized in a problem domain) or feature values (when available data does not cover all features that need to be initialized in the model). The simplifications and judgments that modelers make to construct a model are not commonly reported or rely on evasive justifications such as ‘for the sake of simplicity’, which adds another layer of uncertainty. In this paper, we present the first framework to transparently and systematically investigate which factors should be included in a model, where assumptions will be needed, and what level of uncertainty will be produced. We demonstrate that it is computationally prohibitive (i.e. NP-Hard) to create a model that supports a set of interventions while minimizing uncertainty. Since heuristics are necessary, we formally specify and evaluate two common strategies that emphasize different aspects of a model, such as building the ‘simplest’ model in number of rules or actively avoiding uncertainty.
ResearchGate has not been able to resolve any citations for this publication.
The Overview, Design concepts and Details (ODD) protocol for describing Individual- and Agent-Based Models (ABMs) is now widely accepted and used to document such models in journal articles. As a standard- ized document for providing a consistent, logical and readable account of the structure and dynamics of ABMs, some research groups also find it useful as a workflow for model design. Even so, there are still limitations to ODD that obstruct its more widespread adoption. Such limitations are discussed and addressed in this paper: the limited availability of guidance on how to use ODD; the length of ODD documents; limitations of ODD for highly complex models; lack of sufficient details of many ODDs to enable reimplementation without access to the model code; and the lack of provision for sections in the document structure covering model design ratio- nale, the model’s underlying narrative, and the means by which the model’s fitness for purpose is evaluated. We document the steps we have taken to provide better guidance on: structuring complex ODDs and an ODD summary for inclusion in a journal article (with full details in supplementary material; Table 1); using ODD to point readers to relevant sections of the model code; update the document structure to include sections on model rationale and evaluation. We also further advocate the need for standard descriptions of simulation ex- periments and argue that ODD can in principle be used for any type of simulation model. Thereby ODD would provide a lingua franca for simulation modelling.
Systems thinking (ST) skills are often the foundation of sustainability science curricula. Though ST skill sets are used as a basic approach to reasoning about complex environmental problems, there are gaps in our understanding regarding the best ways to promote and assess ST learning in classrooms. Since ST learning provides Science, Technology, Engineering, and Mathematics (STEM) students’ important skills and awareness to participate in environmental problem-solving, addressing these gaps is an important STEM learning contribution. We have created guidelines for teaching and measuring ST skills derived from a hybrid of a literature review and through case study data collection. Our approach is based on semi-quantitative cognitive mapping techniques meant to support deep reasoning about the complexities of social–ecological issues. We begin by arguing that ST should be evaluated on a continuum of understanding rather than a binary of correct/incorrect or present/absent. We then suggest four fundamental dimensions of teaching and evaluating ST which include: (1) system structure, (2) system function, (3) identification of leverage points for change, and (4) trade-off analysis. Finally, we use a case study to show how these ideas can be assessed through cognitive maps to help students develop deep system understanding and the capacity to propose innovative solutions to sustainability problems.
Artificial societies used to guide and evaluate policies should be built by following "best practices". However, this goal may be challenged by the complexity of artificial societies and the interdependence of their sub-systems (e.g., built environment, social norms). We created a list of seven practices based on simulation methods, specific aspects of quantitative individual models, and data-driven modeling. By evaluating published models for public health with respect to these ideal practices, we noted significant gaps between current and ideal practices on key items such as replicability and uncertainty. We outlined opportunities to address such gaps, such as integrative models and advances in the computational machinery used to build simulations.
Complex problems often require coordinated actions from stakeholders. Agreeing on a course of action can be challenging as stakeholders have different views or ‘mental models’ of how a problem is shaped by many interacting causes. Participatory modeling allows to externalize mental models in forms such as causal maps. Participants can be guided by a trained facilitator (with limitations of costs and availability) or use a free software (with limited guidance). Neither solution easily copes with large causal maps, for instance by preventing redundant concepts. In this paper, we leveraged voice-activated virtual assistants to create causal models at any time, without costs, and by avoiding redundant concepts. Our three case studies demonstrated that our artificial facilitator could create causal maps similar to previous studies. However, it is limited by current technologies to identify concepts when the user speaks (i.e. entities), and its design had to follow pre-specified rules in the absence of sufficient data to generate rules by discriminative machine-learned methods.
Models are predominantly developed using either quantitative data (e.g., for structured equation models) or qualitative data obtained through questionnaires designed by researchers (e.g., for fuzzy cognitive maps). The wide availability of social media data and advances in natural language processing raise the possibility of developing models from qualitative data naturally produced by users. This is of particular interest for public health surveillance and policymaking, as social media provide the opinions of constituents. In this paper, we contrast a model produced by social media with one produced via expert reports. We use the same process to derive a model in each case, thus focusing our analysis on the impact of source selection. We found that three expert reports were sufficient to touch on more aspects of a complex problem (measured by the number of relationships) than several million tweets. Consequently, developing a model exclusively from social media may lead to oversimplifying a problem. This may be avoided by complementing social media with expert reports. Alternatively, future research should explore whether a much larger volume of tweets would be needed, which also calls for improvements in scalable methods to transform qualitative data into models.
Multiple Indonesian cities currently aim to qualify as ‘smart cities’. Previous research on defining smart cities (e.g., the implementation-oriented Maturity Model) tends to focus on components over interrelationships, is challenging to apply to a specific context such as Indonesia, and offers limited support for policy-relevant questions. In this paper, we propose to address these shortcomings to support policymakers in identifying concrete action plans in Indonesia specifically. Our approach clarifies interrelationships for the context of use, and supports structural (e.g., what aspects of a ‘smart city’ are impacted by an intervention?) as well as what-if policy questions. We started with a systems science approach to develop a cognitive map of the components and their interrelationships, as is increasingly done in participatory modeling and particularly in socio-ecological management. We transformed semi-structured interviews of 10 Indonesian experts into maps and assembled them to create the first comprehensive smart cities cognitive map for Indonesia, totaling 52 concepts and 98 relationships. While a cognitive map already provides support for decision-making (e.g., by identifying loops in the system), it is only conceptual and thus cannot form predictions. Consequently, we extended our cognitive map into a Fuzzy Cognitive Map (FCM), whose inference abilities allow to examine the dynamic response of an indicator (e.g., ‘smart city’) in response to different interventions. As Fuzzy Cognitive Maps include the strengths of interrelationships but not the notion of time, future research may refine our model using System Dynamics. This refinement would support policymakers in investigating when to conduct and/or evaluate an intervention.
Visual analytics (VA) provides an interactive way to explore vast amounts of data and find interesting patterns. This has already benefited the development of computational models, as the patterns found using VA can then become essential elements of the model. Similarly, recent advances in the use of VA for the data cleaning stage are relevant to computational modelling given the importance of having reliable data to populate and check models. In this paper, we demonstrate via case studies of medical models that VA can be very valuable at the conceptual stage, to both examine the fit of a conceptual model with the underlying data and assess possible gaps in the model. The case studies were realized using different modelling tools (e.g., system dynamics or network modelling), which emphasizes that the relevance of VA to medical modelling cuts across techniques. Finally, we discuss how the interdisciplinary nature of modelling for medical applications requires an increased support for collaboration, and we suggest several areas of research to improve the intake and experience of VA for collaborative modelling in medicine.
Two thirds of American adults are now obese or overweight, and this proportion continues to increase. Primary care doctors are placed at the front lines in addressing the obesity problem. However, they often experience difficulties in tackling the complex problem of obesity by reducing it to a simple matter of diet and exercise that ignores the underlying psychosocial mechanisms even though many can be modified. We propose the first fuzzy cognitive map for the diagnosis of obesity based on psychosocial features. In our model, the existence of relationships between factors relies on a thorough literature review, and the strength of these relationships was estimated by a team of experts. The experts’ estimations were converted to values used by our model through different techniques. We found that the choice of technique made little difference to the model's prediction. Through test cases, we show that small descriptions of patients’ cases can be used for diagnosis. These descriptions could be obtained by filling a questionnaire before consulting with a doctor, therefore limiting the increase in consultation time while providing a useful guidance for possible behaviour change.
The forces which affect homelessness are complex and often interactive in nature. Social forces suchas addictions, family breakdown, and mental illness are compounded by structural forces such aslack of available low-cost housing, poor economic conditions, and insufficient mental health services.Together these factors impact levels of homelessness through their dynamic relations. Historic models,which are static in nature, have only been marginally successful in capturing these relationships.
Fuzzy Logic (FL) and fuzzy cognitive maps (FCMs) are particularly suited to the modeling of complexsocial problems, such as homelessness, due to their inherent ability to model intricate, interactivesystems often described in vague conceptual terms and then organize them into a specific, concreteform (i.e., the FCM) which can be readily understood by social scientists and others. Using FLwe converted information, taken from recently published, peer reviewed articles, for a select groupof factors related to homelessness and then calculated the strength of influence (weights) for pairsof factors. We then used these weighted relationships in a FCM to test the effects of increasingor decreasing individual or groups of factors. Results of these trials were explainable according tocurrent empirical knowledge related to homelessness.
Prior graphic maps of homelessness have been of limited use due to the dynamic nature of the conceptsrelated to homelessness. The FCM technique captures greater degrees of dynamism and complexitythan static models, allowing relevant concepts to be manipulated and interacted. This, in turn, allowsfor a much more realistic picture of homelessness. Through network analysis of the FCM we de-termined that Education exerts the greatest force in the model and hence impacts the dynamism andcomplexity of a social problem such as homelessness.
The FCM built to model the complex social system of homelessness reasonably represented realityfor the sample scenarios created. This confirmed that the model worked and that a search of peerreviewed, academic literature is a reasonable foundation upon which to build the model. Further, itwas determined that the direction and strengths of relationships between concepts included in this mapare a reasonable approximation of their action in reality. However, dynamic models are not withouttheir limitations and must be acknowledged as inherently exploratory.
We consider the practical NP-complete problem of finding a minimum weight spanning tree with both edge weights and inner nodes weights. We present two polynomial time algorithms with approximation factors of 2.35 · ln n and 2Hn, respectively, where n is the number of nodes in the graph and Hn is the n-th Harmonic number. This nearly matches the lower bound of (l-in )Hn, for any in ge 0. We also give an approximation algorithm with approximation factor Delta - 1, where Delta is the maximum degree of the graph. For metric spaces, we give a 3.105-approximation algorithm and show that an approximation factor of 1.463 is impossible unless {NP subseteq DTIME[n^{O(log longn)} ]}.
A new approach is suggested under the slogan "Keep it Descriptive Stupid" (KIDS) that encapsulates a trend in increasingly descriptive agent- based social simulation. The KIDS approach entails one starts with the simulation model that relates to the target phenomena in the most straight- forward way possible, taking into account the widest possible range of evidence, including anecdotal accounts and expert opinion. Simplification is only applied if and when the model and evidence justify this. This contrasts sharply with the KISS approach where one starts with the simplest possible model and only moves to a more complex one if forced to. An example multi- agent simulation of domestic water demand and social influence is described.
The Third Edition of Nigel Gilbert's hugely successful Researching Social Life covers the whole range of methods from quantitative to qualitative in a down-to-earth and unthreatening manner. Gilbert's text offers the best coverage of the full scope of research methods of any of the leading textbooks in the field, making this an essential text for any student starting a research methods course or doing a research project. This thoroughly revised text is driven by the expertise of a writing team comprised of internationally-renowned experts in the field. New to the Third Edition are chapters on: - Searching and Reviewing the Literature - Refining the Question - Grounded Theory and Inductive Research - Mixed Methods - Participatory Action Research - Virtual Methods - Narrative Analysis A number of useful features, such as worked examples, case studies, discussion questions, project ideas and checklists are included throughout the book to help those new to research to engage with the material. Researching Social Life follows the 'life cycle' of a typical research project, from initial conception through to eventual publication. Its breadth and depth of coverage make this an indispensable must-have textbook for students on social research methods courses in any discipline.
We improve the approximation ratios for two optimization problems in planar graphs. For node-weighted Steiner tree, a classical network-optimization problem, the best achievable approximation ratio in general graphs is Θ(log n), and nothing better was previously known for planar graphs. We give a constant-factor approximation for planar graphs. Our algorithm generalizes to allow as input any nontrivial minor-closed graph family, and also generalizes to address other optimization problems such as Steiner forest, prizecollecting Steiner tree, and network-formation games. The second problem we address is group Steiner tree: given a graph with edge weights and a collection of groups (subsets of nodes), find a minimum-weight connected subgraph that includes at least one node from each group. The best approximation ratio known in general graphs is O(log3 n), or O(log2 n) when the host graph is a tree. We obtain an O(log npolyloglog n) approximation algorithm for the special case where the graph is planar embedded and each group is the set of nodes on a face. We obtain the same approximation ratio for the minimum-weight tour that must visit each group. © 2014 ACM.
Fuzzy Cognitive Maps were missing in the latest guidance by Kelly et al. (2013), which reviewed five common integrated modeling approaches. Through a scoping review, the paper shows how Fuzzy Cognitive Maps satisfy key criteria for suitability in integrated environmental assessment, such as model purpose, types of data available to build a model, requirements in terms of spatial and temporal scales, stakeholder involvement, and treatment of uncertainty. The paper consequently updates the guiding framework that can be used by modelers to select the most appropriate modeling approach by including Fuzzy Cognitive Map techniques. We argue that the method can be the starting point of any integrated environmental assessment research to map out perspectives and potential system sensitivities. They constitute a great addition to the integrated assessment modeling toolbox as they can complement quantitative model approaches that are invariably constructed around a single perspective that relates to available data.
We examine several notions of what makes a simulation adequate, i.e., good enough for application. Social simulations, or artificial societies, are complex systems that integrate data from multiple sources, model intricate behaviors and processes, and generate large amounts of data. Adequacy, thus, is a nuanced concept that has to be applied at every stage of the development of the simulation, from initial conceptualization to final use. We go through this process step by step, and suggest some best practices for developing adequate models, drawn from experience with developing large-scale social simulations in multiple domains.
The past several years have seen advances in the area of simulation for social‐behavioral modeling. Nevertheless, in many ways, the field has not come of age. This paper argues that the field is being hampered by a lack of understanding by proponents and detractors of the nature of social‐behavioral simulation and the underlying challenges for advancement of the area.
Developing a comprehensive explanation of complex social phenomena is a difficult task that analysts often have to perform using vast collections of text documents. On the one hand, solutions exist to assist analysts in creating causal maps from text documents, but these can only articulate the relationships at work in a problem. On the other hand, Fuzzy Cognitive Maps (FCMs) can articulate these relationships and perform simulations, but no environment exists to help analysts in iteratively developing FCMs from text. In this paper, we detail the design and implementation of the first tool that allows analysts to develop FCMs from text collections, using interactive visualizations. We make three contributions: (i) we combine text mining and FCMs, (ii) we implement the first visual analytics environment built on FCMs, and (iii) we promote a strong feedback loop between interactive data exploration and model building. We provide two case studies exemplifying how to create a model from the ground-up or improve an existing one. Limitations include the increase in display complexity when working with large collection of files, and the reliance on KL-divergence for ad-hoc retrieval. Several improvements are discussed to further support analysts in creating high-quality models through interactive visualizations.
Various tools and methods are used in participatory modelling, at different stages of the process and for different purposes. The diversity of tools and methods can create challenges for stakeholders and modelers when selecting the ones most appropriate for their projects. We offer a systematic overview, assessment, and categorization of methods to assist modelers and stakeholders with their choices and decisions. Most available literature provides little justification or information on the reasons for the use of particular methods or tools in a given study. In most of the cases, it seems that the prior experience and skills of the modelers had a dominant effect on the selection of the methods used. While we have not found any real evidence of this approach being wrong, we do think that putting more thought into the method selection process and choosing the most appropriate method for the project can produce better results. Based on expert opinion and a survey of modelers engaged in participatory processes, we offer practical guidelines to improve decisions about method selection at different stages of the participatory modeling process.
Modeling how patients adopt personal health technology is a challenging problem: Decision-making processes are largely unknown, occur in complex, multi-stakeholder settings, and may play out differently for different products and users. To address this problem, this chapter develops a soft analytics approach, based on Fuzzy Cognitive Maps (FCM) that leads to adoption models that are specific for a particular product and group of adopters. Its empirical grounding is provided by a case study, in which a group of women decides whether to adopt a wearable remote healthcare monitoring device. The adoption model can simulate different product configurations and levels of support and provide insight as to what scenarios will most likely lead to successful adoption. The model can be used by product developers and rollout managers to support technology planning decisions.
This paper provides empirical agent-based modellers with a generic framework that allows for a structured and unambiguous description of the characterisation and parameterisation process. As methodological recommendations depend on contextual circumstances cases are defined to distinguish particular modelling situations, which require different methods for robust model characterisation and parameterisation. Both combined allows in a comparative perspective for analysing how effectively different (sequences of) methods perform in similar contexts.
Recently, Fuzzy Grey Cognitive Map (FGCM) has been proposed as a FCM extension. It is based on Grey System Theory, that it is focused on solving problems with high uncertainty, under discrete incomplete and small data sets. The FGCM nodes are variables, representing grey concepts. The relationships between nodes are represented by directed edges. An edge linking two nodes models the grey causal influence of the causal variable on the effect variable. Since FGCMs are hybrid methods mixing Grey Systems and Fuzzy Cognitive Maps, each cause is measured by its grey intensity. An improved construction process of FGCMs is presented in this study, proposing an intensity value to assign the vibration of the grey causal influence, thus to handle the trust of the causal influence on the effect variable initially prescribed by experts’ suggestions. The explored methodology is implemented in a well-known medical decision making problem pertaining to the problem of radiotherapy treatment planning selection, where the FCMs have previously proved their usefulness in decision support. Through the examined medical problem, the FGCMs demonstrate their functioning and dynamic capabilities to approximate better human decision making.
This chapter discusses the process of designing and building an agent-based model, and suggests a set of steps to follow
when using agent-based modelling as a research method. It starts with defining agent-based modelling and discusses its main
concepts, and then it discusses how to design agents using different architectures. The chapter also suggests a standardized
process consisting of a sequence of steps to develop agent-based models for social science research, and provides examples
to illustrate this process.
Agent-based modelling has become an important tool to investigate socio-ecological processes. Its use is partially driven by increasing demand from decision makers to provide support for understanding the potential implications of decisions in complex situations. While one of the advantages of agent-based modelling is the ability to simulate the implications of human decision-making processes explicitly, methods for providing empirical support for the representation of the behaviour of human agents have not been structured systematically. This paper develops a framework for the parameterisation of human behaviour in agent-based models and develops twelve distinct sequences for the characterisation and parameterisation of human behaviours. Examples are provided to illustrate the most important sequences. This framework is a first step towards a guide for parameterisation of human behaviour in ABM. A structured discussion within the agent-based community is needed to achieve a more definitive guideline.
An innovative and flexible model based on Grey Systems Theory and Fuzzy Cognitive Maps called Fuzzy Grey Cognitive Maps (FGCM) is proposed. It can be adapted to a wide range of problems, specially in multiple meaning-based environments. FGCMs offer some advantages in comparison with others similar tools. First, the FGCM model is defined specifically for multiple meanings (grey) environments. Second, the FGCM technique allows the defining of relationships between concepts. Through this characteristic, decisional models that are more reliable for interrelated environments are defined. Third, the FGCM technique is able to quantify the grey influence of the relationships between concepts. Through this attribute, a better support in grey environments can be reached. Finally, with this FGCM model it is possible to develop a what-if analysis with the purpose of describing possible grey scenarios. IT projects risks are modelled to illustrate the proposed technique.
The characterization of uncertainty is critical in cost-effectiveness analysis, particularly when considering whether additional evidence is needed. In addition to parameter and methodological uncertainty, there are other sources of uncertainty which include simplifications and scientific judgments that have to be made when constructing and interpreting a model of any sort. These have been classified in a number of different ways but can be referred to collectively as structural uncertainties.
Separate reviews were undertaken to identify what forms these other sources of uncertainty take and what other forms of potential methods to explicitly characterize these types of uncertainties in decision analytic models. These methods were demonstrated through application to four decision models each representing one of the four types of uncertainty.
These sources of uncertainty fall into four general themes: 1) inclusion of relevant comparators; 2) inclusion of relevant events; 3) alternative statistical estimation methods; and 4) clinical uncertainty.Two methods to explicitly characterize such uncertainties were identified: model selection and model averaging. In addition, an alternative approach, adding uncertain parameters to represent the source of uncertainty was also considered.The applications demonstrate that cost-effectiveness may be sensitive to these uncertainties and the methods used to characterize them. The value of research was particularly sensitive to these uncertainties and the methods used to characterize it. It is therefore important, for decision-making purposes, to incorporate such uncertainties into the modeling process.
Only parameterizing the uncertainty directly in the model can inform the decision to conduct further research to resolve this source of uncertainty.
In both information and public transport infrastructure construction, it is important to build a high-speed backbone tree to connect users distributed in a large area. We study the topic in this paper and model it as the inner-node weighted minimum spanning tree problem (IWMST), which asks for a spanning tree in a graph G = (V,E) (|V| = n, |E| = m) with the minimum total cost for its edges and non-leaf nodes. This problem is NP-hard because it contains the connected dominating set problem (CDS) as a special case. Since CDS cannot be approximated with a factor of (1-epsiv)H(Delta) (Delta is the maximum degree) unless NP sube DTIME[n<sup>O(log</sup> <sup>log</sup> <sup>n)</sup>] (Guha and Khuller, 1996), we can only expect a poly-logarithmic approximation algorithm for the IWMST problem. To tackle this problem, we first present a general framework for developing poly-logarithmic approximation algorithms. Our framework aims to find a k/k-1 ln n-approximate algorithm (k isin N and k ges 2) for the IWMST problem. Based on this framework, we further design a polynomial time approximation algorithms which can find a 2 ln n-approximate solution in O(mn log n) time.
Discovery phase patterns: building the domain model
- S Stepney
- F A Polack
Empirical Agent-Based Modelling-Challenges and Solutions
- A Smajgl
- O Barreteau