No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Modeling the Process by Which People Try to Explain Complex Things to Others
Abstract
The process of explaining something to another person is more than offering a statement. Explaining means taking the perspective and knowledge of the Learner into account and determining whether the Learner is satisfied. While the nature of explanation—conceived of as a set of statements—has been explored philosophically and empirically, the process of explaining, as an activity, has received less attention. We conducted an archival study, looking at 73 cases of explaining. We were particularly interested in cases in which the explanations focused on the workings of complex systems or technologies. The results generated two models: local explaining to address why a device (such an intelligent system) acted in a surprising way, and global explaining about how a device works. The examination of the processes of explaining as it occurs in natural settings revealed a number of mistaken beliefs about how explaining happens, and what constitutes an explanation that encourages learning.
... Explainable AI has sparked interest in the study of how people explain complex systems, for themselves and to others (Hoffman et al., 2017;Klein et al., 2021). There have appeared many discussions of the quality of explanations, possible methodologies for evaluation (Doshi-Velez and Kim, 2017), and attempts to empirically evaluate explanation quality and the effectiveness of explanations (e.g., Hoffman et al., 2011Hoffman et al., , 2023Miller, 2017;Mueller et al., 2019;Johs et al., 2020;Kenny et al., 2021). ...
... Previous research has shown that global explanations are often accompanied by specific cases and that local explanations contain hints that contribute to global understanding (Klein et al., 2021). In other words, people benefit from having global and local explanations that are integrated. ...
... Many studies rely on unnecessarily large samples of Mechanical Turkers, for the sake of achieving statistically significant effects rather than to seek practically significant effects. Recent attempts at evaluation have included some methodological elements that might lend rigor to the effort (see, for instance, Buçinca et al., 2020;Dodge et al., 2021;Klein et al., 2021;Rosenfeld, 2021;Hoffman et al., 2023). ...
Introduction
The purpose of the Stakeholder Playbook is to enable the developers of explainable AI systems to take into account the different ways in which different stakeholders or role-holders need to “look inside” the AI/XAI systems.
Method
We conducted structured cognitive interviews with senior and mid-career professionals who had direct experience either developing or using AI and/or autonomous systems.
Results
The results show that role-holders need access to others (e.g., trusted engineers and trusted vendors) for them to be able to develop satisfying mental models of AI systems. They need to know how it fails and misleads as much as they need to know how it works. Some stakeholders need to develop an understanding that enables them to explain the AI to someone else and not just satisfy their own sense-making requirements. Only about half of our interviewees said they always wanted explanations or even needed better explanations than the ones that were provided. Based on our empirical evidence, we created a “Playbook” that lists explanation desires, explanation challenges, and explanation cautions for a variety of stakeholder groups and roles.
Discussion
This and other findings seem surprising, if not paradoxical, but they can be resolved by acknowledging that different role-holders have differing skill sets and have different sense-making desires. Individuals often serve in multiple roles and, therefore, can have different immediate goals. The goal of the Playbook is to help XAI developers by guiding the development process and creating explanations that support the different roles.
... But self-explanation is often a highly motivated desire to understand. This assertion derives from psychological experimentation [13] and empirical research on how people explain complex systems to other people [51,52]. ...
... This has been considered in psychology, especially in research on curiosity [58]. A model of self-explanation is emerging in XAI based on the psychological investigation of explanatory reasoning [52] and XAI research on levels of intelligibility [56]. ...
... In parallel with the computer science work, empirical analysis was conducted of a large corpus of cases where a person was the recipient of an explanation of how some sort of complex system works [52]. Many of the cases illustrated how global explanations often include local information (e.g., instances that exemplify rules or principles), and local explanations often include some global information (e.g., a generalization over a class of instances). ...
This paper summarizes the psychological insights and related design challenges that have emerged in the field of Explainable AI (XAI). This summary is organized as a set of principles, some of which have recently been instantiated in XAI research. The primary aspects of implementation to which the principles refer are the design and evaluation stages of XAI system development, that is, principles concerning the design of explanations and the design of experiments for evaluating the performance of XAI systems. The principles can serve as guidance, to ensure that AI systems are human-centered and effectively assist people in solving difficult problems.
... Understanding the process of forming explanations Klein et al. (2021) investigated the process of forming explanations-the ways that people react when they encounter unexpected, anomalous, and surprising items of information and how they try to diagnose what happened. We collected a corpus of 73 cases that were a convenience sample. ...
... For this current project on plausibility, we identified a subset of 23 cases of explanations taken from Klein et al. (2021). The criteria for this subset included having richer details about the explainer or learner's reasoning; and coming from sources we judged to be more reliable than our initial sample. ...
... The stopping point occurs when the plausibility gap is reduced, and cognitive strain is minimized. This aspect of the Plausibility Transition model is consistent with the Klein et al. (2021) model of the process of explaining. Both of these models call out plausibility of transitions as a part of determining the stopping point. ...
When people make plausibility judgments about an assertion, an event, or a piece of evidence, they are gauging whether it makes sense that the event could transpire as it did. Therefore, we can treat plausibility judgments as a part of sensemaking. In this paper, we review the research literature, presenting the different ways that plausibility has been defined and measured. Then we describe the naturalistic research that allowed us to model how plausibility judgments are engaged during the sensemaking process. The model is based on an analysis of 23 cases in which people tried to make sense of complex situations. The model describes the user’s attempts to construct a narrative as a state transition string, relying on plausibility judgments for each transition point. The model has implications for measurement and for training.
... Process-based trust describes how the automation operates and corresponds to the appropriateness of the automation's algorithms in achieving the operator's goals. For this component, automation that is understandable will lead to appropriate trust, which can occur through transparent design (Chen et al., 2014;Lyons, 2013) and/ or providing a global explanation for how the technology works (Klein et al., 2021), possibly through training (Cohen et al., 1998). To illustrate, a remote operator is more likely to trust detect-andavoid automation if they understand (via training or transparency) the general logic of that system causes aircraft to circle back around to missed waypoints after resolving a conflict. ...
... Participant responses indicated that ICAROUS produced behaviors and flight characteristics that were not expected (cf. local explaining activities, Klein et al., 2021). Yet this may be due to the questionnaire targeting the collision avoidance functionality exclusively, rather than also targeting the geofence avoidance functionality specifically. ...
Trust development will play a critical role in remote vehicle operations transitioning from automated (e.g., requiring human oversight) to autonomous systems. Factors that affect trust development were collected during a high-fidelity remote uncrewed aerial system (UAS) simulation. Six UAS operators participated in this study, which consisted of 17 trials across two days per participant. Trust in two highly automated systems were measured pre- and post-study. Perceived risk and familiarity with the systems were measured before the study. Main effects showed performance-based trust and purpose-based trust increased between the pre- and post-study measurements. System familiarity predicted process-based trust. An interaction indicated that operators who rated the systems as riskier showed an increase in a single-item trust scale between the pre- and post-study measurement, whereas participants that rated the systems as less risky maintained a higher trust rating. Individual differences showed operators adapted to why the automation was being used, and trust improved between measurements. Qualitative analysis of open-ended responses revealed themes related to behavioral responses of the aircraft and transparency issues with the automated systems. Results can be used to support training interventions and design recommendations for appropriate trust in increasingly autonomous remote operations, as well as guide future research.
... This model was a useful starting point for developing experimental designs and procedures for empirical evaluation, especially since it showed some of the things that would have to be measured (the dark nodes in Figure 1). However, the model does not take into account the findings from psychological research on how people explain complex systems to other people (see Hoffman, Klein, & Miller, 2011;Klein, Hoffman, et al., 2021). Nor does it take into account the challenges that were discovered in the attempts, dating to the 1980s, to create intelligent tutoring systems (ITSs; see Clancey & Hoffman, 2022). ...
... The key point is that people who are involved in an interaction with an XAI system are abducting to understand what, how, and why the AI does what it does, and not just make sense of the world that is being observed or controlled (e.g., forecasting the weather is guided by the outputs of computational models). The process of sensemaking, or self-explaining a complex system, is deliberative and effortful (Chi, Roy, & Hausmann, 2008;Chi, Siler, Jewong, Yamauchi, & Hausmann, 2001;Klein et al., 2019;Klein, Hoffman, et al., 2021;Renkl & Eitel, 2019). The process of explaining often takes the form of a dialog in which an explainer and a learner collaborate, explore, and co-adapt (Clancey, 1987;Walton, 2011). ...
A challenge in building useful artificial intelligence (AI) systems is that people need to understand how they work in order to achieve appropriate trust and reliance. This has become a topic of considerable interest, manifested as a surge of research on Explainable AI (XAI). Much of the research assumes a model in which the AI automatically generates an explanation and presents it to the user, whose understanding of the explanation leads to better performance. Psychological research on explanatory reasoning shows that this is a limited model. The design of XAI systems must be fully informed by a model of cognition and a model of pedagogy, based on empirical evidence of what happens when people try to explain complex systems to other people and what happens as people try to reason out how a complex system works. In this article we discuss how and why C. S. Peirce's notion of abduction is a best model for XAI. Peirce's notion of abduction as an exploratory activity can be regarded as supported by virtue of its concordance with models of expert reasoning that have been developed by modern applied cognitive psychologists.
... While some may argue that this highly inclusive dataset may dilute the findings, we argue that it is a strength of this study, providing broader understanding of how trust in AI is gained or lost across the broader intelligence cycle. As noted by Klein, et al. (2021), such a "wide net" approach with minimal inclusion criteria is appropriate when data collection is opportunistic (in this case participants with unclassified stories were difficult to come by) and when the objective of the study is more exploratory in nature (e.g. not a more formal meta-analysis). ...
... not a more formal meta-analysis). This wide net and naturalistic approach has enabled greater understanding of phenomena including decision making, insight, and explanations Klein & Jarosz, 2011;Klein, et al., 2021); so it stands to reason that it is also sufficient for investigating trust. ...
Artificial Intelligence (AI) is often viewed as the means by which the intelligence community will cope with increasing amounts of data. There are challenges in adoption, however, as outputs of such systems may be difficult to trust, for a variety of factors. We conducted a naturalistic study using the Critical Incident Technique (CIT) to identify which factors were present in incidents where trust in an AI technology used in intelligence work (i.e., the collection, processing, analysis, and dissemination of intelligence) was gained or lost. We found that explainability and performance of the AI were the most prominent factors in responses; however, several other factors affected the development of trust. Further, most incidents involved two or more trust factors, demonstrating that trust is a multifaceted phenomenon. We also conducted a broader thematic analysis to identify other trends in the data. We found that trust in AI is often affected by the interaction of other people with the AI (i.e., people who develop it or use its outputs), and that involving end users in the development of the AI also affects trust. We provide an overview of key findings, practical implications for design, and possible future areas for research.
... Patterns of cues in the environment are matched, as near as possible, to patterns of cues that are resident in memory. This initiates a comparative process to determine whether the associated explanation matches the situation (Klein et al., 2021). Where a match is established, a response is initiated. ...
Background
Ensuring that pool lifeguards develop the skills necessary to detect drowning victims is challenging given that these situations are relatively rare, unpredictable and are difficult to simulate accurately and safely. Virtual reality potentially provides a safe and ecologically valid approach to training since it offers a near-to-real visual experience, together with the opportunity to practice task-related skills and receive feedback. As a prelude to the development of a training intervention, the aim of this research was to establish the construct validity of virtual reality drowning detection tasks.
Method
Using a repeated measures design, a total of 38 qualified lifeguards and 33 non-lifeguards completed 13 min and 23 min simulated drowning detection tasks that were intended to reflect different levels of sustained attention. During the simulated tasks, participants were asked to monitor a virtual pool and identify any drowning targets with accuracy, response latency, and dwell time recorded.
Results
During the simulated scenarios, pool lifeguards detected drowning targets more frequently and spent less time than non-lifeguards fixating on the drowning target prior to the drowning onset. No significant differences in response latency were evident between lifeguards and non-lifeguards nor for first fixations on the drowning target.
Conclusion
The results provide support for the construct validity of virtual reality lifeguarding scenarios, thereby providing the basis for their development and introduction as a potential training approach for developing and maintaining performance in lifeguarding and drowning detection.
Application
This research provides support for the construct validity of virtual reality simulations as a potential training tool, enabling improvements in the fidelity of training solutions to improve pool lifeguard competency in drowning detection.
... In particular, the building volume is large, the investment is large, and the energy consumption is high. The level of building intelligent engineering largely determines the realization of the overall goal of the building [5]. ...
This paper takes the intelligent engineering as an example and uses the Work Breakdown Structure (WBS) tool to decompose the intelligent engineering into specific subsystems and describe each subsystem. In the field of intelligent engineering construction, it is impossible to ensure that the construction project will not be changed. To reduce scope changes, efforts can be made in three aspects: first, to accurately locate project requirements and find enough work to be done; second, to scientifically define the scope of the project and delete unnecessary work; and third, to effectively control scope changes. Ensure that all work is done to achieve the project goals. This paper starts from the actual project management experience, conducts a comprehensive summary and refinement, proposes key strategies and methods for project scope management, and puts forward the following points of view. Intelligent design should be completed by professional design agencies, and intelligent construction should be completed by professional construction teams. There should be standards for intelligent acceptance to follow. Hope this article will be helpful for project managers.
... An explanation may be critical in determining whether the patient comes to understand the re-diagnosis process as the best approach, or attribute it to a mistake or incompetence, and perhaps stop treatment or seek another provider. Fourth, re-diagnosis situations like these are representative of many of the functions of sensemaking described by Klein et al. (2007) (which inspired Klein et al., (2021)'s model of explanation), and we hoped these situations will provide good opportunities for physicians to explain their reasoning. ...
AI systems are increasingly being developed to provide the first point of contact for patients. These systems are typically focused on question-answering and integrating chat systems with diagnostic algorithms, but are likely to suffer from many of the same deficiencies in explanation that have plagued medical diagnostic systems since the 1970s ( Shortliffe, 1979 ). To provide better guidance about how such systems should approach explanations, we report on an interview study in which we identified explanations that physicians used in the context of re-diagnosis or a change in diagnosis. Seven current and former physicians with a variety of specialties and experience were recruited to take part in the interviews. Several high-level observations were made by reviewing the interview notes. Nine broad categories of explanation emerged from the thematic analysis of the explanation contents. We also present these in a diagnosis meta-timeline that encapsulates many of the commonalities we saw across diagnoses during the interviews. Based on the results, we provided some design recommendations to consider for developing diagnostic AI systems. Altogether, this study suggests explanation strategies, approaches, and methods that might be used by medical diagnostic AI systems to improve user trust and satisfaction with these systems.
As Artificial Intelligence (AI) systems increase in capability, so there are growing concerns over the ways in which the recommendations they provide can affect people's everyday life and decisions. The field of Explainable AI (XAI) aims to address such concerns but there is often a neglect of the human in this process. We present a formal definition of human-centred XAI and illustrate the application of this formalism to the design of a user interface. The user interface supports users in indicating their preferences relevant to a situation and to compare their preferences with those of a computer recommendation system. A user trial is conducted to evaluate the resulting user interface. From the user trial, we believe that users are able to appreciate how their preferences can influence computer recommendations, and how these might contrast with the preferences used by the computer. We provide guidelines of implementing human-centred XAI.
• Objectifs L’explicabilité de l’intelligence artificielle représente un défi technique et organisationnel croissant pour le marketing, soulevant des enjeux éthiques, légaux, mais aussi de performance. Pour le relever, un champ interdisciplinaire (XAI pour eXplainable AI) se développe pour créer et diffuser des outils dédiés à la compréhension des IA, mais la littérature alerte sur les difficultés à les mettre en pratique. • Méthodologie Cet article s’appuie sur deux cas d’étude : deux entreprises (une européenne et une américaine) en pointe sur le développement de solutions d’optimisation de la publicité en ligne. • Résultats Nos résultats montrent que si la nécessité de renforcer les capacités à expliquer les IA est bien reconnue sur un horizon long, deux problèmes peuvent limiter à court terme leur développement : la cristallisation des indicateurs et référentiels de calculs (bases d’apprentissage et d’évaluation notamment) associés au suivi de la performance d’une part, et d’autre part la confusion entre évaluation de la performance prédictive des modèles d’IA et celle de la performance du système marketing dans son ensemble. • Implications managériales Nous discutons de ces écueils au regard notamment du rythme auquel l’XAI pourrait se standardiser en marketing. Ces résultats alertent sur une possible déconnexion de la pratique du marketeur avec certaines dimensions stratégiques du métier. Nous proposons des pistes pour y remédier, notamment la reconnaissance et l’identification de nouvelles expertises en XAI dans le marketing et la création d’indicateurs propres à l’XAI. • Originalité Cette étude vient proactivement questionner une des problématiques centrales de l’IA pour le marketing, qui pourrait freiner son développement. Elle ouvre de nouvelles perspectives sur la dimension managériale de l’XAI et invite à considérer l’émergence de nouvelles expertises au sein du marketing, qui permettraient de recentrer le marketeur au cœur du processus décisionnel.
This is the third in a series of essays about explanation. After laying out the core theoretical concepts in the first article, including aspects of causation and abduction, the second article presented some empirical research to reveal the great variety of purposes and types of causal reasoning, as well as a number of different causal explanation patterns. Taking the notion of reasoning patterns a step further, the author describes a method whereby a decision maker can go from a causal explanation to a viable course of action for making positive change in the future, not to mention aiding decision making in general.
This chapter focuses on reasoning about the causal explanation of events and human activities that are indeterminate and complex. We first consider some classical ideas about physical causation from David Hume and John Stuart Mill, who had significant impact on the psychology of reasoning, and we find in their writings some notions that carry over into modern analyses of cause-effect relations and causal reasoning. We then present the macrocognition paradigm, which is empirically grounded in studies of naturalistic decision making, and approaches the analysis of causal reasoning through investigation of the functional purposes of various forms of causal reasoning. This sets the stage for presentation of our research on how people interpret the causal reasoning that is presented in newspaper articles. Findings include new patterns and models of causal reasoning. We conclude with a discussion of some "myths" about causation that characterize theories but that do not match well to our research findings.
Although insight is often invoked as a phenomenon of problem solving and innovation, it has rarely been studied in a naturalistic fashion. The purpose of the study reported here was to learn more about insights as they occur in field settings as opposed to controlled laboratory conditions. The authors collected a set of 120 examples of insight taken from cognitive task analysis interviews, media accounts, and other sources and coded each incident using a set of 14 features. The results generated a descriptive model of insight that is different from the findings that emerge from research with puzzle problems. It posits multiple pathways for gaining insights. One pathway is triggered by detecting a contradiction. A second pathway is triggered by a need to break through an impasse. The third pathway gets triggered by seeing a connection.
[This is an edited version of the original, unpublished 1985 study that identified recognition-primed decision making, with a new commentary added.] The objective of this study was to examine the way in which decisions are made by highly proficient personnel, under conditions of extreme time pressure, and in environments where the consequences of the decisions could affect lives and property. The domain of fire-fighting was selected, and the research focused on the decisions made by fire ground commanders (FGCs). Interviews were conducted with 26 experienced FGCs (mean experience of 23 years). Each interview covered a critical incident that was nonroutine and that demanded expertise. A total of 156 decision points were probed in this way. In less than 12% of them was there any evidence of simultaneous comparisons and relative evaluation of two or more options. In over 80% of the decision points, the strategy was for the FGCs to use their experience to directly identify the situation as typical of a standard prototype and to identify a course of action as typical for that prototype. In this way, the FGCs handled decision points without any need to consider more than one option. A recognition-primed decision (RPD) model was synthesized from these data, which emphasized the use of recognition rather than calculation or analysis for rapid decision making.
We present the lessons drawn from a review of the main concepts put forward by the designers of decision support systems, joint cognitive systems and cooperative systems. A striking observation is that these systems stumble on user-system cooperation. The main idea of this paper is that interactive system must behave more as an intelligent assistant than as an expert. Key elements of an effective cooperation between a user and a system are making explicit cooperation context and extending consequently the notions of explanations, knowledge acquisition and machine learning.
Generalizing the concepts of joint activity developed by Clark (1996), we describe key aspects of team coordination. Joint activity depends on interpredictability of the participants' attitudes and actions. Such interpredictability is based on common ground-pertinent knowledge, beliefs, and assumptions that are shared among the involved parties. Joint activity assumes a basic compact, which is an agreement (often tacit) to facilitate coordination and prevent its breakdown. One aspect of the Basic Compact is the commitment to some degree of aligning multiple goals. A second aspect is that all parties are expected to bear their portion of the responsibility to establish and sustain common ground and to repair it as needed. We apply our understanding of these features of joint activity to account for issues in the design of automation. Research in software and robotic agents seeks to understand and satisfy requirements for the basic aspects of joint activity. Given the widespread demand for increasing the effectiveness of team play for complex systems that work closely and collaboratively with people, observed shortfalls in these current research efforts are ripe for further exploration and study.
What is needed for good explanation? This paper begins by considering some desiderata for expert system explanation. These desiderata concern not only the form and content of the explanations, but also the impact of explanation generation on the expert system itself— how it is built and how it performs. In this paper, we use these desiderata as a yardstick for measuring progress in the field. The paper describes two major developments that have differentiated explanation in second generation systems from explanation in first generation systems: 1) new architectures have been developed that capture more of the knowledge that is needed for explanation, and 2) more powerful explanation generators have been developed in which explanation generation is viewed as a problem-solving activity in its own right. These developments have led to significant improvements in explanation facilities: the explanations they offer are richer and more coherent, they are better adapted to the user’s needs and knowledge, and the explanation facilities can offer clarifying explanations to correct misunderstandings.
Production rules are a popular representation for encoding heuristic knowledge in programs for scientific and medical problem solving. However, experience with one of these programs, mycin, indicates that the representation has serious limitations: people other than the original rule authors find it difficult to modify the rule set, and the rules are unsuitable for use in other settings, such as for application to teaching. These problems are rooted in fundamental limitations in mycin's original rule representation: the view that expert knowledge can be encoded as a uniform, weakly structured set of if/then associations is found to be wanting.To illustrate these problems, this paper examines mycin's rules from the perspective of a teacher trying to justify them and to convey a problem-solving approach. We discover that individual rules play different roles, have different kinds of justifications, and are constructed using different rationales for the ordering and choice of premise clauses. This design knowledge, consisting of structural and strategic concepts which lie outside the representation, is shown to be procedurally embedded in the rules. Moreover, because the data/hypothesis associations are themselves a proceduralized form of underlying disease models, they can only be supported by appealing to this deeper level of knowledge. Making explicit this structural, strategic and support knowledge enhances the ability to understand and modify the system.
We present an overview of different theories of explanation from the phi- losophy and cognitive science communities. Based on these theories, as well as models of explanation from the knowledge-based systems area, we present a framework for explanation in case-based reasoning (CBR) based on explanation goals. We propose ways that the goals of the user and system designer should be taken into account when deciding what is a good explanation for a given CBR system. Some general types of goals relevant to many CBR systems are identified, and used to survey existing meth- ods of explanation in CBR. Finally, we identify some future challenges.
For pt.1 see ibid., vol.21, no.4, p. 70-73 (2006). In this paper, we have laid out a theory of sensemaking that might be useful for intelligent systems applications. It's a general, empirically grounded account of sensemaking that goes significantly beyond the myths and puts forward some nonobvious, testable hypotheses about the process. When people try to make sense of events, they begin with some perspective, viewpoint, or framework - however minimal. For now, let's use a metaphor and call this a frame. We can express frames in various meaningful forms, including stories, maps, organizational diagrams, or scripts, and can use them in subsequent and parallel processes. Even though frames define what count as data, they themselves actually shape the data Furthermore, frames change as we acquire data. In other words, this is a two-way street: Frames shape and define the relevant data, and data mandate that frames change in nontrivial ways. We examine five areas of empirical findings: causal reasoning, commitment to hypotheses, feedback and learning, sense-making as a skill, and confirmation bias. In each area the Data/Frame model, and the research it's based on, doesn't align with common beliefs. For that reason, the Data/Frame model cannot be considered a depiction of commonsense views
An explanation facility is an important component of an expert system, but current systems for the most part have neglected the importance of tailoring a system's explanations to the user. This paper explores the role of user modeling in generating expert system explanations, making the claim that individualized user models are essential to produce good explanations when the system users vary in their knowledge of the domain, or in their goals, plans, and preferences. To make this argument, a characterization of explanation, and good explanation is made, leading to a presentation of how knowledge about the user affects the various aspects of a good explanation. Individualized user models are not only important, it is practical to obtain them. A method for acquiring a model of the user's beliefs implicitly by "eavesdropping" on the interaction between user and system is presented, along with examples of how this information can be used to tailor an explanation.
Scientific and ‘intuitive’ or ‘folk’ theories are typically characterized as serving three critical functions: prediction, explanation, and control. While prediction and control have clear instrumental value, the value of explanation is less transparent. This paper reviews an emerging body of research from the cognitive sciences suggesting that the process of seeking, generating, and evaluating explanations in fact contributes to future prediction and control, albeit indirectly by facilitating the discovery and confirmation of instrumentally valuable theories. Theoretical and empirical considerations also suggest why explanations may nonetheless feel intrinsically valuable. The paper concludes by considering some implications of the psychology of explanation for a naturalized philosophy of explanation.
Reasoning occupies a central position in studies of both artificial and natural intel-ligence. This paper presents a precise and computationally effective model of the structure of human explanation, based upon the careful analysis of transcipts of actual, naturally occurring conversations between human beings involved in explanation. Explanations are represented by trees whose internal nodes correspond to the three major types of justification which we have found offered for assertions (giving a reason, giving examples, and eliminating alternatives); the real-time process of explanation production is represented by a sequence of transformations on such trees. Focus of attention in explanation is represented by pointers in the tree, and shifts of focus of attention by pointer movement. The ordering and embedding of explanations are considered; we explain why some orderings are easier to understand than others, and we demonstrate that some forms of explanation require multiple pointers, some of which are embedded in others. A fairly complex example is analyzed in detail. The paper emphasizes the view that explanation is a social process, i.e. occurs in particular contexts involving particular people who have particular assumptions and predisposi-tions which significantly influence what actually occurs. Implications of the results of this paper to the design of systems for explanation production and for artificial reasoning are discussed.
Both philosophers and psychologists have argued for the existence of distinct kinds of explanations, including teleological explanations that cite functions or goals, and mechanistic explanations that cite causal mechanisms. Theories of causation, in contrast, have generally been unitary, with dominant theories focusing either on counterfactual dependence or on physical connections. This paper argues that both approaches to causation are psychologically real, with different modes of explanation promoting judgments more or less consistent with each approach. Two sets of experiments isolate the contributions of counterfactual dependence and physical connections in causal ascriptions involving events with people, artifacts, or biological traits, and manipulate whether the events are construed teleologically or mechanistically. The findings suggest that when events are construed teleologically, causal ascriptions are sensitive to counterfactual dependence and relatively insensitive to the presence of physical connections, but when events are construed mechanistically, causal ascriptions are sensitive to both counterfactual dependence and physical connections. The conclusion introduces an account of causation, an "exportable dependence theory," that provides a way to understand the contributions of physical connections and teleology in terms of the functions of causal ascriptions.
Using grounded theory as an example, this paper examines three methodological questions that are generally applicable to all qualitative methods. How should the usual scientific canons be reinterpreted for qualitative research? How should researchers report the procedures and canons used in their research? What evaluative criteria should be used in judging the research products? We propose that the criteria should be adapted to fit the procedures of the method. We demonstrate how this can be done for grounded theory and suggest criteria for evaluating studies following this approach. We argue that other qualitative researchers might be similarly specific about their procedures and evaluative criteria.
Generating and evaluating explanations is spontaneous, ubiquitous and fundamental to our sense of understanding. Recent evidence suggests that in the course of an individual's reasoning, engaging in explanation can have profound effects on the probability assigned to causal claims, on how properties are generalized and on learning. These effects follow from two properties of the structure of explanations: explanations accommodate novel information in the context of prior beliefs, and do so in a way that fosters generalization. The study of explanation thus promises to shed light on core cognitive issues, such as learning, induction and conceptual representation. Moreover, the influence of explanation on learning and inference presents a challenge to theories that neglect the roles of prior knowledge and explanation-based reasoning.
Context needs in cooperative building of explanations [Conference session
- P Brézillon
Brézillon, P. (1994, September). Context needs in cooperative
building of explanations [Conference session]. First European
Conference on Cognitive Science in Industry, pp. 443-450.
An operational account of mental models: A pilot study
- J Borders
- G Klein
- R Besuijen
Borders, J., Klein, G., & Besuijen, R. (2019). An operational
account of mental models: A pilot study [Conference session].
The book of why: The new science of cause and effect
- J Pearl
Pearl, J. (2018). The book of why: The new science of cause and
effect. Basic Books.
Smart machines in education
- K D Forbus
- Feltovich
Forbus, K. D., & Feltovich, P. J. (Eds.). (2001). Smart machines in
education. AAAI Press.
Through an Arab cultural lens
- H A Klein
- G Kuperman
Klein, H. A., & Kuperman, G. (2008). Through an Arab cultural lens.
Military Review, 88(3), 100.
Understanding Air France 447
- B Palmer
Palmer, B. (2013). Understanding Air France 447. William Palmer,
Jr.
“Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI.” Report on Award No. FA8650-17-2-7711, DARPA XAI Program. DTIC accession number AD1073994. ArXiv:190201876
- S T Mueller
- R R Hoffman
- W Clancey
- A Emrey
- G Klein
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
- S T Mueller
- R R Hoffman
- W Clancey
- A Emrey
- G Klein
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G.
(2018). "Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and
Bibliography for Explainable AI." Report on Award No. FA8650-17-2-7711, DARPA XAI Program. DTIC accession number
AD1073994. ArXiv:190201876.