Preprint

The Plausibility Gap: A model of sensemaking

Authors:
  • ShadowBox LLC & MacroCognition LLC
Preprints and early-stage research may not have been peer reviewed yet.
To read the file of this research, you can request a copy directly from the authors.

Abstract

When people make plausibility judgments about an assertion, an event, or a piece of evidence, they are gauging whether it makes sense. Therefore, we can treat plausibility judgments as sensemaking activities. In this paper, we review the research literature, presenting the different ways that plausibility has been defined and measured. Then we describe the research program that allowed us to formulate our sensemaking perspective on plausibility. The model is based on an analysis of 23 cases, most of which involved understanding and interacting with information technology. The resulting model describes the user’s attempts to construct a narrative as a state transition string, relying on plausibility judgments.

No file available

Request Full-text Paper PDF

To read the file of this research,
you can request a copy directly from the authors.

Article
Full-text available
Readers confronted with unfamiliar and controversial scientific debates tend to rely on simple heuristics such as the perceived plausibility to focus their cognitive resources on specific information during comprehension. In the present experiment, we tested the assumption that plausibility judgments as an integral part of comprehension are used as a simple heuristic to distribute cognitive resources to controversial texts, leading to a better comprehension of information judged as plausible. To experimentally vary perceived plausibility, participants (N = 54 university students) watched one of two video versions on the controversy of spider silk. The videos provided identical factual information but took opposing argumentative claims on the issue (pro vs. con). Afterwards, participants read two conflicting texts (pro vs. con) on the same issue. Plausibility judgments and comprehension for the texts were assessed. In line with the hypothesized mediation model, results revealed that the belief manipulation (i.e., the video versions) affected the perceived plausibility of the controversial texts, which in turn influenced the comprehension of the two texts. The effect of the belief-manipulation, that is, participants' better comprehension of the text that took the same argumentative stance as the video, was fully mediated by perceived plausibility. These results are relevant for educational interventions to improve the comprehension of controversial but unfamiliar scientific studies and for theories on the role of plausibility in (multiple) text comprehension.
Article
Full-text available
This chapter focuses on reasoning about the causal explanation of events and human activities that are indeterminate and complex. We first consider some classical ideas about physical causation from David Hume and John Stuart Mill, who had significant impact on the psychology of reasoning, and we find in their writings some notions that carry over into modern analyses of cause-effect relations and causal reasoning. We then present the macrocognition paradigm, which is empirically grounded in studies of naturalistic decision making, and approaches the analysis of causal reasoning through investigation of the functional purposes of various forms of causal reasoning. This sets the stage for presentation of our research on how people interpret the causal reasoning that is presented in newspaper articles. Findings include new patterns and models of causal reasoning. We conclude with a discussion of some "myths" about causation that characterize theories but that do not match well to our research findings.
Article
Full-text available
Plausibility is judged on a daily basis in a wide range of cognitive phenomena, yet the study of plausibility in its own right has been long neglected in cognitive science. In this paper, we present the Knowledge-Fitting Theory of Plausibility that incorporates both concept-coherence (i.e. the conceptual structure and relatedness of a scenario) and word-coherence (i.e. the distributional properties of the individual words used to describe a scenario) in plausibility judgement. We also present the Plausibility Analysis Model (PAM), which is an implementation of this theory and the first computational mode to specifically address the issue of human plausibility judgements.
Chapter
Full-text available
The naturalistic approach to training decision making takes as its starting point the way real decision makers make decisions. Unlike normative approaches that are based on decision theory or other formal models, naturalistic training is in no rush to replace ordinary decision processes with methods that may appear to have theoretical value, but which are qualitatively different from the methods that proficient decision makers use. If a house is basically sound, it can be better to renovate than to tear down and rebuild from scratch.
Article
Full-text available
In some theories of sentence comprehension, linguistically relevant lexical knowledge, such as selectional restrictions, is privileged in terms of the time-course of its access and influence. We examined whether event knowledge computed by combining multiple concepts can rapidly influence language understanding even in the absence of selectional restriction violations. Specifically, we investigated whether instruments can combine with actions to influence comprehension of ensuing patients of (as in Rayner, Warren, Juhuasz, & Liversedge, 2004; Warren & McConnell, 2007). Instrument-verb-patient triplets were created in a norming study designed to tap directly into event knowledge. In self-paced reading (Experiment 1), participants were faster to read patient nouns, such as hair, when they were typical of the instrument-action pair (Donna used the shampoo to wash vs. the hose to wash). Experiment 2 showed that these results were not due to direct instrument-patient relations. Experiment 3 replicated Experiment 1 using eyetracking, with effects of event typicality observed in first fixation and gaze durations on the patient noun. This research demonstrates that conceptual event-based expectations are computed and used rapidly and dynamically during on-line language comprehension. We discuss relationships among plausibility and predictability, as well as their implications. We conclude that selectional restrictions may be best considered as event-based conceptual knowledge rather than lexical-grammatical knowledge.
Article
Full-text available
This article reports on an effort to explore the differences between two approaches to intuition and expertise that are often viewed as conflicting: heuristics and biases (HB) and naturalistic decision making (NDM). Starting from the obvious fact that professional intuition is sometimes marvelous and sometimes flawed, the authors attempt to map the boundary conditions that separate true intuitive skill from overconfident and biased impressions. They conclude that evaluating the likely quality of an intuitive judgment requires an assessment of the predictability of the environment in which the judgment is made and of the individual's opportunity to learn the regularities of that environment. Subjective experience is not a reliable indicator of judgment accuracy.
Article
Full-text available
Our goal was to investigate the basis of human plausibility judgements. Previous research had suggested that plausibility is affected by two factors: concept coherence (the inferences made between parts of a discourse) and word coherence (the distributional properties of the words used). In two experiments, participants were asked to rate the plausibility of sentence pairs describing events. In the first, we manipulated concept coherence by using different inference types to link the sentences in a pair (e.g., causal or temporal). In the second, we manipulated word coherence by using latent semantic analysis, so two sentence pairs describing the same event had different distributional properties. The results showed that inference type affects plausibility; sentence pairs linked by causal inferences were rated highest, followed by attributal, temporal, and unrelated inferences. The distributional manipulations had no reliable effect on plausibility ratings. We conclude that the processes involved in rating plausibility are based on evaluating concept coherence, not word coherence.
Article
Full-text available
For pt.1 see ibid., vol.21, no.4, p. 70-73 (2006). In this paper, we have laid out a theory of sensemaking that might be useful for intelligent systems applications. It's a general, empirically grounded account of sensemaking that goes significantly beyond the myths and puts forward some nonobvious, testable hypotheses about the process. When people try to make sense of events, they begin with some perspective, viewpoint, or framework - however minimal. For now, let's use a metaphor and call this a frame. We can express frames in various meaningful forms, including stories, maps, organizational diagrams, or scripts, and can use them in subsequent and parallel processes. Even though frames define what count as data, they themselves actually shape the data Furthermore, frames change as we acquire data. In other words, this is a two-way street: Frames shape and define the relevant data, and data mandate that frames change in nontrivial ways. We examine five areas of empirical findings: causal reasoning, commitment to hypotheses, feedback and learning, sense-making as a skill, and confirmation bias. In each area the Data/Frame model, and the research it's based on, doesn't align with common beliefs. For that reason, the Data/Frame model cannot be considered a depiction of commonsense views
Article
Full-text available
Sensemaking, which can be defined as the way people make sense out of their experience in the world, is a motivated, continuous effort to understand connections, which can be among people, places, and events, to anticipate their trajectories and act effectively. Sensemaking serves several functions such as it satisfies a need or drive to comprehend, or guides the exploration of information. Sensemaking helps to test and improve the plausibility of person's explanations and explain apparent anomalies. Sensemaking, which has become an umbrella term for efforts at building intelligent systems provides an empirical base that anchors the theoretical ruminations in concrete examples and findings. These findings serve as a rationale for questioning some assumptions that underlie the drive to make intelligent sensemaking systems.
Article
When individuals have questions about scientific issues, they often search the Internet. Evaluating sources of information and claims they find has become more difficult in the post-truth era. Students are often taught source evaluation techniques, but the proliferation of “fake news” has resulted in a misinformation arms race. As searchers get more sophisticated identifying misleading information, so do purveyors of information who intend to mislead. We draw on a theoretical model of plausibility judgments and current theory and research in source evaluation to suggest that the post-truth era elevates the need for critical evaluation of online information about scientific issues. We argue that explicitly reappraising plausibility judgments may be a crucial addition to evaluating the connections between sources of information and knowledge claims. Individuals who search for and read a scientific article online should ask themselves: Is this explanation plausible, and how do I know?
Article
Students are often asked to make evaluations of scientific explanations, which may involve judgments about the plausibility of competing alternatives. We examined undergraduate students' critical evaluations and plausibility perceptions of climate change when reading two different types of text: expository and refutation. Our findings show that greater critical evaluation and higher plausibility related to more knowledge after reading, but only with the refutation text. Furthermore, we found that greater plausibility had a stronger effect on knowledge after reading the refutation text, whereas greater background knowledge had a stronger effect on knowledge after reading the expository text. We also examined changes in students' plausibility and knowledge. There was significant changes in plausibility and knowledge with the refutation text, but no significant change in either variable with the expository text. These results suggest that evaluations and judgments about plausibility may be factors contributing to the refutation effect and knowledge reconstruction.
Article
Plausibility judgments rarely have been addressed empirically in conceptual change research. Recent research, however, suggests that these judgments may be pivotal to conceptual change about certain topics where a gap exists between what scientists and laypersons find plausible. Based on a philosophical and empirical foundation, this article presents a theoretical model of the role of plausibility judgments in conceptual change (PJCC) with implications for epistemic cognition. The PJCC moves beyond the limitations of cold cognitive processing by incorporating the warmer constructs of affect, motivation, and motivated reasoning. We review recent research in plausibility judgments that informed the PJCC's development and discuss components of the model. Of particular note is the importance of using explicit critical evaluation to reappraise plausibility judgments that may have been originally made through implicit cognitive processes. We also suggest potentially productive areas of future research based on the PJCC model.
Article
Many decisions are based on beliefs concerning the likelihood of uncertain events such as the outcome of an election, the guilt of a defendant, or the future value of the dollar. Occasionally, beliefs concerning uncertain events are expressed in numerical form as odds or subjective probabilities. In general, the heuristics are quite useful, but sometimes they lead to severe and systematic errors. The subjective assessment of probability resembles the subjective assessment of physical quantities such as distance or size. These judgments are all based on data of limited validity, which are processed according to heuristic rules. However, the reliance on this rule leads to systematic errors in the estimation of distance. This chapter describes three heuristics that are employed in making judgments under uncertainty. The first is representativeness, which is usually employed when people are asked to judge the probability that an object or event belongs to a class or event. The second is the availability of instances or scenarios, which is often employed when people are asked to assess the frequency of a class or the plausibility of a particular development, and the third is adjustment from an anchor, which is usually employed in numerical prediction when a relevant value is available.
Article
We discuss the cognitive and the psy- chophysical determinants of choice in risky and risk- less contexts. The psychophysics of value induce risk aversion in the domain of gains and risk seeking in the domain of losses. The psychophysics of chance induce overweighting of sure things and of improbable events, relative to events of moderate probability. De- cision problems can be described or framed in multiple ways that give rise to different preferences, contrary to the invariance criterion of rational choice. The pro- cess of mental accounting, in which people organize the outcomes of transactions, explains some anomalies of consumer behavior. In particular, the acceptability of an option can depend on whether a negative outcome is evaluated as a cost or as an uncompensated loss. The relation between decision values and experience values is discussed. Making decisions is like speaking prose—people do it all the time, knowingly or unknowingly. It is hardly surprising, then, that the topic of decision making is shared by many disciplines, from mathematics and statistics, through economics and political science, to sociology and psychology. The study of decisions ad- dresses both normative and descriptive questions. The normative analysis is concerned with the nature of rationality and the logic of decision making. The de- scriptive analysis, in contrast, is concerned with peo- ple's beliefs and preferences as they are, not as they should be. The tension between normative and de- scriptive considerations characterizes much of the study of judgment and choice. Analyses of decision making commonly distin- guish risky and riskless choices. The paradigmatic example of decision under risk is the acceptability of a gamble that yields monetary outcomes with specified probabilities. A typical riskless decision concerns the acceptability of a transaction in which a good or a service is exchanged for money or labor. In the first part of this article we present an analysis of the cog- nitive and psychophysical factors that determine the value of risky prospects. In the second part we extend this analysis to transactions and trades. Risky Choice Risky choices, such as whether or not to take an umbrella and whether or not to go to war, are made without advance knowledge of their consequences. Because the consequences of such actions depend on uncertain events such as the weather or the opponent's resolve, the choice of an act may be construed as the acceptance of a gamble that can yield various out- comes with different probabilities. It is therefore nat- ural that the study of decision making under risk has focused on choices between simple gambles with monetary outcomes and specified probabilities, in the hope that these simple problems will reveal basic at- titudes toward risk and value. We shall sketch an approach to risky choice that
Article
People feel they understand complex phenomena with far greater precision, coherence, and depth than they really do; they are subject to an illusion-an illusion of explanatory depth. The illusion is far stronger for explanatory knowledge than many other kinds of knowledge, such as that for facts, procedures or narratives. The illusion for explanatory knowledge is most robust where the environment supports real-time explanations with visible mechanisms. We demonstrate the illusion of depth with explanatory knowledge in Studies 1-6. Then we show differences in overconfidence about knowledge across different knowledge domains in Studies 7-10. Finally, we explore the mechanisms behind the initial confidence and behind overconfidence in Studies 11 and 12, and discuss the implications of our findings for the roles of intuitive theories in concepts and cognition. © 2002 Leonid Rozenblit. Published by Cognitive Science Society, Inc. All rights reserved.
Article
The paper outlines a computational theory of human plausible reasoning constructed from analysis of people's answers to everyday questions. Like logic, the theory is expressed in a content-independent formalism. Unlike logic, the theory specifies how different information in memory affects the certainty of the conclusions drawn. The theory consists of a dimensionalized space of different inference types and their certainty conditions, including a variety of meta-inference types where the inference depends on the person's knowledge about his own knowledge. The protocols from people's answers to questions are analyzed in terms of the different inference types. The paper also discusses how memory is structured in multiple ways to spport the different inference types, and how the information found in memory determines which inference types are triggered.
Article
Plausibility has been implicated as playing a critical role in many cognitive phenomena from comprehension to problem solving. Yet, across cognitive science, plausibility is usually treated as an operationalized variable or metric rather than being explained or studied in itself. This article describes a new cognitive model of plausibility, the Plausibility Analysis Model (PAM), which is aimed at modeling human plausibility judgment. This model uses commonsense knowledge of concept-coherence to determine the degree of plausibility of a target scenario. In essence, a highly plausible scenario is one that fits prior knowledge well: with many different sources of corroboration, without complexity of explanation, and with minimal conjecture. A detailed simulation of empirical plausibility findings is reported, which shows a close correspondence between the model and human judgments. In addition, a sensitivity analysis demonstrates that PAM is robust in its operations.
Article
Two experiments were conducted in order to find out whether textual features of narratives differentially affect credibility judgments made by judges having different levels of absorption (a disposition associated with rich visual imagination). Participants in both experiments were exposed to a textual narrative and requested to judge whether the narrator actually experienced the event he described in his story. In Experiment 1, the narrative varied in terms of language (literal, figurative) and plausibility (ordinary, anomalous). In Experiment 2, the narrative varied in terms of language only. The participants' perceptions of the plausibility of the story described and the extent to which they were absorbed in reading were measured. The data from both experiments together suggest that the groups applied entirely different criteria in credibility judgments. For high-absorption individuals, their credibility judgment depends on the degree to which the text can be assimilated into their own vivid imagination, whereas for low-absorption individuals it depends mainly on plausibility. That is, high-absorption individuals applied an experiential mental set while judging the credibility of the narrator, whereas low-absorption individuals applied an instrumental mental set. Possible cognitive mechanisms and implications for credibility judgments are discussed.
Article
Many abductive understanding systems generate explanations by a backwards chaining process that is neutral both to the explainer's previous experience in similar situations and to why the explainer is attempting to explain. This article examines the relationship of such models to an approach that uses case-based reasoning to generate explanations. In this case-based model, the generation of abductive explanations is focused by prior experience and by goal-based criteria reflecting current information needs. The article analyzes the commitments and contributions of this case-based model as applied to the task of building good explanations of anomalous events in everyday understanding. The article identifies six central issues for abductive explanation, compares how these issues are addressed in traditional and case-based explanation models, and discusses benefits of the case-based approach for facilitating generation of plausible and useful explanations in domains that are complex and imperfectly understood.
An operational account of mental models: A pilot study
  • J Borders
  • G Klein
  • R Besuijen
Borders, J., Klein, G., and Besuijen, R. (2019). An operational account of mental models: A pilot study. International Conference on Naturalistic Decision Making, San Francisco, CA.
Making the implausible plausible
  • L Connell
Connell, L. (2004). Making the implausible plausible. In Proceedings of the annual meeting of the Cognitive Science Society (Vol. 26, No. 26).
PAM: A cognitive model of plausibility
  • L Connell
  • M T Keane
Connell, L., and Keane, M. T. (2003). PAM: A cognitive model of plausibility. In Proceedings of the annual meeting of the Cognitive Science Society (Vol. 25, No. 25).
The roots of plausibility: The role of coherence and distributional knowledge in plausibility judgements
  • L Connell
  • M T Keane
Connell, L., and Keane, M. T. (2002). The roots of plausibility: The role of coherence and distributional knowledge in plausibility judgements. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 24, No. 24).
March)). Sacred and profane: How not to negotiate with believers. The New Yorker
  • M Gladwell
Gladwell, M. (2014, March)). Sacred and profane: How not to negotiate with believers. The New Yorker. [https://www.newyorker.com].
Challenges of disposition decision making for pediatric trauma patients in the emergency department
  • B Z Hose
  • P Carayon
  • P Hoonakker
  • A Wooldridge
  • T Brazelton
  • S Dean
  • A Gurses
Hose, B. Z., Carayon, P., Hoonakker, P., Wooldridge, A., Brazelton, T., Dean, S., and Gurses, A. (2018, August). Challenges of disposition decision making for pediatric trauma patients in the emergency department. In Congress of the International Ergonomics Association (pp. 339-345). Springer, Cham.
A formal model of plausibility monitoring in language comprehension
  • M B Isberner
  • G Kern-Isberner
Isberner, M. B., and Kern-Isberner, G. (2016, March). A formal model of plausibility monitoring in language comprehension. In The Twenty-Ninth International Flairs Conference.
Understanding Air France 447
  • Bill Palmer
Palmer, Bill (2013). Understanding Air France 447. William Palmer, Jr.
Plausible reasoning: An introduction to the theory and practice of plausibilistic inference
  • N Rescher
Rescher, N. (1976). Plausible reasoning: An introduction to the theory and practice of plausibilistic inference. Amsterdam, the Netherlands: Van Gorcum.