Updating, Evidence Evaluation, and Operators: The Steering of Belief

Abstract

Decades of findings in psychology appear to indicate that human belief is thoroughly irrational. At best, beliefs might be formed by heuristic processes that predictably lead to suboptimal outcomes. At worst, they are slaves to motivated reasoning, which allows people to come to whichever conclusions they prefer. In this paper, we argue that belief is best understood as the outcome of multiple processes, some rational and others susceptible to irrational influences. We suggest that belief updating, narrowly construed, may be a rational process that is uniquely sensitive to evidence and cognitively impenetrable to desires or incentives. Before any updating can occur, however, a series of processes mediate between information in the world and subjectively compelling evidence. We distinguish between updating proper and processes of evidence search, acceptance, hypothesis specification, integration of relevant information, and reasoning. We review research from philosophy and psychology which highlights the computational difficulty inherent to each of these problems and conclude that solutions must be heuristic and fallible. Beyond incidental failures, evidence evaluation processes, unlike updating, are penetrable to motivation and as such, may be biased by people’s desires and goals. We propose a framework for integrating research on belief with three levels of hierarchical influence. At level one, updating is approximately Bayesian. Level two processes search for and evaluate evidence and are necessarily heuristic and exhibit bounded rationality (Simon, 1956) given the difficulty of the problems they have to solve. Finally, at level three, factors including information representation and individual differences imply different operators for manipulating information. Our framework offers a more nuanced understanding of belief, permits a more granular localization of irrationality, and may help reconcile opposing theories of whether humans are rational or irrational.