# Joseph Y. HalpernCornell University | CU · Department of Computer Science

Joseph Y. Halpern

## About

516

Publications

43,137

Reads

**How we measure 'reads'**

A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more

31,457

Citations

Citations since 2016

## Publications

Publications (516)

As autonomous systems rapidly become ubiquitous, there is a growing need for a legal and regulatory framework to address when and how such a system harms someone. There have been several attempts within the philosophy literature to define harm, but none of them has proven capable of dealing with with the many examples that have been presented, lead...

In a companion paper (Beckers et al. 2022), we defined a qualitative notion of harm: either harm is caused, or it is not. For practical applications, we often need to quantify harm; for example, we may want to choose the lest harmful of a set of possible interventions. We first present a quantitative definition of harm in a deterministic context in...

We review the literature on models that try to explain human behavior in social interactions described by normal-form games with monetary payoffs. We start by covering social and moral preferences. We then focus on the growing body of research showing that people react to the language in which actions are described, especially when it activates mor...

We review the literature on models that try to explain human behavior in social interactions described by normal-form games with monetary payoffs. We start by covering social and moral preferences. We then focus on the growing body of research showing that people react to the language in which actions are described, especially when it activates mor...

We offer a general theoretical framework for brain and behavior that is evolutionarily and computationally plausible. The brain in our abstract model is a network of nodes and edges. Although it has some similarities to standard neural network models, as we show, there are some significant differences. Both nodes and edges in our network have weigh...

Secure function computation has been thoroughly studied and optimized in the past decades. We extend techniques used for secure computation to simulate arbitrary protocols involving a mediator. The key feature of our notion of simulation is that it is bidirectional: not only does the simulation produce only outputs that could happen in the original...

The question of how people hold others responsible has motivated decades of theorizing and empirical work. In this paper, we develop and test a computational model that bridges the gap between broad but qualitative framework theories, and quantitative but narrow models. In our model, responsibility judgments are the result of two cognitive processe...

In most contemporary approaches to decision making under uncertainty, a decision problem is described by a set of states and set of outcomes, and a rich set of acts, which are functions from states to outcomes over which the decision maker (DM) has preferences. Many interesting decision problems, however, do not come with a state space and an outco...

In Savage's classic decision-theoretic framework, actions are formally defined as functions from states to outcomes. But where do the state space and outcome space come from? Expanding on recent work by Blume, Easley, and Halpern (BEH), we consider a language-based framework in which actions are identified with (conditional) descriptions in a simpl...

Abraham, Dolev, Geffner, and Halpern proved that, in asynchronous systems, a $(k,t)$-robust equilibrium for $n$ players and a trusted mediator can be implemented without the mediator as long as $n > 4(k+t)$, where an equilibrium is $(k,t)$-robust if, roughly speaking, no coalition of $t$ players can decrease the payoff of any of the other players,...

We investigate how to model the beliefs of an agent who becomes more aware. We use the framework of Halpern and Rego (2013) by adding probability, and define a notion of a model transition that describes constraints on how, if an agent becomes aware of a new formula $\phi$ in state $s$ of a model $M$, she transitions to state $s^*$ in a model $M^*$...

We define Bayesian games with intentions by introducing a distinction between “intended” and “actual” actions, generalizing both Bayesian games and (static) psychological games Geanakoplos et al. (1989). We propose a new solution concept for this framework and prove that Nash equilibria in static psychological games correspond to a special class of...

We investigate how to model the beliefs of an agent who becomes more aware. We use the framework of Halpern and Rego (2013) by adding probability, and define a notion of a model transition that describes constraints on how, if an agent becomes aware of a new formula φ in state s of a model M, she transitions to state s* in a model M*. We then discu...

In many real-world settings, a decision-maker must combine information provided by different experts in order to decide on an effective policy. Alrajeh, Chockler, and Halpern [2018] showed how to combine causal models that are compatible in the sense that, for variables that appear in both models, the experts agree on the causal structure. In this...

We provide a game-theoretic analysis of consensus, assuming that processes are controlled by rational agents and may fail by crashing. We consider agents that \emph{care only about consensus}: that is, (a) an agent's utility depends only on the consensus value achieved (and not, for example, on the number of messages the agent sends) and (b) agents...

Many objectives can be achieved (or may be achieved more effectively) only by a group of agents executing a team plan. If a team plan fails, it is often of interest to determine what caused the failure, the degree of responsibility of each agent for the failure, and the degree of blame attached to each agent. We show how team plans can be represent...

We formalize decision-making problems in robotics and automated control using continuous MDPs and actions that take place over continuous time intervals. We then approximate the continuous MDP using finer and finer discretizations. Doing this results in a family of systems, each of which has an extremely large action space, although only a few acti...

In the past few decades, numerous experiments have shown that humans do not always behave so as to maximize their material payoff. Cooperative behavior when noncooperation is a dominant strategy (with respect to the material payoffs) is particularly puzzling. Here we propose a novel approach to explain cooperation, assuming what Halpern and Pass ca...

How do people hold others responsible for their actions? In this paper, we test and extend a computational framework originally introduced by Gerstenberg et al. (2018) that assigns responsibility as a function of two factors: a dispositional inference that captures what we learn about a person's character from their action, and the causal role that...

We consider computational games, sequences of games \(\mathcal {G}=(G_1,G_2,\ldots )\) where, for all n, \(G_n\) has the same set of players. Computational games arise in electronic money systems such as Bitcoin, in cryptographic protocols, and in the study of generative adversarial networks in machine learning. Assuming that one-way functions exis...

Brandenburger, Friedenberg, and Keisler provide an epistemic characterization of iterated admissibility (IA), also known as iterated deletion of weakly dominated strategies, where uncertainty is represented using LPSs (lexicographic probability sequences). Their characterization holds in a rich structure called a complete structure, where all types...

We examine sequential equilibrium in the context of computational games (Halpern and Pass 2015), where agents are charged for computation. In such games, an agent can rationally choose to forget, so issues of imperfect recall arise. In this setting, we consider two notions of sequential equilibrium. One is an ex ante notion, where a player chooses...

We develop a modal logic to capture partial awareness. The logic has three building blocks: objects, properties, and concepts. Properties are unary predicates on objects; concepts are Boolean combinations of properties. We take an agent to be partially aware of a concept if she is aware of the concept without being aware of the properties that defi...

We consider a sequence of successively more restrictive definitions of abstraction for causal models, starting with a notion introduced by Rubenstein et al. (2017) called exact transformation that applies to probabilistic causal models, moving to a notion of uniform transformation that applies to deterministic causal models and does not allow diffe...

We provide a formal definition of blameworthiness in settings where multiple agents can collaborate to avoid a negative outcome. We first provide a method for ascribing blameworthiness to groups relative to an epistemic state (a distribution over causal models that describe how the outcome might arise). We then show how we can go from an ascription...

A mediator can help non-cooperative agents obtain an equilibrium that may otherwise not be possible. We study the ability of players to obtain the same equilibrium without a mediator, using only cheap talk, that is, nonbinding pre-play communication. Previous work has considered this problem in a synchronous setting. Here we consider the effect of...

Scientific models describe natural phenomena at different levels of abstraction. Abstract descriptions can provide the basis for interventions on the system and explanation of observed phenomena at a level of granularity that is coarser than the most fundamental account of the system. Beckers and Halpern (2019), building on work of Rubenstein et al...

Scientific models describe natural phenomena at different levels of abstraction. Abstract descriptions can provide the basis for interventions on the system and explanation of observed phenomena at a level of granularity that is coarser than the most fundamental account of the system. Beckers and Halpern (2019), building on work of Rubenstein et al...

Secure function computation has been thoroughly studied and optimized in the past decades. We extend techniques used for secure computation to simulate arbitrary protocols involving a mediator. The key feature of our notion of simulation is that it is bidirectional: not only does the simulation produce only outputs that could happen in the original...

We study the problem of computing an ϵ-Nash equilibrium in repeated games. Earlier work by Borgs et al. (2010) suggests that this problem is intractable. We show that if we make a slight change to their model—modeling the players as polynomial-time Turing machines that maintain state—and make a standard cryptographic assumption (that public-key cry...

We provide a formal definition of blameworthiness in settings where multiple agents can collaborate to avoid a negative outcome. We first provide a method for ascribing blameworthiness to groups relative to an epistemic state (a distribution over causal models that describe how the outcome might arise). We then show how we can go from an ascription...

We do a game-theoretic analysis of leader election, under the assumption that each agent prefers to have some leader than no leader at all. We show that it is possible to obtain a fair Nash equilibrium, where each agent has an equal probability of being elected leader, in a completely connected network, in a bidirectional ring, and a unidirectional...

We consider a sequence of successively more restrictive definitions of abstraction for causal models, starting with a notion introduced by Rubenstein et al. (2017) called exact transformation that applies to probabilistic causal models, moving to a notion of uniform transformation that applies to deterministic causal models and does not allow diffe...

We develop a modal logic to capture partial awareness. The logic has three building blocks: objects, properties, and concepts. Properties are unary predicates on objects; concepts are Boolean combinations of properties. We take an agent to be partially aware of a concept if she is aware of the concept without being aware of the properties that defi...

Sufficient conditions are given under which ratifiable acts exist.

We provide formal definitions of degree of blameworthiness and intention relative to an epistemic state (a probability over causal models and a utility function on outcomes). These, together with a definition of actual causality, provide the key ingredients for moral responsibility judgments. We show that these definitions give insight into commons...

A traditional assumption in game theory is that players are opaque to one
another -- if a player changes strategies, then this change in strategies does
not affect the choice of other players' strategies. In many situations this is
an unrealistic assumption. We develop a framework for reasoning about games
where the players may be translucent to on...

We consider the problem of detecting norm violations in open multi-agent systems (MAS). In this extended abstract, we outline the approach of [Alechina et al., 2018], and show how, using ideas from scrip systems, we can design mechanisms where the agents comprising the MAS are incentivised to monitor the actions of other agents for norm violations.

We consider the problem of detecting norm violations in open multi-agent systems (MAS).We show how, using ideas from scrip systems, we can design mechanisms where the agents comprising the MAS are incentivised to monitor the actions of other agents for norm violations. The cost of providing the incentives is not borne by the MAS and does not come f...

A mediator can help non-cooperative agents obtain an equilibrium that may otherwise not be possible. We study the ability of players to obtain the same equilibrium without a mediator, using only cheap talk, that is, nonbinding pre-play communication. Previous work has considered this problem in a synchronous setting. Here we consider the effect of...

McNamara, Trimmer, and Houston (2012) claim to provide an explanation of certain systematic deviations from rational behavior using a mechanism that could arise through natural selection. We provide an arguably much simpler mechanism in terms of computational limitations, that performs better in the environment described by McNamara, Trimmer, and H...

When humans and other animals make cultural innovations, they also change their environment, thereby imposing new selective pressures that can modify their biological traits. For example, there is evidence that dairy farming by humans favored alleles for adult lactose tolerance. Similarly, the invention of cooking possibly affected the evolution of...

Characterizations of Nash equilibrium, correlated equilibrium, and rationalizability in terms of common knowledge of rationality are well known. Analogous characterizations of sequential equilibrium, (trembling hand) perfect equilibrium, and quasi-perfect equilibrium using results of Halpern [2009].

When economists represent and reason about knowledge, they typically do so at a semantic (or set-theoretic) level. In this paper, it is argued that there are also benefits in using a syntactic representation.

We provide a sound and complete axiomatization for a class of logics appropriate for reasoning about the rationality of players in games, and show that essentially the same axiomatization applies to a very wide class of decision rules. We also consider games in which players may be uncertain as to what decision rules their opponents are using, and...

We present an approach to incentivising monitoring for norm violations in open multi-agent systems such as Wikipedia. In such systems, there is no crisp definition of a norm violation; rather, it is a matter of judgement whether an agent's behaviour conforms to generally accepted standards of behaviour. Agents may legitimately disagree about border...

We present an approach to incentivising monitoring for norm violations in open multi-agent systems such as Wikipedia. In such systems, there is no crisp definition of a norm violation; rather, it is a matter of judgement whether an agent’s behaviour conforms to generally accepted standards of behaviour. Agents may legitimately disagree about border...

Many objectives can be achieved (or may be achieved more effectively) only by a group of agents executing a team plan. If a team plan fails, it is often of interest to determine what caused the failure, the degree of responsibility of each agent for the failure, and the degree of blame attached to each agent. We show how team plans can be represent...

A first-order conditional logic is considered, with semantics given by a variant of Ïμ -semantics where φ → ψ means that Pr (ψ | φ) approaches 1 super-polynomially-faster than any inverse polynomial. This type of convergence is needed for reasoning about security protocols. A complete axiomatization is provided for this semantics, and it is shown h...

McNamara, Trimmer, and Houston [2012] claim to provide an explanation of certain systematic deviation from rational behavior using a mechanism that could arise through natural selection. We provide an arguably much simpler explanation. To argue convincingly that the type of mechanism proposed by McNamara et al. is adaptive, and is likely to be sele...

We provide a game-theoretic analysis of consensus, assuming that processes are controlled by rational agents and may fail by crashing. We consider agents that care only about consensus: that is, (a) an agent's utility depends only on the consensus value achieved (and not, for example, on the number of messages the agent sends) and (b) agents strict...

We define solution concepts appropriate for computationally bounded players playing a fixed finite game. To do so, we need to define what it means for a computational game, which is a sequence of games that get larger in some appropriate sense, to represent a single finite underlying extensive-form game. Roughly speaking, we require all the games i...

We show that standard Bayesian games cannot represent the full spectrum of belief-dependent preferences. However, by introducing a fundamental distinction between intended and actual strategies, we remove this limitation. We define Bayesian games with intentions, generalizing both Bayesian games and psychological games, and prove that Nash equilibr...

The menu-dependent nature of regret-minimization creates subtleties when it
is applied to dynamic decision problems. Firstly, it is not clear whether
\emph{forgone opportunities} should be included in the \emph{menu}, with
respect to which regrets are computed, at different points of the decision
problem. If forgone opportunities are included, howe...

Chateauneuf and Faro (J Math Econ 45:535–558, 2009) axiomatize a weighted version of maxmin expected utility over acts with nonnegative utilities, where weights are represented by a confidence function. We argue that their representation is only one of many possible, and we axiomatize a more natural form of maxmin weighted expected utility. We also...

We consider the problem of detecting norm violations in open multi-agent systems (MAS). We show how, using ideas from scrip systems, we can design mechanisms where the agents comprising the MAS are incentivised to monitor the actions of other agents for norm violations. The cost of providing the incentives is not borne by the MAS and does not come...

For changing opinion, represented by an assignment of probabilities to propositions, the criterion proposed is motivated by the requirement that the assignment should have, and maintain, the possibility of matching in some appropriate sense statistical proportions in a population. This ‘tracking’ criterion implies limitations on policies for updati...

Aumann has proved that common knowledge of substantive rationality implies the backwards induction solution in games of perfect information. Stalnaker has proved that it does not. Roughly speaking, a player is substantively rational if, for all vertices v, if the player were to reach vertex v, then the player would be rational at vertex v. It is sh...

As a highly consequential biological trait, a memory “bottleneck” cannot escape selection pressures. It must therefore co-evolve with other cognitive mechanisms rather than act as an independent constraint. Recent theory and an implemented model of language acquisition suggest that a limit on working memory may evolve to help learning. Furthermore,...

Causality plays a central role in the way people structure the world; we constantly seek causal explanations for our observations. But what does it even mean that an event C "actually caused" event E? The problem of defining actual causation goes beyond mere philosophical speculation. For example, in many legal arguments, it is precisely what needs...

Recently, Halpern and Leung suggested representing uncertainty by a set of weighted probability measures, and suggested a way of making decisions based on this representation of uncertainty: maximizing weighted regret. Their paper does not answer an apparently simpler question: what it means, according to this representation of uncertainty, for an...

We present a formal logic for quantitative reasoning about security
properties of network protocols. The system allows us to derive concrete
security bounds that can be used to choose key lengths and other security
parameters. We provide axioms for reasoning about digital signatures and random
nonces, with security properties based on the concrete...

We consider a setting where a decision maker’s uncertainty is represented by a set of probability measures, rather than a single measure. Measure-by-measure updating of such a set of measures upon acquiring new information is well known to suffer from problems. To deal with these problems, we propose using weighted sets of probabilities: a represen...

We consider sequences of games $\mathcal{G}=\{G_1,G_2,\ldots\}$ where, for
all $n$, $G_n$ has the same set of players. Such sequences arise in the
analysis of running time of players in games, in electronic money systems such
as Bitcoin and in cryptographic protocols. Assuming that one-way functions
exist, we prove that there is a sequence of 2-pla...

This short note discusses the role of syntax vs. semantics and the interplay
between logic, philosophy, and language in computer science and game theory.

We define solution concepts appropriate for computationally bounded players
playing a fixed finite game. To do so, we need to define what it means for a
\emph{computational game}, which is a sequence of games that get larger in some
appropriate sense, to represent a single finite underlying extensive-form game.
Roughly speaking, we require all the...

The original Halpern-Pearl definition of causality [Halpern and Pearl, 2001]
was updated in the journal version of the paper [Halpern and Pearl, 2005] to
deal with some problems pointed out by Hopkins and Pearl [2003]. Here the
definition is modified yet again, in a way that (a) leads to a simpler
definition, (b) handles the problems pointed out by...

In the last few decades, numerous experiments have shown that humans do not always behave so as to maximize their material payoff. Cooperative behavior when non-cooperation is a dominant strategy (with respect to the material payoffs) is particularly puzzling. Here we propose a novel approach to explain cooperation, assuming what Halpern and Pass (...

This chapter provides an introduction to some basic concepts of epistemic
logic, basic formal languages, their semantics, and proof systems. It also
contains an overview of the handbook, and a brief history of epistemic logic
and pointers to the literature.

Standard economic models cannot capture the fact that information is often ambiguous, and is interpreted in multiple ways. Using a framework that distinguishes between the language in which statements are made and the interpretation of statements, we demonstrate that, unlike in the case where there is no ambiguity, players may come to have differen...

How do people assign responsibility for the outcome of an election? In previous work, we have shown that responsibility judgments in achievement contexts are affected by the probability that a person’s contribution is necessary, and by how close it was to being pivotal (Lagnado, Gerstenberg, & Zultan, 2013). Here we focus on responsibility judgment...

Nash equilibrium (NE) assumes that players always make a best response.
However, this is not always true; sometimes people cooperate even it is not a
best response to do so. For example, in the Prisoner's Dilemma, people often
cooperate. Are there rules underlying cooperative behavior? In an effort to
answer this question, we propose a new equilibr...

We study the problem of finding a subgame-perfect equilibrium in repeated games. In earlier work [Halpern, Pass and Seeman 2014], we showed how to efficiently find an (approximate) Nash equilibrium if assuming that players are computationally bounded (and making standard cryptographic hardness assumptions); in contrast, as demonstrated in the work...

Causal models
defined in terms of structural equations have proved to be quite a powerful way of representing knowledge regarding causality. However, a number of authors have given examples that seem to show that the Halpern–Pearl (HP) definition of causality (Halpern & Pearl, 2005) gives intuitively unreasonable answers. Here it is shown that, for...

Halpern and Pearl introduced a definition of actual causality; Eiter and
Lukasiewicz showed that computing whether X=x is a cause of Y=y is NP-complete
in binary models (where all variables can take on only two values) and\
Sigma_2^P-complete in general models. In the final version of their paper,
Halpern and Pearl slightly modified the definition...

In the last few decades, numerous experiments have shown that humans do not
always behave so as to maximize their material payoff. Cooperative behavior
when non-cooperation is a dominant strategy (with respect to the material
payoffs) is particularly puzzling. Here we propose a novel approach to explain
cooperation, assuming what Halpern and Pass (...

We study type spaces where a player's type at a state is a conditional probability on the space. We axiomatize these spaces using conditional belief operators, examining three additional axioms of increasing strength. First, introspection, which requires the agent to be unconditionally certain of her beliefs. Second, echo, according to which the un...

Markov decision processes (MDPs) are widely used for modeling decision-making
problems in robotics, automated control, and economics. Traditional MDPs assume
that the decision maker (DM) knows all states and actions. However, this may
not be true in many situations of interest. We define a new framework, MDPs
with unawareness (MDPUs) to deal with t...

Halpern and Pearl introduced a definition of actual causality; Eiter and Lukasiewicz showed that computing whether X = x is a cause of Y = y is NP-complete in binary models (where all variables can take on only two values) and \Sigma^P_2-complete in general models. In the final version of their paper, Halpern and Pearl slightly modified the definit...

There have been two major lines of research aimed at capturing resource-bounded players in game theory. The first, initiated by Rubinstein (), charges an agent for doing costly computation; the second, initiated by Neyman (), does not charge for computation, but limits the computation that agents can do, typically by modeling agents as finite autom...

We show how game-theoretic solution concepts such as Nash equilibrium, correlated equilibrium, rationalizability, and sequential equilibrium can be given a uniform definition in terms of a knowledge-based program with counterfactual semantics. In a precise sense, this program can be viewed as providing a procedural characterization of rationality.

Standard models of multi-agent modal logic do not capture the fact that
information is often \emph{ambiguous}, and may be interpreted in different ways
by different agents. We propose a framework that can model this, and consider
different semantics that capture different assumptions about the agents'
beliefs regarding whether or not there is ambig...

We study the problem of computing an $\epsilon$-Nash equilibrium in repeated
games. Earlier work by Borgs et al.[2010] suggests that this problem is
intractable. We show that if we make a slight change to their model---modeling
the players as polynomial-time Turing machines that maintain state (rather than
stateless polynomial-time Turing machines)...

We introduce language-based games, a generalization of psychological games
[6] that can also capture reference-dependent preferences [7]. The idea is to
extend the domain of the utility function to situations, maximal consistent
sets in some language. The role of the underlying language in this framework is
thus particularly critical. Of special in...

We do a game-theoretic analysis of leader election, under the assumption that each agent prefers to have some leader than to have no leader at all. We show that it is possible to obtain a fair Nash equilibrium, where each agent has an equal probability of being elected leader, in a completely connected network, in a bidirectional ring, and a unidir...

Recent work in psychology and experimental philosophy has shown that
judgments of actual causation are often influenced by consideration of
defaults, typicality, and normality. A number of philosophers and computer
scientists have also suggested that an appeal to such factors can help deal
with problems facing existing accounts of actual causation....

Recently, Halpern and Leung suggested representing uncertainty by a weighted
set of probability measures, and suggested a way of making decisions based on
this representation of uncertainty: maximizing weighted regret. Their paper
does not answer an apparently simpler question: what it means, according to
this representation of uncertainty, for an...

We examine sequential equilibrium in the context of computational games [Halpern and Pass, 2011a], where agents are charged for computation. In such games, an agent can rationally choose to forget, so issues of imperfect recall arise. In this setting, we consider two notions of sequential equilibrium. One is an ex ante notion, where a player choose...

Judea Pearl (2000) was the first to propose a definition of actual causation using causal models. A number of authors have suggested that an adequate account of actual causation must appeal not only to causal structure but also to considerations of normality. In Halpern and Hitchcock (2011), we offer a definition of actual causation using extended...

Playersʼ beliefs may be incompatible, in the sense that player i can assign probability 1 to an event E to which player j assigns probability 0. One way to block incompatibility is to assume a common prior. We consider here a different approach: we require playersʼ beliefs to be conservative, in the sense that all players must ascribe the actual wo...

Nash equilibrium (NE) assumes that players always make a best response. However, this is not always true; sometimes people cooperate even it is not a best response to do so. For example, in the Prisoner's Dilemma, people often coop-erate. We consider two solution concepts that were intro-duced recently that try to capture such cooperation in two-pl...

In previous work [BGHK92, BGHK93], we have studied the random-worlds approach
-- a particular (and quite powerful) method for generating degrees of belief
(i.e., subjective probabilities) from a knowledge base consisting of objective
(first-order, statistical, and default) information. But allowing a knowledge
base to contain only objective informa...

We examine a new approach to modeling uncertainty based on plausibility
measures, where a plausibility measure just associates with an event its
plausibility, an element is some partially ordered set. This approach is easily
seen to generalize other approaches to modeling uncertainty, such as
probability measures, belief functions, and possibility...

The study of belief change has been an active area in philosophy and AI. In
recent years two special cases of belief change, belief revision and belief
update, have been studied in detail. Roughly, revision treats a surprising
observation as a sign that previous beliefs were wrong, while update treats a
surprising observation as an indication that...

Conditioning is the generally agreed-upon method for updating probability
distributions when one learns that an event is certainly true. But it has been
argued that we need other rules, in particular the rule of cross-entropy
minimization, to handle updates that involve uncertain information. In this
paper we re-examine such a case: van Fraassen's...