Judea Pearl’s research while affiliated with University of California, Los Angeles and other places


Ad

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (260)


Personalized decision making – A conceptual introduction
  • Article
  • Full-text available

April 2023

·

101 Reads

·

30 Citations

Journal of Causal Inference

·

Judea Pearl

Personalized decision making targets the behavior of a specific individual, while population-based decision making concerns a subpopulation resembling that individual. This article clarifies the distinction between the two and explains why the former leads to more informed decisions. We further show that by combining experimental and observational studies, we can obtain valuable information about individual behavior and, consequently, improve decisions over those obtained from experimental studies alone. In particular, we show examples where such a combination discriminates between individuals who can benefit from a treatment and those who cannot – information that would not be revealed by experimental studies alone. We outline areas where this method could be of benefit to both policy makers and individuals involved.

Download


Epsilon-Identifiability of Causal Quantities

January 2023

·

42 Reads

Identifying the effects of causes and causes of effects is vital in virtually every scientific field. Often, however, the needed probabilities may not be fully identifiable from the data sources available. This paper shows how partial identifiability is still possible for several probabilities of causation. We term this epsilon-identifiability and demonstrate its usefulness in cases where the behavior of certain subpopulations can be restricted to within some narrow bounds. In particular, we show how unidentifiable causal effects and counterfactual probabilities can be narrowly bounded when such allowances are made. Often those allowances are easily measured and reasonably assumed. Finally, epsilon-identifiability is applied to the unit selection problem.


Figure 1: (P (y x ), P (y x )) v.s. Expected increased lower bound E(L − L).
Figure 2: (P (y x ), P (y x )) v.s. Expected decreased upper bound E(U − U ).
Probabilities of Causation: Role of Observational Data

October 2022

·

72 Reads

Probabilities of causation play a crucial role in modern decision-making. Pearl defined three binary probabilities of causation, the probability of necessity and sufficiency (PNS), the probability of sufficiency (PS), and the probability of necessity (PN). These probabilities were then bounded by Tian and Pearl using a combination of experimental and observational data. However, observational data are not always available in practice; in such a case, Tian and Pearl's Theorem provided valid but less effective bounds using pure experimental data. In this paper, we discuss the conditions that observational data are worth considering to improve the quality of the bounds. More specifically, we defined the expected improvement of the bounds by assuming the observational distributions are uniformly distributed on their feasible interval. We further applied the proposed theorems to the unit selection problem defined by Li and Pearl.


Figure 2: Estimation of the bounds of PNS for the first model using different size of samples.
Figure 3: Estimation of the bounds of PNS for the second model using different size of samples.
Figure 4: Average error of estimations using different size of data.
Probabilities of Causation: Adequate Size of Experimental and Observational Samples

October 2022

·

58 Reads

The probabilities of causation are commonly used to solve decision-making problems. Tian and Pearl derived sharp bounds for the probability of necessity and sufficiency (PNS), the probability of sufficiency (PS), and the probability of necessity (PN) using experimental and observational data. The assumption is that one is in possession of a large enough sample to permit an accurate estimation of the experimental and observational distributions. In this study, we present a method for determining the sample size needed for such estimation, when a given confidence interval (CI) is specified. We further show by simulation that the proposed sample size delivered stable estimations of the bounds of PNS.



Causation and decision: On Dawid’s “Decision theoretic foundation of statistical causality”

September 2022

·

63 Reads

·

4 Citations

Journal of Causal Inference

In a recent issue of this journal, Philip Dawid (2021) proposes a framework for causal inference that is based on statistical decision theory and that is, in many aspects, compatible with the familiar framework of causal graphs (e.g., Directed Acyclic Graphs (DAGs)). This editorial compares the methodological features of the two frameworks as well as their epistemological basis.


Female vs male CATE
Personalized Decision Making -- A Conceptual Introduction

August 2022

·

31 Reads

Personalized decision making targets the behavior of a specific individual, while population-based decision making concerns a sub-population resembling that individual. This paper clarifies the distinction between the two and explains why the former leads to more informed decisions. We further show that by combining experimental and observational studies we can obtain valuable information about individual behavior and, consequently, improve decisions over those obtained from experimental studies alone.


Figure 5: PNS bounds for causal diagram of Figure 4a
Figure 6: PNS bounds for causal diagram of Figure 1a
Figure 7: PNS bounds for causal diagram of Figure 4b among narrowed samples referenced by part med 2 in Table 2
Causes of Effects: Learning Individual Responses from Population Data

July 2022

·

41 Reads

·

30 Citations

The problem of individualization is crucial in almost every field of science. Identifying causes of specific observed events is likewise essential for accurate decision making as well as explanation. However, such tasks invoke counterfactual relationships, and are therefore indeterminable from population data. For example, the probability of benefiting from a treatment concerns an individual having a favorable outcome if treated and an unfavorable outcome if untreated; it cannot be estimated from experimental data, even when conditioned on fine-grained features, because we cannot test both possibilities for an individual. Tian and Pearl provided bounds on this and other probabilities of causation using a combination of experimental and observational data. Those bounds, though tight, can be narrowed significantly when structural information is available in the form of a causal model. This added information may provide the power to solve central problems, such as explainable AI, legal responsibility, and personalized medicine, all of which demand counterfactual logic. This paper derives, analyzes, and characterizes these new bounds, and illustrates some of their practical applications.


Figure 14: Structural Causal Model M and its associated graph G
Figure 16: Model 18 explained
A Crash Course in Good and Bad Controls

May 2022

·

89 Reads

·

318 Citations

Sociological Methods & Research

Many students of statistics and econometrics express frustration with the way a problem known as “bad control” is treated in the traditional literature. The issue arises when the addition of a variable to a regression equation produces an unintended discrepancy between the regression coefficient and the effect that the coefficient is intended to represent. Avoiding such discrepancies presents a challenge to all analysts in the data intensive sciences. This note describes graphical tools for understanding, visualizing, and resolving the problem through a series of illustrative examples. By making this “crash course” accessible to instructors and practitioners, we hope to avail these tools to a broader community of scientists concerned with the causal interpretation of regression models.


Ad

Citations (83)


... With access to larger datasets, especially in the context of digital trace data, we expect a greater emphasis on new methods that can be combined with machine learning (ML) to disentangle causality in observational datasets with potentially many more variables than observations (Athey & Imbens, 2017). These integrative approaches can also offer valuable insights into understanding heterogeneity, which can bring us closer to estimating individual causal effects-and meaningfully distinguish between personalized and population-based decision-making (Athey & Imbens, 2016;Mueller and Pearl, 2023;Wager & Athey, 2018). To overcome challenges in applying causal theories or targeting policy interventions, the Data for Policy community is increasingly encouraging counterfactual thinking, especially by leveraging a combination of both experimental and observational data. ...

Reference:

Data technologies and analytics for policy and governance: a landscape review
Personalized decision making – A conceptual introduction

Journal of Causal Inference

... With these assumptions about the underlying process, we concluded that length would be a bad control, resulting in an uninterpretable and misleading regression (Achen 2005;Rohrer 2018); see Cinelli et al. (2022) for an introduction to Directed Acyclic Graphs. Instead, we used regressions as a means to quantify how features changed over time, without implying causality. ...

A Crash Course in Good and Bad Controls

Sociological Methods & Research

... This framework was devised by Rubin and expanded by Hernan and Robins 6 , among others 27 . Pearl (2022), Robins (2022) and Rubin (2022) have provided a thorough discussion of the history of causal inference 28,29,30 . In order to discuss estimands defined in causal inference, we give a brief description of the potential outcomes paradigm. ...

Interview with Judea Pearl
  • Citing Article
  • October 2022

Observational Studies

... In addition, DAGs have been useful in describing typical biases (e.g., Refs. [63,64]), finding adjustment variables (e.g., Ref. [65]), and elucidating apparent paradoxes (e.g., Simpson's paradox in Refs. [66,67]). ...

Recovering from Selection Bias in Causal and Statistical Inference
  • Citing Article
  • June 2014

Proceedings of the AAAI Conference on Artificial Intelligence

... More recently, Li and Pearl [2024b] extended the definitions and bounds to a more general form. Additionally, Mueller et al. [2022], as well as Dawid et al. [2017], demonstrated that these bounds could be further refined given specific causal structures. However, any above estimation of the probabilities of causation requires both observational and experimental data. ...

Causes of Effects: Learning Individual Responses from Population Data

... In the practical scenarios where causal effects are estimated by observational data, unobserved confounding factors may exist. Therefore, many methods have also been developed to address them, such as using the front-door, the back-door criterion [21], do-calculus [22], instrumental variables [41], and other methods. However, the above methods greatly depend on the causal structure of the data conforming to the correlation criterion of front-door, back-door, do-calculus, and instrumental variables. ...

External Validity: From Do -Calculus to Transportability Across Populations
  • Citing Chapter
  • February 2022

... To address the aforementioned issues, we consider instead a weaker form of task-guidance where quality metrics can be aligned with task-specific information without requiring the expense of training a new task model directly on the dataset of interest (D4). In this setting shown in Figure 6, the computation of Q is dependent not only on the image X but on the label set Y. The task T is used as a selection variable (Zadrozny, 2004;Bareinboim et al., 2022) on which the dataset is conditioned, and as a collider in the DAG, T creates an association between M, Q. Q uses information about the label set Y. The task T is viewed as a selection variable which influences both the labels Y and Q. ...

Recovering from Selection Bias in Causal and Statistical Inference
  • Citing Chapter
  • February 2022

... In principle, an EL problem like Odeen can be approached by training an end-to-end neural network to predictŷ = 1 P i (x ′ ), given as input a set of observations D i and a single sample x ′ (see Figure 11 C, left). Such a model would assume that all the information needed to solve the task is embedded in the data, ignoring the explanations; we may call it a "radical empiricist" approach (Pearl 2021). A variant that includes the explanations in the pipeline can be done by adding a textual head to the network. ...

Radical empiricism and machine learning research

Journal of Causal Inference

... The conflict intensity analysis employs several methodological strategies to address potential identification challenges. First, to prevent post-treatment bias (the "bad control" problem) [35,36], all control variables are lagged by one year (1y) regardless of temporal specification, Second, SPEI provides a plausibly exogenous drought measure (see Section 5.1.2) that is commonly used in observational studies of conflict [37,34,38,11,9]. One potential source of omitted variable bias requires attention: climate change may cause drought probability to shift non-uniformly across grid-cells over time in ways that correlate with conflict risk factors [32]. ...

A Crash Course in Good and Bad Controls

SSRN Electronic Journal