Sendhil Mullainathan

Sendhil Mullainathan
University of Chicago | UC

About

226
Publications
69,672
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
37,271
Citations
Citations since 2016
77 Research Items
22421 Citations
201620172018201920202021202201,0002,0003,0004,000
201620172018201920202021202201,0002,0003,0004,000
201620172018201920202021202201,0002,0003,0004,000
201620172018201920202021202201,0002,0003,0004,000

Publications

Publications (226)
Article
Open datasets, curated around unsolved medical problems, are vital to the development of computational research in medicine, but remain in short supply. Nightingale Open Science, a non-profit computing platform, was founded to catalyse research in this nascent field.
Preprint
O’Donnell et al. (“ODAL”) (1) claim to audit the “scarcity literature” through a series of replications. Although we applaud the audit’s goals, we found serious issues that invalidate its conclusions. Notably, the paper fails as an audit of the scarcity literature. (1) It includes studies that are not about resource scarcity and even studies that a...
Preprint
Online platforms have a wealth of data, run countless experiments and use industrial-scale algorithms to optimize user experience. Despite this, many users seem to regret the time they spend on these platforms. One possible explanation is that incentives are misaligned: platforms are not optimizing for user happiness. We suggest the problem runs de...
Article
Full-text available
Significance Encouraging vaccination is a pressing policy problem. Our megastudy with 689,693 Walmart pharmacy customers demonstrates that text-based reminders can encourage pharmacy vaccination and establishes what kinds of messages work best. We tested 22 different text reminders using a variety of different behavioral science principles to nudge...
Article
Full-text available
Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes¹. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals². The lack of comparability of such individual investigations limits their po...
Article
How effective are physicians at diagnosing heart attacks? We study this question on a patient-by-patient basis by building a machine learning model of individual risk. We then contrast a physician’s decision to test a patient with the algorithm’s predicted risk; in cases of disagreement, we use data on health outcomes to judge whether the algorithm...
Article
Algorithms (in some form) are already widely used in the criminal justice system. We draw lessons from this experience for what is to come for the rest of society as machine learning diffuses. We find economists and other social scientists have a key role to play in shaping the impact of algorithms, in part through improving the tools used to build...
Article
Computational social science is more than just large repositories of digital data and the computational methods needed to construct and analyse them. It also represents a convergence of different fields with different ways of thinking about and doing science. The goal of this Perspective is to provide some clarity around how these approaches differ...
Article
Algorithms trained to predict mismeasured proxy variables can reproduce and scale up racial bias. This mechanism of algorithmic bias is distinct from others in the literature and harder to detect. We show this using examples from health care, but the forces we consider apply to a range of other important social sectors.
Article
Full-text available
Underserved populations experience higher levels of pain. These disparities persist even after controlling for the objective severity of diseases like osteoarthritis, as graded by human physicians using medical images, raising the possibility that underserved patients’ pain stems from factors external to the knee, such as stress. Here we use a deep...
Preprint
Full-text available
Machine learning models depend on the quality of input data. As electronic health records are widely adopted, the amount of data in health care is growing, along with complaints about the quality of medical notes. We use two prediction tasks, readmission prediction and in-hospital mortality prediction, to characterize the value of information in me...
Preprint
Understanding what leads to effective conversations can aid the design of better computer-mediated communication platforms. In particular, prior observational work has sought to identify behaviors of individuals that correlate to their conversational efficiency. However, translating such correlations to causal interpretations is a necessary step in...
Preprint
This paper introduces Data Stations, a new data architecture that we are designing to tackle some of the most challenging data problems that we face today: access to sensitive data; data discovery and integration; and governance and compliance. Data Stations depart from modern data lakes in that both data and derived data products, such as machine...
Article
The Coronavirus Aid, Relief, and Economic Security (CARES) Act and Paycheck Protection Program together designated $175 billion for coronavirus disease 2019 (COVID-19) response efforts and reimbursement to health care entities for expenses or lost revenues.
Article
Full-text available
Preventing discrimination requires that we have means of detecting it, and this can be enormously difficult when human beings are making the underlying decisions. As applied today, algorithms can increase the risk of discrimination. But as we argue here, algorithms by their nature require a far greater level of specificity than is usually possible...
Article
There are widespread concerns that the growing use of machine learning algorithms in important decisions may reproduce and reinforce existing discrimination against legally protected groups. Most of the attention to date on issues of “algorithmic bias” or “algorithmic fairness” has come from computer scientists and machine learning researchers. We...
Article
Racial bias in health algorithms The U.S. health care system uses commercial algorithms to guide health decisions. Obermeyer et al. find evidence of racial bias in one widely used algorithm, such that Black patients assigned the same level of risk by the algorithm are sicker than White patients (see the Perspective by Benjamin). The authors estimat...
Preprint
We use machine learning to provide a tractable measure of the amount of predictable variation in the data that a theory captures, which we call its "completeness." We apply this measure to three problems: assigning certain equivalents to lotteries, initial play in games, and human generation of random sequences. We discover considerable variation i...
Conference Paper
Algorithms can be a powerful aid to decision-making - particularly when decisions rely, even implicitly, on predictions [7]. We are already seeing algorithms play this role in domains including hiring, education, lending, medicine, and criminal justice [2, 6, 10]. As is typical in machine learning applications, accuracy is an important measure for...
Article
A debt trap occurs when someone takes on a high-interest-rate loan and is barely able to pay back the interest, and thus perpetually finds themselves in debt (often by refinancing). Studying such practices is important for understanding financial decision-making of households in dire circumstances, and also for setting appropriate consumer protecti...
Article
Patient-specific risk scores are used to identify individuals at elevated risk for cardiovascular disease. Typically, risk scores are based on patient habits and medical history - age, sex, race, smoking behavior, and prior vital signs and diagnoses. We explore an alternative source of information, a patient's raw electrocardiogram recording, and d...
Article
Concerns about the dissemination of spurious results have led to calls for pre-analysis plans (PAPs) to avoid ex-post “p-hacking.” But often the conceptual hypotheses being tested do not imply the level of specificity required for a PAP. In this paper we suggest a framework for PAPs that capitalize on the availability of causal machine-learning (ML...
Preprint
In a wide array of areas, algorithms are matching and surpassing the performance of human experts, leading to consideration of the roles of human judgment and algorithmic prediction in these domains. The discussion around these developments, however, has implicitly equated the specific task of prediction with the general task of automation. We argu...
Article
Computer algorithms are increasingly being used to predict people's preferences and make recommendations. Although people frequently encounter these algorithms because they are cheap to scale, we do not know how they compare to human judgment. Here, we compare computer recommender systems to human recommenders in a domain that affords humans many a...
Preprint
The law forbids discrimination. But the ambiguity of human decision-making often makes it extraordinarily hard for the legal system to know whether anyone has actually discriminated. To understand how algorithms affect discrimination, we must therefore also understand how they affect the problem of detecting discrimination. By one measure, algorith...
Conference Paper
A single algorithm drives an important health care decision for over 70 million people in the US. When health systems anticipate that a patient will have especially complex and intensive future health care needs, she is enrolled in a 'care management' program, which provides considerable additional resources: greater attention from trained provider...
Article
Full-text available
The law forbids discrimination. But the ambiguity of human decision-making often makes it hard for the legal system to know whether anyone has discriminated. To understand how algorithms affect discrimination, we must understand how they affect the detection of discrimination. With the appropriate requirements in place, algorithms create the potent...
Article
Shah et al. (2012) examined how different forms of scarcity affect attention and borrowing behavior. Results from a series of lab experiments suggested that (1) various forms of scarcity have similar effects on cognition and behavior, (2) scarcity leads to attentional shifts and greater focus (3) scarcity can lead people to over-borrow, and (4) sca...
Preprint
An electrocardiogram (EKG) is a common, non-invasive test that measures the electrical activity of a patient's heart. EKGs contain useful diagnostic information about patient health that may be absent from other electronic health record (EHR) data. As multi-dimensional waveforms, they could be modeled using generic machine learning tools, such as a...
Preprint
Databases of electronic health records (EHRs) are increasingly used to inform clinical decisions. Machine learning methods can find patterns in EHRs that are predictive of future adverse outcomes. However, statistical models may be built upon patterns of health-seeking behavior that vary across patient subpopulations, leading to poor predictive per...
Preprint
Algorithmic predictions are increasingly used to aid, or in some cases supplant, human decision-making, and this development has placed new demands on the outputs of machine learning procedures. To facilitate human interaction, we desire that they output prediction functions that are in some fashion simple or interpretable. And because they influen...
Preprint
Large labeled datasets for supervised learning are frequently constructed by assigning each instance to multiple human evaluators, and this leads to disagreement in the labels associated with a single instance. Here we consider the question of predicting the level of disagreement for a given instance, and we find an interesting phenomenon: direct p...
Article
End-of-life health care spending In the United States, one-quarter of Medicare spending occurs in the last 12 months of life, which is commonly seen as evidence of waste. Einav et al. used predictive modeling to reassess this interpretation. From detailed Medicare claims data, the extent to which spending is concentrated not just on those who die,...
Article
Concerns that algorithms may discriminate against certain groups have led to numerous efforts to 'blind' the algorithm to race. We argue that this intuitive perspective is misleading and may do harm. Our primary result is exceedingly simple, yet often overlooked. A preference for fairness should not change the choice of estimator. Equity preference...
Article
Recent research has studied how resource scarcity draws attention and creates cognitive load. As a result, scarcity improves some dimensions of cognitive function, while worsening others. Still, there remains a fundamental question: how does scarcity influence the content of cognition? In this article, we find that poor individuals (i.e., those fac...
Article
Can machine learning improve human decision making? Bail decisions provide a good test case. Millions of times each year, judges make jail-or-release decisions that hinge on a prediction of what a defendant would do if released. The concreteness of the prediction task combined with the volume of data available makes this a promising machine-learnin...
Article
Full-text available
Objective To estimate individual level body temperature and to correlate it with other measures of physiology and health. Design Observational cohort study. Setting Outpatient clinics of a large academic hospital, 2009-14. Participants 35 488 patients who neither received a diagnosis for infections nor were prescribed antibiotics, in whom temper...
Data
Supplemental information: eTables 1-3 and eFigures 1 and 2
Conference Paper
Evaluating whether machines improve on human performance is one of the central questions of machine learning. However, there are many domains where the data is selectively labeled, in the sense that the observed outcomes are themselves a consequence of the existing choices of the human decision-makers. For instance, in the context of judicial bail...
Article
An increasing number of domains are providing us with detailed trace data on human decisions in settings where we can evaluate the quality of these decisions via an algorithm. Motivated by this development, an emerging line of work has begun to consider whether we can characterize and predict the kinds of decisions where people are likely to make e...
Article
A core challenge in the analysis of experimental data is that the impact of some intervention is often not entirely captured by a single, well-defined outcome. Instead there may be a large number of outcome variables that are potentially affected and of interest. In this paper, we propose a data-driven approach rooted in machine learning to the pro...
Article
When we test a theory using data, it is common to focus on correctness: do the predictions of the theory match what we see in the data? But we also care about completeness: how much of the predictable variation in the data is captured by the theory? This question is difficult to answer, because in general we do not know how much "predictable variat...
Conference Paper
When we test a theory using data, it is common to focus on correctness: do the predictions of the theory match what we see in the data? But we also care about completeness: how much of the predictable variation in the data is captured by the theory? This question is difficult to answer, because in general we do not know how much "predictable variat...
Conference Paper
A broad range of on-line behaviors are mediated by interfaces in which people make choices among sets of options. A rich and growing line of work in the behavioral sciences indicate that human choices follow not only from the utility of alternatives, but also from the choice set in which alternatives are presented. In this work we study comparison-...
Article
A broad range of on-line behaviors are mediated by interfaces in which people make choices among sets of options. A rich and growing line of work in the behavioral sciences indicate that human choices follow not only from the utility of alternatives, but also from the choice set in which alternatives are presented. In this work we study comparison-...
Article
Machines are increasingly doing "intelligent" things. Face recognition algorithms use a large dataset of photos labeled as having a face or not to estimate a function that predicts the presence y of a face from pixels x. This similarity to econometrics raises questions: How do these new empirical tools fit with what we know? As empirical economists...
Article
Machine learning tools are beginning to be deployed en masse in health care. While the statistical underpinnings of these techniques have been questioned with regard to causality and stability, we highlight a different concern here, relating to measurement issues. A characteristic feature of health data, unlike other applications of machine learnin...
Article
We present the results of three large-scale randomized controlled trials (RCTs) carried out in Chicago, testing interventions to reduce crime and dropout by changing the decision-making of economically disadvantaged youth. We study a program called Becoming a Man (BAM), developed by the non-profit Youth Guidance, in two RCTs implemented in 2009–10...
Article
Policymakers and researchers are increasingly interested in using experimental methods to inform the design of social policy. The most common approach, at least in developed countries, is to carry out large-scale randomized trials of the policies of interest, or what we call here policy evaluations. In this chapter, we argue that in some circumstan...
Article
We provide evidence from field experiments with three different banks that reminder messages increase commitment attainment for clients who recently opened commitment savings accounts. Messages that mention both savings goals and financial incentives are particularly effective, whereas other content variations such as gain versus loss framing do no...
Article
Full-text available
Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there...
Article
An increasing number of domains are providing us with detailed trace data on human decisions in settings where we can evaluate the quality of these decisions via an algorithm. Motivated by this development, an emerging line of work has begun to consider whether we can characterize and predict the kinds of decisions where people are likely to make e...
Article
Economists have become increasingly interested in studying the nature of production functions in social policy applications, with the goal of improving productivity. Traditionally models have assumed workers are homogenous inputs. However, in practice, substantial variability in productivity means the marginal productivity of labor depends substant...
Article
All individuals rely on a fundamental set of mental capacities and functions, or bandwidth, in their economic and non-economic lives. Yet, many factors associated with poverty, such as malnutrition, alcohol consumption, or sleep deprivation, may tax this capacity. Previous research has demonstrated that such taxes often significantly alter judgment...
Article
Self-control problems change the logic of agency theory by partly aligning the interests of the firm and worker: both now value contracts that elicit future effort. Findings from a year-long field experiment with full-time data entry workers support this idea. First, workers increase output by voluntarily choosing dominated contracts (which penaliz...
Article
Most empirical policy work focuses on causal inference. We argue an important class of policy problems does not require causal inference but instead requires predictive inference. Solving these “prediction policy problems” requires more than simple regression techniques, since these are tuned to generating unbiased estimates of coefficients rather...
Article
Economic models of decision making assume that people have a stable way of thinking about value. In contrast, psychology has shown that people's preferences are often malleable and influenced by normatively irrelevant contextual features. Whereas economics derives its predictions from the assumption that people navigate a world of scarce resources,...
Article
We consider a model of technological learning under which people “learn through noticing”: they choose which input dimensions to attend to and subsequently learn about from available data. Using this model, we show how people with a great deal of experience may persistently be off the production frontier because they fail to notice important featur...
Conference Paper
Social scientists increasingly criticize the use of machine learning techniques to understand human behavior. Criticisms include: (1) They are atheoretical and hence of limited scientific value; (2) They do not address causality and are hence of limited policy value; and (3) They are uninterpretable and hence of limited generalizability value (outs...
Article
Full-text available
Antiretroviral therapy (ART), a treatment that significantly delays the onset of AIDS, has recently become available throughout many African countries, rapidly reversing the downward trend in life ex-pectancy due to AIDS. Economic theory predicts that a longer life expectancy increases the value of human capital investment. The effect of life expec...
Article
Full-text available
Successful development programs rely on people to behave and choose in certain ways, and behavioral economics helps us understand why people behave and choose as they do. This paper sketches how to design development programs and policies in ways that are cognizant of and informed by the insights behavioral economics provides into human behavior. I...