Article

Guilty Artificial Minds: Folk Attributions of Mens Rea and Culpability to Artificially Intelligent Agents

Authors:
  • National Yang Ming Chiao Tung University
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

While philosophers hold that it is patently absurd to blame robots or hold them morally responsible [1], a series of recent empirical studies suggest that people do ascribe blame to AI systems and robots in certain contexts [2]. This is disconcerting: Blame might be shifted from the owners, users or designers of AI systems to the systems themselves, leading to the diminished accountability of the responsible human agents [3]. In this paper, we explore one of the potential underlying reasons for robot blame, namely the folk's willingness to ascribe inculpating mental states or "mens rea" to robots. In a vignette-based experiment (N=513), we presented participants with a situation in which an agent knowingly runs the risk of bringing about substantial harm. We manipulated agent type (human v. group agent v. AI-driven robot) and outcome (neutral v. bad), and measured both moral judgment (wrongness of the action and blameworthiness of the agent) and mental states attributed to the agent (recklessness and the desire to inflict harm). We found that (i) judgments of wrongness and blame were relatively similar across agent types, possibly because (ii) attributions of mental states were, as suspected, similar across agent types. This raised the question - also explored in the experiment - whether people attribute knowledge and desire to robots in a merely metaphorical way (e.g., the robot "knew" rather than really knew). However, (iii), according to our data people were unwilling to downgrade to mens rea in a merely metaphorical sense when given the chance. Finally, (iv), we report a surprising and novel finding, which we call the inverse outcome effect on robot blame: People were less willing to blame artificial agents for bad outcomes than for neutral outcomes. This suggests that they are implicitly aware of the dangers of overattributing blame to robots when harm comes to pass, such as inappropriately letting the responsible human agent off the moral hook.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The current analysis, which does not document an aversion toward algorithm-mediated investments, is in contrast to earlier research that showed an aversion toward algorithms making investment decisions (Niszczota and Kaszás, 2020). Similar to Kirchkamp and Strobel (2019) and Stuart and Kneer (2021), sharing responsibility with machines led to neither the diffusion nor the exacerbation of responsibility. ...
Preprint
Full-text available
Current research does not resolve how people will judge harmful activist investments if machines (machine learning algorithms) are involved in the investment process as advisors, and not the ones “pulling the trigger”. On the one hand, machines might diffuse responsibility for making a socially responsible, but harmful investment. On the other hand, machines could exacerbate the blame that is assigned to the investment fund, which can be penalized for outsourcing part of the decision process to an algorithm. We attempted to resolve this issue experimentally. In our experiment (N = 956), participants judge either a human research team or a machine learning algorithm as the source of advice to the investment team to short-sell a company that they suspect of committing financial fraud. Results suggest that investment funds will be similarly blameworthy for an error regardless of using human or machine intelligence to support their decision to short-sell a company. This finding highlights a novel and relevant circumstance in which reliance on algorithms does not backfire by making the final decision-maker (e.g., an investment fund) more blameworthy. Nor does it lessen the perceived blameworthiness of the final decision-maker, by making algorithms into “electronic scapegoats” for providing well-intended but harmful advice.
... J o u r n a l P r e -p r o o f attribution of multiple factors (Shaver, 1985). Emotional theories of blame focus on holding other people responsible when we have reactive feelings of resentment or indignation (Strawson, 1962;Wallace, 1994). Conative theories of blame, on the other hand, place an emphasis on intentions, proposing that blame occurs when a person has acted badly coupled with the desire that they had not performed the act at all (Sher, 2006). ...
Article
Full-text available
In general, people will judge a morally wrong behavior when perpetrated by an artificial intelligence (AI) as still being wrong. But moral judgements are complex, and therefore we explore how moral judgements made about AIs differ from those made about humans in real-world situations. In contrast to much of the current research on the morality of AIs, we examine real-world encounters where an AI commits a moral wrong as reported by participants in previous research. We adapt these to create nearly identical scenarios with human perpetrators. In Study 1, across scenarios, humans are perceived as more wrong, intentional, and blameworthy compared to AIs. In Study 2, we replicate those results and find that showing the participants the contrasting scenario – showing the AI scenario when one is rating the human scenario or vice versa – does not have a significant effect on moral judgements. An exploratory word-frequency analysis and illustrative quotes from participants’ open-ended explanations show that AIs are more often denied agency and perceived as programmed and therefore unintentional in producing the moral outcome. In contrast, humans are perceived as highly agentic and intentional, either fully responsible for the wrongdoing or not morally culpable because the behavior was perceived as a mistake.
... What does explain the suprising effect of praise and blame in our study? Stuart and Kneer (2021) have found that people assign more knowledge and blame to an autonomously acting robot when it doesn't commit any harm compared to when it commits harm. This "inverse outcome effect" mirrors the results found in our study. ...
Chapter
Full-text available
In this paper, we call for a new approach to the psychology of free will attribution. While past research in experimental philosophy and psychology has mostly been focused on reasoning-based judgment ("the courtroom approach"), we argue that like agency and mindedness, free will can also be experienced perceptually ("the perceptual approach"). We further propose a new model of free will attribution-the agency model-according to which the experience of free will is elicited by the perceptual cues that prompt the attribution of agency. Finally, developing new stimuli that fit the perceptual approach, we present some preliminary evidence in support of the agency model.
Article
The rapid deployment of semi-autonomous systems (i.e., systems requiring human monitoring such as Uber AVs) poses ethical challenges when these systems face morally-laden situations. We ask how people evaluate morally-laden decisions of humans who monitor these systems in situations of unavoidable harm. We conducted three pre-registered experiments (total N = 1811), using modified trolley moral dilemma scenarios. Our findings suggest that people apply different criteria when judging morality and deserved punishment of regular-car versus AV drivers. Regular-car drivers are judged according to a consequentialist minimizing harm criterion, whereas AV drivers are judged according to whether or not they took action, with a more favorable prior for acting. Integrating judgment and decision-making research with moral psychology, the current research illuminates how the presence versus absence of automation affects moral judgments.
Article
Full-text available
In two experiments (total N=693) we explored whether people are willing to consider paintings made by AI-driven robots as art, and robots as artists. Across the two experiments, we manipulated three factors: (i) agent type (AI-driven robot v. human agent), (ii) behavior type (intentional creation of a painting v. accidental creation), and (iii) object type (abstract v. representational painting). We found that people judge robot paintings and human paintings as art to roughly the same extent. However, people are much less willing to consider robots as artists than humans, which is partially explained by the fact that they are less disposed to attribute artistic intentions to robots.
Article
Full-text available
Artificial agents are progressively becoming more present in everyday-life situations and more sophisticated in their interaction affordances. In some specific cases, like Google Duplex, GPT-3 bots or Deep Mind’s AlphaGo Zero, their capabilities reach or exceed human levels. The use contexts of everyday life necessitate making such agents understandable by laypeople. At the same time, displaying human levels of social behavior has kindled the debate over the adoption of Dennett’s ‘intentional stance’. By means of a comparative analysis of the literature on robots and virtual agents, we defend the thesis that approaching these artificial agents ‘as if’ they had intentions and forms of social, goal-oriented rationality is the only way to deal with their complexity on a daily base. Specifically, we claim that this is the only viable strategy for non-expert users to understand, predict and perhaps learn from artificial agents’ behavior in everyday social contexts. Furthermore, we argue that as long as agents are transparent about their design principles and functionality, attributing intentions to their actions is not only essential, but also ethical. Additionally, we propose design guidelines inspired by the debate over the adoption of the intentional stance.
Article
Full-text available
In two experiments (total N=693) we explored whether people are willing to consider paintings made by AI-driven robots as art, and robots as artists. Across the two experiments, we manipulated three factors: (i) agent type (AI-driven robot v. human agent), (ii) behavior type (intentional creation of a painting v. accidental creation), and (iii) object type (abstract v. representational painting). We found that people judge robot paintings and human painting as art to roughly the same extent. However, people are much less willing to consider robots as artists than humans, which is partially explained by the fact that they are less disposed to attribute artistic intentions to robots. Keywords: artificial intelligence, creativity, aesthetics, AI art, mental states
Article
Full-text available
Recent research shows-somewhat astonishingly-that people are willing to ascribe moral blame to AI-driven systems when they cause harm [1]-[4]. In this paper, we explore the moral-psychological underpinnings of these findings. Our hypothesis was that the reason why people ascribe moral blame to AI systems is that they consider them capable of entertaining inculpating mental states (what is called mens rea in the law). To explore this hypothesis, we created a scenario in which an AI system runs a risk of poisoning people by using a novel type of fertilizer. Manipulating the computational (or quasi-cognitive) abilities of the AI system in a between-subjects design, we tested whether people's willingness to ascribe knowledge of a substantial risk of harm (i.e., recklessness) and blame to the AI system. Furthermore, we investigated whether the ascription of recklessness and blame to the AI system would influence the perceived blameworthiness of the system's user (or owner). In an experiment with 347 participants, we found (i) that people are willing to ascribe blame to AI systems in contexts of recklessness, (ii) that blame ascriptions depend strongly on the willingness to attribute recklessness and (iii) that the latter, in turn, depends on the perceived "cognitive" capacities of the system. Furthermore, our results suggest (iv) that the higher the computational sophistication of the AI system, the more blame is shifted from the human user to the AI system.
Article
Full-text available
Increasingly complex and autonomous systems require machine ethics to maximize the benefits and minimize the risks to society arising from the new technology. It is challenging to decide which type of ethical theory to employ and how to implement it effectively. This survey provides a threefold contribution. First, it introduces a trimorphic taxonomy to analyze machine ethics implementations with respect to their object (ethical theories), as well as their nontechnical and technical aspects. Second, an exhaustive selection and description of relevant works is presented. Third, applying the new taxonomy to the selected works, dominant research patterns, and lessons for the field are identified, and future directions for research are suggested.
Article
Full-text available
This paper proposes to address the question of the legal status of artificial intelligence (AI) from a perspective that is unique to itself. Which means that, rather than attempting to place AI in the box of legal personhood—where many other nonhuman entities already reside, in a legal space where they are in a state of constant friction with humans—we will see whether these inhabitants can be placed in a different box: not that of legal personhood, but that of the intelligent machine. I accordingly argue that we have made a mistake in not clearly maintaining the original separation between legal persons and human persons: this is how legal persons—particularly as the concept applies to corporations—have so far been claiming and gaining the rights ascribed to humans, on the basis of their personhood. AI instead suggests the need to work outside the framework of this fiction: it suggests that we should stop using the fiction of the person for something that is not a person in the first place and in the most original and primary sense of this word. I propose a different metaphor, the metaphor of the intelligent machine, to this end drawing on the work of Dan-Cohen (2016), who advanced this metaphor to argue that it fits corporations better than the metaphor of the person. In conclusion, we should be able to see that this metaphor of the intelligent machine makes a better fit not only for corporations but also for AI and robotics.
Article
Full-text available
In a landmark essay, Andreas Matthias claimed that current developments in autonomous, artificially intelligent (AI) systems are creating a so-called responsibility gap, which is allegedly ever-widening and stands to undermine both the moral and legal frameworks of our society. But how severe is the threat posed by emerging technologies? In fact, a great number of authors have indicated that the fear is thoroughly instilled. The most pessimistic are calling for a drastic scaling-back or complete moratorium on AI systems, while the optimists aim to show that the gap can be bridged nonetheless. Contrary to both camps, I argue against the prevailing assumption that there is a technology-based responsibility gap. I show how moral responsibility is a dynamic and flexible process, one that can effectively encompass emerging technological entities.
Article
Full-text available
A robot's decision to harm a person is sometimes considered to be the ultimate proof of it gaining a human-like mind. Here, we contrasted predictions about attribution of mental capacities from moral typecasting theory, with the denial of agency from dehumanization literature. Experiments 1 and 2 investigated mind perception for intentionally and accidentally harmful robotic agents based on text and image vignettes. Experiment 3 disambiguated agent intention (malevolent and benevolent), and additionally varied the type of agent (robotic and human) using short computer generated animations. Harmful robotic agents were consistently imbued with mental states to a lower degree than benevolent agents, supporting the dehumanization account. Further results revealed that a human moral patient appeared to suffer less when depicted with a robotic agent than with another human. The findings suggest that future robots may become subject to human-like dehumanization mechanisms, which challenges the established beliefs about anthropomorphism in the domain of moral interactions.
Article
Full-text available
AI research is growing rapidly raising various ethical issues related to safety, risks, and other effects widely discussed in the literature. We believe that in order to adequately address those issues and engage in a productive normative discussion it is necessary to examine key concepts and categories. One such category is anthropomorphism. It is a well-known fact that AI’s functionalities and innovations are often anthropomorphized (i.e., described and conceived as characterized by human traits). The general public’s anthropomorphic attitudes and some of their ethical consequences (particularly in the context of social robots and their interaction with humans) have been widely discussed in the literature. However, how anthropomorphism permeates AI research itself (i.e., in the very language of computer scientists, designers, and programmers), and what the epistemological and ethical consequences of this might be have received less attention. In this paper we explore this issue. We first set the methodological/theoretical stage, making a distinction between a normative and a conceptual approach to the issues. Next, after a brief analysis of anthropomorphism and its manifestations in the public, we explore its presence within AI research with a particular focus on brain-inspired AI. Finally, on the basis of our analysis, we identify some potential epistemological and ethical consequences of the use of anthropomorphic language and discourse within the AI research community, thus reinforcing the need of complementing the practical with a conceptual analysis.
Article
Full-text available
When two actors have the same mental states but one happens to harm another person (unlucky actor) and the other one does not (lucky actor), the latter elicits milder moral judgment. To understand how this outcome effect would affect post-harm interactions between victims and perpetrators, we examined how the social role from which transgressions are perceived moderates the outcome effect, and how outcome effects on moral judgments transfer to agentic and communal interpersonal goals. Three vignette experiments (N = 950) revealed similar outcome effects on moral judgment across social roles. In contrast, outcome effects on agentic and communal goals varied by social role: Victims exhibited the strongest outcome effects and perpetrators the weakest, with bystanders falling in between. Moral judgment did not mediate the effects of outcome severity on interpersonal goals. We discuss the possibility that outcome severity raises normative expectations regarding post-harm interactions that are unrelated to moral considerations. This article is protected by copyright. All rights reserved
Preprint
Full-text available
When two actors have exactly the same mental states but one happens to harm another person (unlucky actor) and the other one does not (lucky actor), the latter elicits milder moral judgment among bystanders. We hypothesized that the social role from which transgressions are perceived would moderate this outcome effect. In three preregistered experiments (N = 950), we randomly assigned participants to imagine and respond to moral scenarios as actor (i.e., perpetrator), victim, or bystander. The results revealed highly similar outcome effects on moral judgment across social roles. However, as predicted, the social role moderated the strength of the outcome effect on interpersonal goals pertaining to agency and communion. Although in agreement about the blameworthiness of lucky and unlucky actors, victims’ agency and communion were more sensitive to the outcome severity than perpetrators’ agency and communion, with bystanders’ outcome sensitivity falling in between. Outcome severity affected agency and communion directly instead of being mediated by moral judgment. We discuss the possibility that outcome severity raises normative expectations regarding interaction in a transgression’s aftermath that are unrelated to moral considerations.
Article
Full-text available
Artificial intelligence (AI) has historically been conceptualized in anthropomorphic terms. Some algorithms deploy biomimetic designs in a deliberate attempt to effect a sort of digital isomorphism of the human brain. Others leverage more general learning strategies that happen to coincide with popular theories of cognitive science and social epistemology. In this paper, I challenge the anthropomorphic credentials of the neural network algorithm, whose similarities to human cognition I argue are vastly overstated and narrowly construed. I submit that three alternative supervised learning methods—namely lasso penalties, bagging, and boosting—offer subtler, more interesting analogies to human reasoning as both an individual and a social phenomenon. Despite the temptation to fall back on anthropomorphic tropes when discussing AI, however, I conclude that such rhetoric is at best misleading and at worst downright dangerous. The impulse to humanize algorithms is an obstacle to properly conceptualizing the ethical challenges posed by emerging technologies.
Article
Full-text available
Theory of Mind is an inferential system central to human–human communication by which people ascribe mental states to self and other, and then use those deductions to make predictions about others’ behaviors. Despite the likelihood that ToM may also be central to interactions with other types of agents exhibiting similar cues, it is not yet fully known whether humans develop ToM for mechanical agents exhibiting properties of intelligence and sociality. A suite of five tests for implicit ToM were performed (white lie test, behavioral intention task, facial affect inference, vocal affect inference, and false-belief test) for three different robots and a human control. Findings suggest that implicit ToM signals are consistent across variably human-like robots and humans, so long as the social cues are similar and interpretable, but there is no association between implicit ToM signals and explicit mind ascription; findings suggest that heuristics and deliberation of mental status of robots may compete with implicit social-cognitive reactions.
Article
Full-text available
This study aims to probe how children with and without autism spectrum disorders (ASD) attribute false belief to a social robot and predict its action accordingly. Twenty 5- to 7-year-old children with ASD and 20 age- and IQ-matched typically developing (TD) children participated in two false belief tasks adapted for robot settings (change-of-location task and the unexpected-contents task). The results showed that most TD children are capable of attributing false belief to the social robot, that is, they could infer higher level mental states in robots, which extends our understanding in TD children’s perception and cognition on social robots. Conversely, children with ASD still show difficulty in interpreting robots’ mental states compared to their TD peers, which would greatly interfere with their interactions and communications with social robots and might impact on efficiency of robot-based intervention and education approaches. This group difference in attributing false belief to social robots could not be explained by the different perception and categorization of the robot. Our study implies that although children with ASD appear to be highly attracted by social robots, they still have difficulty in understanding mental states when socially interacting with robots, which should be taken into consideration when designing the robot-based intervention approach targeting to improve social behaviors of ASD.
Article
Full-text available
This study employs an experiment to test subjects’ perceptions of an artificial intelligence (AI) crime-predicting agent that produces clearly racist predictions. It used a 2 (human crime predictor/AI crime predictor) x 2 (high/low seriousness of crime) design to test the relationship between the level of autonomy and responsibility for the unjust results. The seriousness of crime was manipulated to examine the relationship between the perceived threat and trust in the authority’s decisions. Participants (N=334) responded to an online questionnaire after reading one of four scenarios with the same story depicting a crime predictor unjustly reporting a higher likelihood of subsequent crimes for a black defendant than for a white defendant for similar crimes. The results indicate that people think that an AI crime predictor has significantly less autonomy than a human crime predictor. However, both the identity of the crime predictor and the seriousness of the crime showed insignificant results on the level of responsibility assigned to the predictor. Also, a clear positive relationship between autonomy and responsibility was found in both human and AI crime predictor scenarios. The implications of the findings for applications and theory are discussed.
Article
Full-text available
We study whether robots can satisfy the conditions of an agent fit to be held morally responsible, with a focus on autonomy and self-control. An analogy between robots and human groups enables us to modify arguments concerning collective responsibility for studying questions of robot responsibility. We employ Mele's history-sensitive account of autonomy and responsibility to argue that even if robots were to have all the capacities required of moral agency, their history would deprive them from autonomy in a responsibility-undermining way. We will also study whether humans and technological artifacts like robots can form hybrid collective agents that could be morally responsible for their actions and give an argument against such a possibility.
Conference Paper
Full-text available
It has long been assumed that when people observe robots they intuitively ascribe mind and intentionality to them, just as they do to humans. However, much of this evidence relies on experimenter-provided questions or self-reported judgments. We propose a new way of investigating people's mental state ascriptions to robots by carefully studying explanations of robot behavior. Since people's explanations of human behavior are deeply grounded in assumptions of mind and intentional agency, explanations of robot behavior can reveal whether such assumptions similarly apply to robots. We designed stimulus behaviors that were representative of a variety of robots in diverse contexts and ensured that people saw the behaviors as equally intentional, desirable, and surprising across both human and robot agents. We provided 121 participants with verbal descriptions of these behaviors and asked them to explain in their own words why the agent (human or robot) had performed them. To systematically analyze the verbal data, we used a theoretically grounded classification method to identify core explanation types. We found that people use the same conceptual toolbox of behavior explanations for both human and robot agents, robustly indicating inferences of intentionality and mind. But people applied specific explanatory tools at somewhat different rates and in somewhat different ways for robots, revealing specific expectations people hold when explaining robot behaviors.
Chapter
Full-text available
Even though morally competent artificial agents have yet to emerge in society, we need insights from empirical science into how people will respond to such agents and how these responses should inform agent design. Three survey studies presented participants with an artificial intelligence (AI) agent, an autonomous drone, or a human drone pilot facing a moral dilemma in a military context: to either launch a missile strike on a terrorist compound but risk the life of a child, or to cancel the strike to protect the child but risk a terrorist attack. Seventy-two percent of respondents were comfortable making moral judgments about the AI in this scenario and fifty-one percent were comfortable making moral judgments about the autonomous drone. These participants applied the same norms to the two artificial agents and the human drone pilot (more than 80% said that the agent should launch the missile). However, people ascribed different patterns of blame to humans and machines as a function of the agent’s decision of how to solve the dilemma. These differences in blame seem to stem from different assumptions about the agents’ embeddedness in social structures and the moral justifications those structures afford. Specifically, people less readily see artificial agents as embedded in social structures and, as a result, they explained and justified their actions differently. As artificial agents will (and already do) perform many actions with moral significance, we must heed such differences in justifications and blame and probe how they affect our interactions with those agents.
Article
Full-text available
There is much to be gained from interdisciplinary efforts to tackle complex psychological notions such as ‘theory of mind’. However, careful and consistent communication is essential when comparing artificial and biological intelligence, say Henry Shevlin and Marta Halina.
Article
Full-text available
As robots become more autonomous, people will see them as more responsible for wrongdoing. Moral psychology suggests that judgments of robot responsibility will hinge on perceived situational awareness, intentionality, and free will—plus anthropomorphism and the robot’s capacity for harm. We also consider questions of robot rights and moral decision-making.
Article
Full-text available
In daily social interactions, we need to be able to navigate efficiently through our social environment. According to Dennett (1971), explaining and predicting others’ behavior with reference to mental states (adopting the intentional stance) allows efficient social interaction. Today we also routinely interact with artificial agents: from Apple’s Siri to GPS navigation systems. In the near future, we might start casually interacting with robots. This paper addresses the question of whether adopting the intentional stance can also occur with respect to artificial agents. We propose a new tool to explore if people adopt the intentional stance toward an artificial agent (humanoid robot). The tool consists in a questionnaire that probes participants’ stance by requiring them to choose the likelihood of an explanation (mentalistic vs. mechanistic) of a behavior of a robot iCub depicted in a naturalistic scenario (a sequence of photographs). The results of the first study conducted with this questionnaire showed that although the explanations were somewhat biased toward the mechanistic stance, a substantial number of mentalistic explanations were also given. This suggests that it is possible to induce adoption of the intentional stance toward artificial agents, at least in some contexts.
Article
Full-text available
Artificial intelligence (AI) agents make decisions that affect individuals and society which can produce outcomes traditionally considered moral violations if performed by humans. Do people attribute the same moral permissibility and fault to AIs and humans when each produces the same moral violation outcome? Additionally, how do people attribute morality when the AI and human are jointly making the decision which produces that violation? We investigate these questions with an experiment that manipulates written descriptions of four real-world scenarios where, originally, a violation outcome was produced by an AI. Our decision-making structures include individual decision-making – either AIs or humans – and joint decision-making – either humans monitoring AIs or AIs recommending to humans. We find that the decision-making structure has little effect on morally faulting AIs, but that humans who monitor AIs are faulted less than solo humans and humans receiving recommendations. Furthermore, people attribute more permission and less fault to AIs compared to humans for the violation in both joint decision-making structures. The blame for joint AI-human wrongdoing suggests the potential for strategic scapegoating of AIs for human moral failings and the need for future research on AI-human teams.
Article
Full-text available
Do people want autonomous machines making moral decisions? Nine studies suggest that that the answer is 'no'-in part because machines lack a complete mind. Studies 1-6 find that people are averse to machines making morally-relevant driving, legal, medical, and military decisions, and that this aversion is mediated by the perception that machines can neither fully think nor feel. Studies 5-6 find that this aversion exists even when moral decisions have positive outcomes. Studies 7-9 briefly investigate three potential routes to increasing the acceptability of machine moral decision-making: limiting the machine to an advisory role (Study 7), increasing machines' perceived experience (Study 8), and increasing machines' perceived expertise (Study 9). Although some of these routes show promise, the aversion to machine moral decision-making is difficult to eliminate. This aversion may prove challenging for the integration of autonomous technology in moral domains including medicine, the law, the military, and self-driving vehicles.
Article
Full-text available
Robots are increasingly envisaged as our future cohabitants. However, while considerable progress has been made in recent years in terms of their technological realization, the ability of robots to interact with humans in an intuitive and social way is still quite limited. An important challenge for social robotics is to determine how to design robots that can perceive the user’s needs, feelings, and intentions, and adapt to users over a broad range of cognitive abilities. It is conceivable that if robots were able to adequately demonstrate these skills, humans would eventually accept them as social companions. We argue that the best way to achieve this is using a systematic experimental approach based on behavioral and physiological neuroscience methods such as motion/eye-tracking, electroencephalography, or functional near-infrared spectroscopy embedded in interactive human–robot paradigms. This approach requires understanding how humans interact with each other, how they perform tasks together and how they develop feelings of social connection over time, and using these insights to formulate design principles that make social robots attuned to the workings of the human brain. In this review, we put forward the argument that the likelihood of artificial agents being perceived as social companions can be increased by designing them in a way that they are perceived as intentional agents that activate areas in the human brain involved in social-cognitive processing. We first review literature related to social-cognitive processes and mechanisms involved in human–human interactions, and highlight the importance of perceiving others as intentional agents to activate these social brain areas. We then discuss how attribution of intentionality can positively affect human–robot interaction by (a) fostering feelings of social connection, empathy and prosociality, and by (b) enhancing performance on joint human–robot tasks. Lastly, we describe circumstances under which attribution of intentionality to robot agents might be disadvantageous, and discuss challenges associated with designing social robots that are inspired by neuroscientific principles.
Article
Full-text available
Many ethicists writing about automated systems (e.g. self-driving cars and autonomous weapons systems) attribute agency to these systems. Not only that, they seemingly attribute an autonomous or independent form of agency to these machines. This leads some ethicists to worry about responsibility-gaps and retribution-gaps in cases where automated systems harm or kill human beings. In this paper, I consider what sorts of agency it makes sense to attribute to most current forms of automated systems, in particular automated cars and military robots. I argue that whereas it indeed makes sense to attribute different forms of fairly sophisticated agency to these machines, we ought not to regard them as acting on their own, independently of any human beings. Rather, the right way to understand the agency exercised by these machines is in terms of human-robot collaborations, where the humans involved initiate, supervise, and manage the agency of their robotic collaborators. This means, I argue, that there is much less room for justified worries about responsibility-gaps and retribution-gaps than many ethicists think.
Article
Full-text available
We are living through an era of increased robotisation. Some authors have already begun to explore the impact of this robotisation on legal rules and practice. In doing so, many highlight potential liability gaps that might arise through robot misbehaviour. Although these gaps are interesting and socially significant, they do not exhaust the possible gaps that might be created by increased robotisation. In this article, I make the case for one of those alternative gaps: the retribution gap. This gap arises from a mismatch between the human desire for retribution and the absence of appropriate subjects of retributive blame. I argue for the potential existence of this gap in an era of increased robotisation; suggest that it is much harder to plug this gap than it is to plug those thus far explored in the literature; and then highlight three important social implications of this gap.
Article
Full-text available
Recently, a number of people have argued that certain entities embodied by groups of agents themselves qualify as agents, with their own (analogs of) beliefs, desires, and intentions; even, some claim, as moral agents. However, others have independently argued that fully-fledged moral agency involves a capacity for reactive attitudes such as guilt and indignation, and these capacities might seem beyond the ken of “collective” or “corporate” agents. Individuals embodying such agents can of course be ashamed, proud, or indignant about what the agent has done. But just as an entity needs to have its own beliefs, desires, and intentions to qualify as a bona fide agent, the required capacity for reactive attitudes is a capacity to have one’s own reactive attitudes. If fully-fledged moral agency requires reactive attitudes, the corporate agent must itself be capable of (analogs of) guilt and indignation. In this paper, we argue that at least certain corporate agents are. Or, more precisely, we argue that if there are bona fide corporate agents, these agents can have the capacities that are both associated with guilt and indignation and plausibly required for moral agency; in particular certain epistemic and motivational capacities.
Article
Full-text available
Moral norms play an essential role in regulating human interaction. With the growing sophistication and proliferation of robots, it is important to understand how ordinary people apply moral norms to robot agents and make moral judgments about their behavior. We report the first comparison of people's moral judgments (of permissibility, wrongness, and blame) about human and robot agents. Two online experiments (total N = 316) found that robots, compared with human agents, were more strongly expected to take an action that sacrifices one person for the good of many (a "utilitarian" choice), and they were blamed more than their human counterparts when they did not make that choice. Though the utilitarian sacrifice was generally seen as permissible for human agents, they were blamed more for choosing this option than for doing nothing. These results provide a first step toward a new field of Moral HRI, which is well placed to help guide the design of social robots.
Article
Full-text available
Anthropomorphism is a phenomenon that describes the human tendency to see human-like shapes in the environment. It has considerable consequences for people’s choices and beliefs. With the increased presence of robots, it is important to investigate the optimal design for this technology. In this paper we discuss the potential benefits and challenges of building anthropomorphic robots, from both a philosophical perspective and from the viewpoint of empirical research in the fields of human-robot interaction and social psychology. We believe that this broad investigation of anthropomorphism will not only help us to understand the phenomenon better, but can also indicate solutions for facilitating the integration of human-like machines in the real world.
Article
Full-text available
We introduce a theory of blame in five parts. Part 1 addresses what blame is: a unique moral judgment that is both cognitive and social, regulates social behavior, fundamentally relies on social cognition, and requires warrant. Using these properties, we distinguish blame from such phenomena as anger, event evaluation, and wrongness judgments. Part 2 offers the heart of the theory: the Path Model of Blame, which identifies the conceptual structure in which blame judgments are embedded and the information processing that generates such judgments. After reviewing evidence for the Path Model, we contrast it with alternative models of blame and moral judgment (Part 3) and use it to account for a number of challenging findings in the literature (Part 4). Part 5 moves from blame as a cognitive judgment to blame as a social act. We situate social blame in the larger family of moral criticism, highlight its communicative nature, and discuss the darker sides of moral criticism. Finally, we show how the Path Model of Blame can bring order to numerous tools of blame management, including denial, justification, and excuse.
Article
Human-automation cooperation has become ubiquitous. In this concept, automation refers to autonomous machines, robots, artificial intelligence, and other autonomous nonhuman agents. A human driver will share control of semiautonomous vehicles (semi-AVs) with an automated system and thus share responsibility for crashes caused by semi-AVs. Research has not clarified whether and why people would attribute different levels of blame and responsibility to automation (and its creators) and its human counterpart when each causes an equivalent crash. We conducted four experiments in two studies (total N = 1,045) to measure different responses (e.g., severity and acceptability judgment, blame and responsibility attribution, compensation judgment) to hypothetical crashes that are caused by the human or the automation in semi-AVs. The results provided previously unidentified evidence of a bias, which we called the "blame attribution asymmetry," a tendency that people will judge the automation-caused crash more harshly, ascribe more blame and responsibility to automation and its creators, and think the victim in this crash should be compensated more. This asymmetry arises in part because of the higher negative affect triggered by the automation-caused crash. This bias has a direct policy implication: a policy allowing "not-safe enough" semi-AVs on roads could backfire, because these AVs will lead to many traffic crashes, which might in turn produce greater psychological costs and deter more people from adopting them. Other theoretical and policy implications of our findings were also discussed.
Book
In The Myth of Morality, Richard Joyce argues that moral discourse is hopelessly flawed. At the heart of ordinary moral judgements is a notion of moral inescapability, or practical authority, which, upon investigation, cannot be reasonably defended. Joyce argues that natural selection is to blame, in that it has provided us with a tendency to invest the world with values that it does not contain, and demands that it does not make. Should we therefore do away with morality, as we did away with other faulty notions such as witches? Possibly not. We may be able to carry on with morality as a 'useful fiction' - allowing it to have a regulative influence on our lives and decisions, perhaps even playing a central role - while not committing ourselves to believing or asserting falsehoods, and thus not being subject to accusations of 'error'.
Article
The increasing presence of robots in society necessitates a deeper understanding into what attitudes people have toward robots. People may treat robots as mechanistic artifacts or may consider them to be intentional agents. This might result in explaining robots' behavior as stemming from operations of the mind (intentional interpretation) or as a result of mechanistic design (mechanistic interpretation). Here, we examined whether individual attitudes toward robots can be differentiated on the basis of default neural activity pattern during resting state, measured with electroencephalogram (EEG). Participants observed scenarios in which a humanoid robot was depicted performing various actions embedded in daily contexts. Before they were introduced to the task, we measured their resting state EEG activity. We found that resting state EEG beta activity differentiated people who were later inclined toward interpreting robot behaviors as either mechanistic or intentional. This pattern is similar to the pattern of activity in the default mode network, which was previously demonstrated to have a social role. In addition, gamma activity observed when participants were making decisions about a robot's behavior indicates a relationship between theory of mind and said attitudes. Thus, we provide evidence that individual biases toward treating robots as either intentional agents or mechanistic artifacts can be detected at the neural level, already in a resting state EEG signal.
Article
This study conducted an experiment to test how the level of blame differs between an artificial intelligence (AI) and a human driver based on attribution theory and computers are social actors (CASA). It used a 2 (human vs. AI driver) x 2 (victim survived vs. victim died) x 2 (female vs. male driver) design. After reading a given scenario, participants (N = 284) were asked to assign a level of responsibility to the driver. The participants blamed drivers more when the driver was AI compared to when the driver was a human. Also, the higher level of blame was shown when the result was more severe. However, gender bias was found not to be significant when faulting drivers. These results indicate that the intention of blaming AI comes from the perception of dissimilarity and the seriousness of outcomes influences the level of blame. Implications of findings for applications and theory are discussed.
Article
In our daily lives, we need to predict and understand others’ behavior in order to navigate through our social environment. Predictions concerning other humans’ behavior usually refer to their mental states, such as beliefs or intentions. Such a predictive strategy is called ‘adoption of the intentional stance.’ In this paper, we review literature related to the concept of intentional stance from the perspectives of philosophy, psychology, human development, culture, and human-robot interaction. We propose that adopting the intentional stance might be a pivotal factor in facilitating social attunement with artificial agents. The paper first reviews the theoretical considerations regarding the intentional stance and examines literature related to the development of the intentional stance across life span. Subsequently, we discuss cultural norms as grounded in the intentional stance, and finally, we focus on the issue of adopting the intentional stance toward artificial agents, such as humanoid robots. At the dawn of the artificial intelligence era, the question of how – and also when – we predict and explain robots’ behavior by referring to mental states is of high interest. The paper concludes with a discussion on ethical consequences of adopting the intentional stance toward robots, and sketches future directions in research on this topic.
Article
This paper brings together a multi-disciplinary perspective from systems engineering, ethics, and law to articulate a common language in which to reason about the multi-faceted problem of assuring the safety of autonomous systems. The paper's focus is on the “gaps” that arise across the development process: the semantic gap, where normal conditions for a complete specification of intended functionality are not present; the responsibility gap, where normal conditions for holding human actors morally responsible for harm are not present; and the liability gap, where normal conditions for securing compensation to victims of harm are not present. By categorising these “gaps” we can expose with greater precision key sources of uncertainty and risk with autonomous systems. This can inform the development of more detailed models of safety assurance and contribute to more effective risk control.
Article
Moral philosophers and psychologists often assume that people judge morally lucky and morally unlucky agents differently, an assumption that stands at the heart of the Puzzle of Moral Luck. We examine whether the asymmetry is found for reflective intuitions regarding wrongness, blame, permissibility, and punishment judg- ments, whether people’s concrete, case-based judgments align with their explicit, abstract principles regarding moral luck, and what psychological mechanisms might drive the effect. Our experiments produce three findings: First, in within-subjects experiments favorable to reflective deliberation, the vast majority of people judge a lucky and an unlucky agent as equally blameworthy, and their actions as equally wrong and permissible. The philosophical Puzzle of Moral Luck, and the challenge to the very possibility of systematic ethics it is frequently taken to engender, thus simply do not arise. Second, punishment judgments are significantly more outcome- dependent than wrongness, blame, and permissibility judgments. While this constitutes evidence in favor of current Dual Process Theories of moral judgment, the latter need to be qualified: punishment and blame judgments do not seem to be driven by the same process, as is commonly argued in the literature. Third, in between-subjects experiments, outcome has an effect on all four types of moral judgments. This effect is mediated by negligence ascriptions and can ultimately be explained as due to differing probability ascriptions across cases.
Article
An important topic in the field of social and developmental psychology is how humans attribute mental traits and states to others. With the growing presence of robots in society, humans are confronted with a new category of social agents. This paper presents an empirical study demonstrating how psychological theory may be used for the human interpretation of robot behavior. Specifically, in this study we applied Weiner's Theory of Social Conduct as a theoretical background for studying attributions of agency and responsibility to NAO robots. Our results suggest that if a robot's failure appears to be caused by its (lack of) effort, as compared to its (lack of) ability, human observers attribute significantly more agency and, although to a lesser extent, more responsibility to the robot. However, affective and behavioral responses to robots differ in such cases as compared to reactions to human agents.
Article
This book is amongst the first academic treatments of the emerging debate on autonomous weapons. Autonomous weapons are capable, once programmed, of searching for and engaging a target without direct intervention by a human operator. Critics of these weapons claim that ‘taking the human out-of-the-loop’ represents a further step towards the de-humanisation of warfare, while advocates of this type of technology contend that the power of machine autonomy can potentially be harnessed in order to prevent war crimes. This book provides a thorough and critical assessment of these two positions. Written by a political philosopher at the forefront of the autonomous weapons debate, the book clearly assesses the ethical and legal ramifications of autonomous weapons, and presents a novel ethical argument against fully autonomous weapons.
Chapter
Moral luck is a puzzling aspect of our psychology: Why do we punish outcomes that were not intended (i.e. accidents)? Prevailing psychological accounts of moral luck characterize it as an accident or error, stemming either from a re-evaluation of the agent's mental state or from negative affect aroused by the bad outcome itself. While these models have strong evidence in their favor, neither can account for the unique influence of accidental outcomes on punishment judgments, compared with other categories of moral judgment. Why might punishment be particularly sensitive to moral luck? We suggest that such sensitivity is easily understood from the broader perspective of punishment's ultimate adaptive goal: Changing others? behavior by exploiting their capacity to learn. This pedagogical perspective accounts for the exceptional influence of outcomes on punitive sentiments and makes predictions for additional moderators of punishment. We review evidence supporting the pedagogical hypothesis of punishment and discuss fruitful directions for future research.
Article
Robotic warfare has now become a real prospect. One issue that has generated heated debate concerns the development of ‘Killer Robots’. These are weapons that, once programmed, are capable of finding and engaging a target without supervision by a human operator. From a conceptual perspective, the debate on Killer Robots has been rather confused, not least because it is unclear how central elements of these weapons can be defined. Offering a precise take on the relevant conceptual issues, the article contends that Killer Robots are best seen as executors of targeting decisions made by their human programmers. However, from a normative perspective, the execution of targeting decisions by Killer Robots should worry us. The article argues that what is morally bad about Killer Robots is that they replace human agency in warfare with artificial agency, a development which should be resisted. Finally, the article contends that the issue of agency points to a wider problem in just war theory, namely the role of moral rights in our normative reasoning on armed conflict.
Book
This book develops a pluralistic quality of will theory of responsibility, motivated by our ambivalence to real life cases of marginal agency, such as those with clinical depression, scrupulosity, psychopathy, autism, intellectual disability, and more. Our ambivalent responses suggest that such agents are responsible in some ways but not others. A tripartite theory is developed to account for this fact of our ambivalence via exploration of the appropriateness conditions of three distinct categories of our pan-cultural emotional responsibility responses (sentiments). The first type of responsibility, attributability, is about the quality of an agent's character, as expressed in various attitudes. The second type of responsibility, answerability, is about the quality of an agent's judgments of the worth of certain kinds of reasons. The third type of responsibility, accountability, is about the quality of an agent's regard for others, which itself implicates various empathic capacities. In Part 1 of the book, the tripartite theory is developed and defended. In Part 2 of the book, the tripartite theory's predictions about several specific marginal cases are tested, once certain empirical details about the nature of those agents have been filled in.
Article
IntroductionModern weapons of war have undergone precipitous technological change over the past generation and the future portends even greater advances. Of particular interest are so-called ‘autonomous weapon systems’ (henceforth, AWS), that will someday purportedly have the ability to make life and death targeting decisions ‘on their own.’ Many have strong moral intuitions against such weapons, and public concern over AWS is growing. A coalition of several non-governmental organizations, for example, has raised the alarm through their highly publicized ‘Campaign to Stop Killer Robots’ in an effort to enact an international ban on fully autonomous weapons.See Campaign to Stop Killer Robots at http://www.stopkillerrobots.org/. The views of the campaign are well represented by the work of its most publicly visible spokesperson, Noel Sharkey. See, for example, Sharkey (2010). Despite the strong and widespread sentiments against such weapons, however, proffered philosophical arguments aga ...