António Barata Lopes’s research while affiliated with Agrupamento Escolas MDS and other places

When speaking of moral judgment, we refer to a function of recognizing appropriate or condemnable actions and the possibility of choice between them by agents. Their ability to construct possible causal sequences enables them to devise alternatives in which choosing one implies setting aside others. This internal deliberation requires a cognitive ability, namely that of constructing counterfactual arguments. These serve not just to analyse possible futures, being prospective, but also to analyse past situations, by imagining the gains or losses resulting from alternatives to the actions actually carried out, given evaluative information subsequently known. Counterfactual thinking is in thus a prerequisite for AI agents concerned with Law cases, in order to pass judgement and, additionally, for evaluation of the ongoing governance of such AI agents. Moreover, given the wide cognitive empowerment of counterfactual reasoning in the human individual, namely in making judgments, the question arises of how the presence of individuals with this ability can improve cooperation and consensus in populations of otherwise self-regarding individuals. Our results, using Evolutionary Game Theory (EGT), suggest that counterfactual thinking fosters coordination in collective action problems occurring in large populations and has limited impact on cooperation dilemmas in which such coordination is not required.

We present a summary of research that we have conducted employing AI to better understand human morality. This summary adumbrates theoretical fundamentals and considers how to regulate development of powerful new AI technologies. The latter research aim is benevolent AI, with fair distribution of benefits associated with the development of these and related technologies, avoiding disparities of power and wealth due to unregulated competition. Our approach avoids statistical models employed in other approaches to solve moral dilemmas, because these are “blind” to natural constraints on moral agents, and risk perpetuating mistakes. Instead, our approach employs, for instance, psychologically realistic counterfactual reasoning in group dynamics. The present paper reviews studies involving factors fundamental to human moral motivation, including egoism vs. altruism, commitment vs. defaulting, guilt vs. non-guilt, apology plus forgiveness, counterfactual collaboration, among other factors fundamental in the motivation of moral action. These being basic elements in most moral systems, our studies deliver generalizable conclusions that inform efforts to achieve greater sustainability and global benefit, regardless of cultural specificities in constituents.

Intelligence and cognition are presently viewed as processes of evolution. This finding—as well as the detailed research into the dynamics of these processes—supports the idea that the various evolutionary moments are intertwined in a logic of a successive multi-level emergence.

When speaking of moral conscience, we are referring to a function of recognizing appropriate or condemnable action, and the possibility of choice between them. In fact, it would make no sense to talk about morals or ethics, if for each situation we had only one possible answer. Morality is justified because the agent can choose among possible actions. His ability to construct possible causal sequences enables him to devise alternatives in which choosing one implies setting aside the other. This typology of internal deliberation requires certain cognitive capacities, namely that of constructing counterfactual arguments. These serve not only to analyse possible futures, being prospective, but also to analyse past situations, by imagining the gains or losses resulting from imagining alternatives to the action actually carried out. Compared to social learning, where the subject can only mimic certain behaviours, the construction of counterfactuals is much richer and more fruitful. Thus, for machines to be equipped with effective moral capacity, it is necessary to equip them with the ability to construct and analyse counterfactual situations.

An intelligent agent will, inherently, be an autonomous agent. Assuming this thesis is pertinent, it becomes necessary to clarify the notion of autonomy and its prerequisites. Initially, the difficulties inherent in developing ways of thinking that make it effective must be acknowledged. In fact, most individuals deliberate and decide on concrete aspects of their lives yet are unable to do so critically enough. This requires a complex set of prerequisites to be met, which we make explicit. Among them is the ability to construct hypothetical counterfactual scenarios, which support the analysis of possible futures, thereby leading the subject to the construction of a non-standard identity, of his own preference and choice. In the realm of AI, the notions of genetic algorithms and emergence, allow for an engineered approximation of what is viewed as autonomy in humans. Indeed, a machine can follow trial and error procedures, finding unexpected solutions to problems itself, or in conjunction with other machines. In theory, though we are mindful of the difficulties inherent in the construction of autonomy, nothing in principle prevents machines from attaining it.

The fact that machines can recognize emotions, or even be programmed with something functionally similar to an emotion, does not mean that they exhibit moral behaviour. The laws defined by Isaac Asimov are of little use if a machine agent has to make decisions in complex scenarios. It must be borne in mind that morality is primarily a group phenomenon. It serves to regulate the relationship among individuals having different motivations regarding the cohesion and benefit of that group. Concomitantly, it moderates expectations about one another. It is necessary to make sure agents do not hide malevolent purposes, that they are capable of acknowledging errors and to act accordingly. One must begin somewhere, even without presently possessing a detailed knowledge of human morality, to the extent of programming ethical machines in full possession of all the functions of justification and argumentation that underlie decisions. This chapter will discuss the bringing out of a moral lexicon shareable by most cultures. The specific case of guilt and the capacity to recognize it is present in all cultures. It can be computer-simulated and can be a starting point for exploring this field.

Interpreting the cognitive development of Humanity as a liberating process, the role of a conceivable superintelligence is questioned. Admittedly, there is nowadays a kind of “arms race” which aims to build ever more flexible algorithms, with a view towards what is now dubbed Artificial General Intelligence (A.G.I.). There is a perception that leading this race will have immeasurable competitive advantages but first the technical difficulties of such an undertaking should be noted. Even highly sophisticated systems like AlphaGo are a long way from the general intelligence of a human being. On the other hand, there is an inscription of this aim in our cultural matrix, and this is as challenging as it is paradoxical. Firstly, in the name of our individual and collective freedom, we killed God. We are currently working hard to find forms of intelligence that surpass us, thereby enabling much more effective social control. Again, the analysis of the shaping mythology of our culture will help us spell out the problem, this time through the magical powers of the goddess Circe.

Looking into the nature and evolutionary History of Humanity, we find that symbiotic processes are far from new. Not only are they present in Biology, but also in our relationship with other animals and archaic machines, multipliers of force and speed. Only on the basis of a less than 25,000-year-old illusion do we perceive ourselves as exclusive holders of the top of the knowledge chain; but in this exclusivity we have always been accompanied by projections of transcendent beings, or expectations about extra-terrestrials. Until about 25,000 years ago we shared the planet with other hominids, with whom the Sapiens had close relationships. Neanderthals have disappeared, but not all of their genes. Thus, the concept of symbiosis occupies centre stage in the understanding of evolutionary cognition. This has not been seen as too problematic. However, as AI evolves, this may change. Such scenarios should convene citizens for informed debates on the topics and processes of scientific inquiry. Icarus is an example of the abuse of technologies engendered by Daedalus, and symbolizes the risks we face. But it does not have to be so: the problems faced will be better solved with more, never less, properly conducted science and technology.

Symbiotic processes have a special impact on children and young people. Born in a world of technological tools paraphernalia linked to the Internet and the most widespread media, they cannot even conceive of a life where they would not be permanently connected to the network. Traditional notions associated with privacy are thus questioned without much awareness. The impacts of fragmented information, of the way social networks summon reactivity and immediate emotional response, of the permanent presence of the other mediated by a smartphone, a tablet or a computer, are not yet thought out and conceived in all their consequences. However, the phenomena of scattered and diffuse identity and the emergence of behaviours intolerant to frustration are becoming increasingly evident. In a world where in each of those present in the network there constitute within themselves like one alter ego (or more), youths have difficulty in structuring a solid and differentiating identity, caving before the multiple pressures they are subjected to. Perhaps in the near future the notion of building a differentiated identity will not have the same pertinence it has today.

Since human moral decisions often have an emotional coating—either through empathy or antipathy—it is necessary to address the possibility of developing emotionally motivated machines. This is considered one of the limits of AI when doubting its progress. “Machines will never be emotionally motivated, for they have no endocrine system to endow them with emotions.” This thesis systematically ignores the role of emotions in humans, as well as what we should really expect from cognitive machines. First, we will highlight that, from a functional viewpoint, emotions play an anticipatory role, preparing possible responses for an organism. Then we will caution that emotions are not a human particularity, since there are many other species that have them too. Hence, there is nothing to prevent a machine from being able, in advance and using its power to conceive counterfactuals, to conjecture alternative possible answers. In the future we will not have fearful or sad computers, but there will be within them the role that fear or sadness play in our decision-making processes. Even based on what is already achievable today, we will soon have robots capable of interpreting and interacting with human emotions.