Figure - available from: Nature Human Behaviour
This content is subject to copyright. Terms and conditions apply.
Interaction with a real-world AI system amplifies human bias (n = 100) a, Experimental design. The experiment consisted of three stages. In stage 1, participants were presented with images featuring six individuals from different race and gender groups: a White man, a White woman, an Asian man, an Asian woman, a Black man and a Black woman. On each trial, participants selected the person who they thought was most likely to be a financial manager. In stage 2, for each trial, three images of financial managers generated by Stable Diffusion were randomly chosen and presented to the participants. In the control condition, participants were presented with three images of fractals instead. In stage 3, participants repeated the task from stage 1, allowing measurement of the change in participants’ choices before versus after exposure to the AI-generated images. b, The results revealed a significant increase in participants’ inclination to choose White men as financial managers after being exposed to AI-generated images, but not after being exposed to fractal neutral images (control). The error bars represent s.e.m. Face stimuli in a reproduced from ref. ⁵⁹ under a Creative Commons licence CC BY 4.0.
Source publication
Artificial intelligence (AI) technologies are rapidly advancing, enhancing human capabilities across various fields spanning from finance to medicine. Despite their numerous advantages, AI systems can exhibit biased judgements in domains ranging from perception to emotion. Here, in a series of experiments (n = 1,401 participants), we reveal a feedb...
Citations
... With generative AI's more reciprocal interaction style, understanding such interaction feedback loops is increasingly urgent. A recent study by Glickman and Sharot [13] provides the first evidence that AI biases can amplify human biases over time across domains such as perception, emotion, and social judgment. In one experiment, a generative AI system that over-represented (white) men as financial managers influenced users to make more biased judgments than they had before, illustrating how AI bias can amplify human bias. ...
Isolated perspectives have often paved the way for great scientific discoveries. However, many breakthroughs only emerged when moving away from singular views towards interactions. Discussions on Artificial Intelligence (AI) typically treat human and AI bias as distinct challenges, leaving their dynamic interplay and compounding potential largely unexplored. Recent research suggests that biased AI can amplify human cognitive biases, while well-calibrated systems might help mitigate them. In this position paper, I advocate for transcending beyond separate treatment of human and AI biases and instead focus on their interaction effects. I argue that a comprehensive framework, one that maps (compound human-AI) biases to mitigation strategies, is essential for understanding and protecting human cognition, and I outline concrete steps for its development.
... These loops amplify distortions over successive learning cycles, reducing information diversity and degrading response quality, neutrality, and factual accuracy [13,15]. Although individual biases may appear insignificant in a single response, their repeated reinforcement leads to long-term model drift and entrenched distortions within the AI knowledge base [15,16]. ...
... The AI then provides a response that aligns with its training data and previous user interactions [15]. If the user finds the response acceptable and engages with it, the AI interprets this as a sign of usefulness, reinforcing similar outputs in future interactions [16]. Over time, the user's engagement influences the AI's learning patterns, narrowing the range of responses and limiting exposure to diverse or opposing perspectives. ...
Large Language Models (LLMs) are critical tools for knowledge generation and decision-making in fields such as science, business, governance, and education. However, these models are increasingly prone to Bias, Misinformation, and Errors (BME) due to multi-level feedback loops that exacerbate distortions over iterative training cycles. This paper presents a comprehensive framework for understanding these feedback mechanisms-User-AI Interaction, Algorithmic Curation, and Training Data Feedback-as primary drivers of model drift and information quality decay. We introduce three novel metrics-Bias Amplification Rate (BAR), Echo Chamber Propagation Index (ECPI), and Information Quality Decay (IQD) Score-to quantify and track the impact of feedback-driven bias propagation. Simulations demonstrate how these metrics reveal evolving risks in LLMs over successive iterations. Our findings emphasize the urgency of implementing lifecycle-wide governance frameworks incorporating real-time bias detection, algorithmic fairness constraints, and human-in-the-loop verification to ensure the long-term reliability, neutrality, and accuracy of LLM-generated outputs.
... Critically, these human biases interact with algorithmic and systemic biases embedded in AI systems and datasets [45]. This interaction creates a harmful feedback loop that can amplify both types of biases: human biases influencing how users interact with AI systems, and AI biases potentially reinforcing human cognitive biases, negatively impacting the quality of learning and critical thinking [24] (See Fig. 4). ...
While generative artificial intelligence (Gen AI) increasingly transforms academic environments, a critical gap exists in understanding and mitigating human biases in AI interactions, such as anchoring and confirmation bias. This position paper advocates for metacognitive AI literacy interventions to help university students critically engage with AI and address biases across the Human-AI interaction workflows. The paper presents the importance of considering (1) metacognitive support with deliberate friction focusing on human bias; (2) bi-directional Human-AI interaction intervention addressing both input formulation and output interpretation; and (3) adaptive scaffolding that responds to diverse user engagement patterns. These frameworks are illustrated through ongoing work on "DeBiasMe," AIED (AI in Education) interventions designed to enhance awareness of cognitive biases while empowering user agency in AI interactions. The paper invites multiple stakeholders to engage in discussions on design and evaluation methods for scaffolding mechanisms, bias visualization, and analysis frameworks. This position contributes to the emerging field of AI-augmented learning by emphasizing the critical role of metacognition in helping students navigate the complex interaction between human, statistical, and systemic biases in AI use while highlighting how cognitive adaptation to AI systems must be explicitly integrated into comprehensive AI literacy frameworks.
... Conversely, Level III restores discriminability, achieving peak performance on NoXi-I (UAR: 89.8%; very good friend recall: 94.1%) and modest gains on NoXi-J (UAR: 68.7%). This could be due to intimate behaviours such as synchronized emotional expression, infectious laughter, conversational depth, and prolonged mutual gazing, which tend to occur more frequently between close friends [68,69]. 2) Dyadic Interaction Ablation Study: To investigate the factors that contribute to relationship recognition, we conducted an interaction ablation study by masking multimodal information during dyadic interactions. ...
Dyadic social relationships, which refer to relationships between two individuals who know each other through repeated interactions (or not), are shaped by shared spatial and temporal experiences. Current computational methods for modeling these relationships face three major challenges: (1) the failure to model asymmetric relationships, e.g., one individual may perceive the other as a friend while the other perceives them as an acquaintance, (2) the disruption of continuous interactions by discrete frame sampling, which segments the temporal continuity of interaction in real-world scenarios, and (3) the limitation to consider periodic behavioral cues, such as rhythmic vocalizations or recurrent gestures, which are crucial for inferring the evolution of dyadic relationships. To address these challenges, we propose AsyReC, a multimodal graph-based framework for asymmetric dyadic relationship classification, with three core innovations: (i) a triplet graph neural network with node-edge dual attention that dynamically weights multimodal cues to capture interaction asymmetries (addressing challenge 1); (ii) a clip-level relationship learning architecture that preserves temporal continuity, enabling fine-grained modeling of real-world interaction dynamics (addressing challenge 2); and (iii) a periodic temporal encoder that projects time indices onto sine/cosine waveforms to model recurrent behavioral patterns (addressing challenge 3). Extensive experiments on two public datasets demonstrate state-of-the-art performance, while ablation studies validate the critical role of asymmetric interaction modeling and periodic temporal encoding in improving the robustness of dyadic relationship classification in real-world scenarios. Our code is publicly available at: https://github.com/tw-repository/AsyReC.
... In fact, it is the interests of owners and operators that are behind all this. A recent study in cognitive science demonstrates how human interaction with AI mutually reinforces biases [23]. In contrast, no such reinforcement occurs in human-to-human interaction, which may indicate that people uncritically accept the AI model outputs. ...
... As the above study proves [23] very minor initial biases on the part of humans or AI can be significantly amplified by a positive feedback loop and produce a considerable effect in the end. Even more threatening is the deliberate use of such biases by the owner companies, which can completely disorient people, cause them to gradually lose their autonomy, and ultimately turn them into a resource that businesses and governments are fighting over. ...
The legal regulation of artificial intelligence is one of the most pressing and debated topics at the national and international levels. The rapid development of artificial intelligence can significantly change the existing reality and leads to fundamentally new challenges for lawmaking and law enforcement, in particular in the field of human rights. The main purpose of the article is to determine whether the new European legal instruments on artificial intelligence (in particular, the European Union’s AI Act and the Council of Europe’s Framework Convention on AI) reflect these technological threats and protect the personal autonomy of individuals. To achieve this goal, the article reveals the essence of personal autonomy and its significance for human rights and the legal system, as well as identifies the directions of the actual and potential impact of artificial intelligence on personal autonomy. The theoretical and methodological foundation of the study is Joseph Raz’s theory of personal autonomy which allows to identify the main problems and contradictions in the use of artificial intelligence and to shape proposals for responding to actual threats. Based on the idea of the fundamental role of personal autonomy, the article shows how the introduction of artificial intelligence, driven by the interests of particular actors, negatively affects the position, rights and capacities of individuals. In particular, the author identifies three directions of such influence: high-tech manipulation of people, distortion of their perception through myths and misconceptions, and formation of the appropriate online architecture and social norms. Based on the analysis of legal documents, two approaches to the regulation of artificial intelligence are identified. The first approach relegates personal autonomy to the periphery and suggests that problems should be solved through cooperation between governments and business by using risk assessment tools. This should result in ready-made solutions that are offered to people. The second human-centred approach emphasises the protection of personal autonomy. However, detailed norms within this approach have not yet been created, and their development requires theoretical elaborations. In this regard, the primary focus should be on preserving and improving the conditions of autonomy that are threatened by the misuse of artificial intelligence.
... Feedback inconsistency, subjectivity, and overreliance on prior interactions can reinforce biased behaviors in RL agents. Studies show that feedback loops between humans and AI can amplify biases, as humans often underestimate AI influence [16]. Additionally, simplifications in existing HITL-RL models fail to capture the nuances of human decision-making, leading to suboptimal performance in real-world applications [17]. ...
Reinforcement learning often faces challenges with reward misalignment, where agents optimize for given rewards but fail to exhibit the desired behaviors. This occurs when the reward function incentivizes proxy behaviors that diverge from the true objective. While human-in-the-loop (HIL) methods can help, they may exacerbate the problem, as humans are prone to biases that lead to inconsistent, subjective, or misaligned feedback, complicating the learning process. To address these issues, we propose two key contributions. First, we extend the use of zero-shot, off-the-shelf large language models (LLMs) for reward shaping beyond natural language processing (NLP) to continuous control tasks. By leveraging LLMs as direct feedback providers, we replace surrogate models trained on human feedback, which often suffer from the bias inherent in the feedback data it is trained on. Second, we introduce a hybrid framework (LLM-HFBF) that enables LLMs to identify and correct biases in human feedback while incorporating this feedback into the reward shaping process. The LLM-HFBF framework creates a more balanced and reliable system by addressing both the limitations of LLMs (e.g., lack of domain-specific knowledge) and human supervision (e.g., inherent biases). By enabling human feedback bias flagging and correction, our approach improves reinforcement learning performance and reduces reliance on potentially biased human guidance. Empirical experiments show that biased human feedback significantly reduces performance, with average episodic reward (AER) dropping from 28.472 in (unbiased approaches) to 7.039 (biased with conservative bias). In contrast, LLM-based approaches maintain a matching AER like unbiased feedback, even in custom edge case scenarios.
... For example, as outlined in the previous section, AI tools can influence the topics that people communicate about (Poddar et al., 2023), as well as influence the views that users communicate (Jakesch et al., 2023a(Jakesch et al., , 2023bWilliams-Ceci et al., 2024). Glickman and Sharot (2024) also found that when humans interact with AI tools, human biases are significantly more amplified compared to when humans interact with other humans. This is due to AI systems amplifying existing biases in a way which is subtle and unbeknownst to the user, meaning the user is unaware of the influence of the AI tool, making them more prone to internalising existing biases (Glickman & Sharot, 2024). ...
... Glickman and Sharot (2024) also found that when humans interact with AI tools, human biases are significantly more amplified compared to when humans interact with other humans. This is due to AI systems amplifying existing biases in a way which is subtle and unbeknownst to the user, meaning the user is unaware of the influence of the AI tool, making them more prone to internalising existing biases (Glickman & Sharot, 2024). If people who lack subject expertise rely on AI tools when mediating their communications (e.g. , these biases and errors, even if small, can proliferate over time. ...
The rapid adoption of commercial Generative Artificial Intelligence (Gen AI) products raises important questions around the impact this technology will have on our communicative interactions. This paper provides an analysis of some of the potential implications that Artificial Intelligence-Mediated Communication (AI-MC) may have on epistemic trust in online communications, specifically on social media. We argue that AI-MC poses a risk to epistemic trust being diminished in online communications on both normative and descriptive grounds. Descriptively, AI-MC seems to (roughly) lower levels of epistemic trust. Normatively, we argue that this brings about the following dilemma. On the one hand, there are at least some instances where we should epistemically trust AI-MC less, and therefore the reduction in epistemic trust is justified in these instances. On the other hand, there are also instances where we epistemically trust AI-MC less, but this reduction in epistemic trust is not justified, resulting in discrimination and epistemic injustice in these instances. The difficulty in knowing which of these two groups any instance of AI-MC belongs to brings about the AI-MC dilemma: We must choose between maintaining normal levels of epistemic trust and risking epistemic gullibility when reduced trust is justified, or adopting generally reduced epistemic trust and risking epistemic injustice when such reduced trust is unjustified. Navigating this choice between problematic alternatives creates a significant challenge for social media as an epistemic environment.
... ,40,67 . There is therefore an urgent ...
Emotions relate to climate change action in various ways. Here we elaborate on how the expansion of digital social networks and advances in artificial intelligence, ranging from recommender systems to generative AI, may affect the way people perceive and engage emotionally on climate change. We develop a simple framework that links individual and collective emotions, AI, and climate action, and suggest three critical areas in need of further investigation.
... One thought-provoking conjecture -supported by the simulated diffusion patterns reported here -is that once artificial agents are injected into a social system, contagion will spread wider. At the same time, because of the limited scope of this Report, it remains open to test the amplifier effect in real-world platforms where artificial and human decision-makers interact longitudinally [13], and to delimit in what conditions we can expect it to occur. ...
Recent advances in artificial intelligence have led to the proliferation of artificial agents in social contexts, ranging from education to online social media and financial markets, among many others. The increasing rate at which artificial and human agents interact makes it urgent to understand the consequences of human-machine interactions for the propagation of new ideas, products, and behaviors in society. Across two distinct empirical contexts, we find here that artificial agents lead to significantly faster and wider social contagion. To this end, we replicate a choice experiment previously conducted with human subjects by using artificial agents powered by large language models (LLMs). We use the experiment's results to measure the adoption thresholds of artificial agents and their impact on the spread of social contagion. We find that artificial agents tend to exhibit lower adoption thresholds than humans, which leads to wider network-based social contagions. Our findings suggest that the increased presence of artificial agents in real-world networks may accelerate behavioral shifts, potentially in unforeseen ways.
... When language games scale up to personalized coaching, large-group ideation, or even collective problem-solving, the possibility of overreliance on AI systems becomes increasingly significant (Glickman & Sharot, 2024). Users seeking practical answers or emotional support may compromise their own critical thinking, thereby transferring autonomy to opaque algorithmic processes. ...
The evolution of large language models (LLMs) toward artificial superhuman intelligence (ASI) hinges on data reproduction, a cyclical process in which models generate, curate and retrain on novel data to refine capabilities. Current methods, however, risk getting stuck in a data reproduction trap: optimizing outputs within fixed human-generated distributions in a closed loop leads to stagnation, as models merely recombine existing knowledge rather than explore new frontiers. In this paper, we propose language games as a pathway to expanded data reproduction, breaking this cycle through three mechanisms: (1) \textit{role fluidity}, which enhances data diversity and coverage by enabling multi-agent systems to dynamically shift roles across tasks; (2) \textit{reward variety}, embedding multiple feedback criteria that can drive complex intelligent behaviors; and (3) \textit{rule plasticity}, iteratively evolving interaction constraints to foster learnability, thereby injecting continual novelty. By scaling language games into global sociotechnical ecosystems, human-AI co-evolution generates unbounded data streams that drive open-ended exploration. This framework redefines data reproduction not as a closed loop but as an engine for superhuman intelligence.
