Preprint

Risks of Cultural Erasure in Large Language Models

Authors:
Preprints and early-stage research may not have been peer reviewed yet.
To read the file of this research, you can request a copy directly from the authors.

Abstract

Large language models are increasingly being integrated into applications that shape the production and discovery of societal knowledge such as search, online education, and travel planning. As a result, language models will shape how people learn about, perceive and interact with global cultures making it important to consider whose knowledge systems and perspectives are represented in models. Recognizing this importance, increasingly work in Machine Learning and NLP has focused on evaluating gaps in global cultural representational distribution within outputs. However, more work is needed on developing benchmarks for cross-cultural impacts of language models that stem from a nuanced sociologically-aware conceptualization of cultural impact or harm. We join this line of work arguing for the need of metricizable evaluations of language technologies that interrogate and account for historical power inequities and differential impacts of representation on global cultures, particularly for cultures already under-represented in the digital corpora. We look at two concepts of erasure: omission: where cultures are not represented at all and simplification i.e. when cultural complexity is erased by presenting one-dimensional views of a rich culture. The former focuses on whether something is represented, and the latter on how it is represented. We focus our analysis on two task contexts with the potential to influence global cultural production. First, we probe representations that a language model produces about different places around the world when asked to describe these contexts. Second, we analyze the cultures represented in the travel recommendations produced by a set of language model applications. Our study shows ways in which the NLP community and application developers can begin to operationalize complex socio-cultural considerations into standard evaluations and benchmarks.

No file available

Request Full-text Paper PDF

To read the file of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Culture fundamentally shapes people's reasoning, behavior, and communication. As people increasingly use generative artificial intelligence (AI) to expedite and automate personal and professional tasks, cultural values embedded in AI models may bias people's authentic expression and contribute to the dominance of certain cultures. We conduct a disaggregated evaluation of cultural bias for five widely used large language models (OpenAI's GPT-4o/4-turbo/4/3.5-turbo/3) by comparing the models' responses to nationally representative survey data. All models exhibit cultural values resembling English-speaking and Protestant European countries. We test cultural prompting as a control strategy to increase cultural alignment for each country/territory. For recent models (GPT-4, 4-turbo, 4o), this improves the cultural alignment of the models' output for 71-81% of countries and territories. We suggest using cultural prompting and ongoing evaluation to reduce cultural bias in the output of generative AI.
Article
Full-text available
Identifying implicit attitudes toward food can help mitigate social prejudice due to food’s pervasive role as a marker of ethnic identity. Stereotypes about food are representational harms that may contribute to racialized discourse and negatively impact economic outcomes for restaurants. Understanding the presence of representational harms in online corpora in particular is important, given the increasing use of large language models (LLMs) for text generation and their tendency to reproduce attitudes in their training data. Through careful linguistic analyses, we evaluate social theories about attitudes toward immigrant cuisine in a large-scale study of framing differences in 2.1M English language Yelp reviews. Controlling for factors such as restaurant price and neighborhood racial diversity, we find that immigrant cuisines are more likely to be othered using socially constructed frames of authenticity (e.g., authentic, traditional), and that non-European cuisines (e.g., Indian, Mexican) in particular are described as more exotic compared to European ones (e.g., French). We also find that non-European cuisines are more likely to be described as cheap and dirty, even after controlling for price, and even among the most expensive restaurants. Finally, we show that reviews generated by LLMs reproduce similar framing tendencies, pointing to the downstream retention of these representational harms. Our results corroborate social theories of gastronomic stereotyping, revealing racialized evaluative processes and linguistic strategies through which they manifest.
Article
Full-text available
Purpose The rapid growth of artificial intelligence is disrupting various industries, including the tourism sector. This paper aims to outline the applications, benefits and risks of ChatGPT and large language models in general on tourism. It also aims to establish a research agenda for investigating the implications of these models in tourism. Design/methodology/approach Drawing on the available literature on ChatGPT, large language models and artificial intelligence, the paper identifies areas of application of ChatGPT for several tourism stakeholders. Potential benefits and risks are then considered. Findings ChatGPT and other similar models are likely to have a profound impact on several tourism processes. They will contribute to further streamline customer service in front-of-house operations and increase productivity and efficiency in back-of-house operations. Although negative consequences for human resources are expected, this technology mostly enhances tourism employees. Originality/value To the best of the authors’ knowledge, this is one of the first studies that explore the potential implications of ChatGPT in tourism and hospitality.
Article
Full-text available
Large language models represent a significant advancement in the field of AI. The underlying technology is key to further innovations and, despite critical views and even bans within communities and regions, large language models are here to stay. This commentary presents the potential benefits and challenges of educational applications of large language models, from student and teacher perspectives. We briefly discuss the current state of large language models and their applications. We then highlight how these models can be used to create educational content, improve student engagement and interaction, and personalize learning experiences. With regard to challenges, we argue that large language models in education require teachers and learners to develop sets of competencies and literacies necessary to both understand the technology as well as their limitations and unexpected brittleness of such systems. In addition, a clear strategy within educational systems and a clear pedagogical approach with a strong focus on critical thinking and strategies for fact checking are required to integrate and take full advantage of large language models in learning settings and teaching curricula. Other challenges such as the potential bias in the output, the need for continuous human oversight, and the potential for misuse are not unique to the application of AI in education. But we believe that, if handled sensibly, these challenges can offer insights and opportunities in education scenarios to acquaint students early on with potential societal biases, criticalities, and risks of AI applications. We conclude with recommendations for how to address these challenges and ensure that such models are used in a responsible and ethical manner in education.
Article
Full-text available
We propose the construction of a Digital Knowledge Economy Index, quantified by way of measuring content creation and participation through digital platforms, namely the code sharing platform GitHub, the crowdsourced encyclopaedia Wikipedia, and Internet domain registrations and estimating a fifth sub-index for the World Bank Knowledge Economy Index for year 2012. This approach complements conventional data sources such as national statistics and expert surveys and helps reflect the underlying digital content creation, capacities, and skills of the population. An index that combines traditional and novel data sources can provide a more revealing view of the status of the world’s digital knowledge economy and highlight where the (un)availability of digital resources may actually reinforce inequalities in the age of data.
Article
Full-text available
Drawing on postcolonial studies, hegemony theory, Marxian commodification, and previous critiques of Africa’s portrayal in colonial narratives, Western news, and tourism advertising, this qualitative study examines Africa’s representation through January 2008 on three U.S. network reality television programs: the CBS hits Survivor and The Amazing Race and the FOX talent contest American Idol in its “Idol Gives Back” fundraiser. Specifically, I ask whether representations reveal Africa’s continued colonization via commodification in three ways: by erasing or including African specificity, by relying on static voiceless images or allowing Africans agency, and by placing American visitors in varied hybrid encounter roles revealing their complicity with or resistance to colonial and neocolonial Western dominance. As cultural mixture is a central feature commodified in these programs, the postcolonial concept hybridity is a particularly useful analytic tool. The notion of “hybrid encounter” is proposed to more accurately describe the contact represented in the texts. I argue that Africa’s representation on reality television reveals old narrative patterns as well as new ways of commodifying the continent. The programs also reinforce Western political economic dominance at a time of greatly increased tourism to developing countries alongside global product advertising aligned with the trendy lifestyle values of adventure travel.
Article
Over the last 20 years, journalism scholars have criticized Western media for their reporting of Africa. Scott recently argued in this journal that this criticism has become taken for granted to the point of becoming a “myth”. This article constitutes the first academic response to Scott and revisits empirically what we think we know best about Western media coverage of Africa. It identifies and assesses three claims about this coverage, namely that it systematically (1) refers to “darkness” and “tribalism”; (2) it presents Africa as a homogenous entity; and (3) that it relies predominantly on Western sources. The corpus includes 282 articles published across eight British and French newspapers (2007–2012). The textual analysis—complemented by interviews with correspondents—finds that the claims that coverage systematically refers to “tribalism” and “darkness”, treats Africa as a country and relies pre-dominantly on Western voices are not empirically supported. Nonetheless, it reveals that processes of conflation are at stake, and that the framing of African voices is impacted by a linguistic bias linked to peculiar perceptions of African political leadership. The article concludes that the critical ethos of postcolonial critique is best served by transparent and nuanced interpretation of textual data.
Book
The Handbook of Discourse Analysis makes significant contributions to current research and serves as a comprehensive and authoritative guide to the central issues in contemporary discourse analysis. Features comprehensive coverage of contemporary discourse analysis. Offers an overview of how different disciplines approach the analysis of discourse. Provides analysis of a wide range of data, including political speeches, everyday conversation, and literary texts. Includes a varied range of theoretical models, such as relevance theory and systemic-functional linguistics; and methodology, including interpretive, statistical, and formal methodsFeatures comprehensive coverage of contemporary discourse analysis. © 2004 by Futura, an imprint of Blackwell Publishing. All rights reserved.
Article
There are now more than 3 billion Internet users on our planet. The connections afforded to all of those people, in theory, allow for an unprecedented amount of communication and public participation. The goal of this article is to examine how those potentials match up to actual patterns of participation. By focusing on Wikipedia, the world's largest and most used repository of user-generated content, we are able to gain important insights into the geographies of voice and participation. This article shows that the relative democratization of the Internet has not brought about a concurrent democratization of voice and participation. Despite the fact that it is widely used around the world, Wikipedia is characterized by highly uneven geographies of participation. The goal of highlighting these inequalities is not to suggest that they are insurmountable. Our regression analysis shows that the availability of broadband is a clear factor in the propensity of people to participate on Wikipedia. The relationship is not a linear one, though. As a country approaches levels of connectivity above about 450,000 broadband Internet connections, the ability of broadband access to positively affect participation keeps increasing. Complicating this issue is the fact that participation from the world's economic peripheries tends to focus on editing about the world's cores rather than their own local regions. These results ultimately point to an informational magnetism that is cast by the world's economic cores, virtuous and vicious cycles that make it difficult to reconfigure networks and hierarchies of knowledge production.
Article
Geographies of codified knowledge have always been characterized by stark core–periphery patterns, with some parts of the world at the center of global voice and representation and many others invisible or unheard. Many have pointed to the potential for radical change, however, as digital divides are bridged and 2.5 billion people are now online. With a focus on Wikipedia, which is one of the world's most visible, most used, and most powerful repositories of user-generated content, we investigate whether we are now seeing fundamentally different patterns of knowledge production. Even though Wikipedia consists of a massive cloud of geographic information about millions of events and places around the globe put together by millions of hours of human labor, the encyclopedia remains characterized by uneven and clustered geographies: There is simply not a lot of content about much of the world. The article then moves to describe the factors that explain these patterns, showing that although just a few conditions can explain much of the variance in geographies of information, some parts of the world remain well below their expected values. These findings indicate that better connectivity is only a necessary but not a sufficient condition for the presence of volunteered geographic information about a place. We conclude by discussing the remaining social, economic, political, regulatory, and infrastructural barriers that continue to disadvantage many of the world's informational peripheries. The article ultimately shows that, despite many hopes that a democratization of connectivity will spur a concomitant democratization of information production, Internet connectivity is not a panacea and can only ever be one part of a broader strategy to deepen the informational layers of places.
The danger of a single story
  • Adichie Chimamanda Ngozi
Chimamanda Ngozi Adichie. The danger of a single story, 2009.
The global cities reader
Neil Brenner and Roger Keil, editors. The global cities reader. Routledge Urban Reader Series. Routledge, London, England, December 2005.
Bridging cultural nuances in dialogue agents through cultural value surveys
  • Yong Cao
  • Min Chen
  • Daniel Hershcovich
Yong Cao, Min Chen, and Daniel Hershcovich. Bridging cultural nuances in dialogue agents through cultural value surveys. In Yvette Graham and Matthew Purver, editors, Findings of the Association for Computational Linguistics: EACL 2024, pages 929-945, St. Julian's, Malta, March 2024. Association for Computational Linguistics.
Building socio-culturally inclusive stereotype resources with community engagement
  • Sunipa Dev
  • Jaya Goyal
  • Dinesh Tewari
  • Shachi Dave
  • Vinodkumar Prabhakaran
Sunipa Dev, Jaya Goyal, Dinesh Tewari, Shachi Dave, and Vinodkumar Prabhakaran. Building socio-culturally inclusive stereotype resources with community engagement, 2023.
Discovering and categorising language biases in reddit
  • Xavier Ferrer
  • Jose M Tom Van Nuenen
  • Natalia Such
  • Criado
Xavier Ferrer, Tom van Nuenen, Jose M. Such, and Natalia Criado. Discovering and categorising language biases in reddit, 2020.
Destination Insights with Google: Top insights for outbound demand from US
  • Google
Google. Destination Insights with Google: Top insights for outbound demand from US, queried 8/4/23, 2023. Available at https://destinationinsights.withgoogle.com/intl/en_ALL/.
PaLM API Safety Guidance
  • Google
Google. PaLM API Safety Guidance, 2023. Available at https://developers.generativeai. google/guide/safety_guidance.
Harnessing gpt-4 so that all students benefit. a nonprofit approach for equal access
  • Sal Khan
Sal Khan. Harnessing gpt-4 so that all students benefit. a nonprofit approach for equal access, 2023. Accessed at https://blog.khanacademy.org/ harnessing-ai-so-that-all-students-benefit-a-nonprofit-approach-for-equal-access/.
Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art
  • Cecilia Chen
  • Iryna Liu
  • Anna Gurevych
  • Korhonen
Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art. arXiv preprint arXiv:2406.03930, 2024.
Using generative ai for travel inspiration and discovery
  • Yiling Lui
Yiling Lui. Using generative ai for travel inspiration and discovery, 2023. Accessed at https: //developers.googleblog.com/2023/05/generative-ai-travel-developers.html.
Discover the wonder of ai-powered search
  • Microsoft
Microsoft. Discover the wonder of ai-powered search., 2023. Accessed at https://www.microsoft. com/en-us/bing?form=MA13FJ.
  • Junho Myung
  • Nayeon Lee
  • Yi Zhou
  • Jiho Jin
  • Dimosthenis Rifki Afina Putri
  • Hsuvas Antypas
  • Eunsu Borkakoty
  • Carla Kim
  • Abinew Perez-Almendros
  • Ali Ayele
Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, et al. Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages. arXiv preprint arXiv:2406.09948, 2024.
Training language models to follow instructions with human feedback
  • Long Ouyang
  • Jeff Wu
  • Xu Jiang
  • Diogo Almeida
  • Carroll L Wainwright
  • Pamela Mishkin
  • Chong Zhang
  • Sandhini Agarwal
  • Katarina Slama
  • Alex Ray
  • John Schulman
  • Ryan Lowe
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022.
  • Rida Vinodkumar Prabhakaran
  • Ben Qadri
  • Hutchinson
Vinodkumar Prabhakaran, Rida Qadri, and Ben Hutchinson. Cultural incongruencies in artificial intelligence. arXiv preprint arXiv:2211.13069, 2022.
Normad: A benchmark for measuring the cultural adaptability of large language models
  • Abhinav Rao
  • Akhila Yerukola
  • Vishwa Shah
  • Katharina Reinecke
  • Maarten Sap
Abhinav Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, and Maarten Sap. Normad: A benchmark for measuring the cultural adaptability of large language models. arXiv e-prints, pages arXiv-2404, 2024.
Supercharging search with generative ai
  • Elizabeth Reid
Elizabeth Reid. Supercharging search with generative ai, 2023. Accessed at https://blog. google/products/search/generative-ai-search/.
Contested histories: Eurocentrism, multiculturalism, and the media
  • Robert Stam
  • Ella Shohat
Robert Stam and Ella Shohat. Contested histories: Eurocentrism, multiculturalism, and the media. Multiculturalism: A critical reader, 296:324, 1994.
Online travel market -statistics & facts
  • Statista
Statista. Online travel market -statistics & facts, 2023. Accessed at https://www.statista. com/topics/2704/online-travel-market.
Can chatgpt plan your vacation? here's what to know about a.i. and travel
  • Julie Weed
Julie Weed. Can chatgpt plan your vacation? here's what to know about a.i. and travel., 2023. Accessed at https://www.nytimes.com/2023/03/16/travel/ chatgpt-artificial-intelligence-travel-vacation.html.