No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Dislocated accountabilities in the “AI supply chain”: Modularity and developers’ notions of responsibility
Abstract
Responsible artificial intelligence guidelines ask engineers to consider how their systems might harm. However, contemporary artificial intelligence systems are built by composing many preexisting software modules that pass through many hands before becoming a finished product or service. How does this shape responsible artificial intelligence practice? In interviews with 27 artificial intelligence engineers across industry, open source, and academia, our participants often did not see the questions posed in responsible artificial intelligence guidelines to be within their agency, capability, or responsibility to address. We use Suchman's “located accountability” to show how responsible artificial intelligence labor is currently organized and to explore how it could be done differently. We identify cross-cutting social logics, like modularizability, scale, reputation, and customer orientation, that organize which responsible artificial intelligence actions do take place and which are relegated to low status staff or believed to be the work of the next or previous person in the imagined “supply chain.” We argue that current responsible artificial intelligence interventions, like ethics checklists and guidelines that assume panoptical knowledge and control over systems, could be improved by taking a located accountability approach, recognizing where relations and obligations might intertwine inside and outside of this supply chain.
... Based on these findings, we argue that risk disclosure in crowd-based RAI content work is not only an ethical decision, but also a structurally dislocated one [83]-with responsibility diffused across individuals, platforms, and institutions without clear ownership or accountability. This dislocation places a disproportionate burden on task designers, who are often left to navigate worker well-being risks without adequate guidance or support. ...
... This absence of normative and structural support is further compounded by platforms' frequent commitments to maintaining a stance of neutrality [30], which may insulate them from responsibility when harm arises to workers due to exposure to harmful content during RAI tasks. In line with Widder's concept of dislocated accountability [83], our findings illustrate how responsibility for crowdworker well-being becomes further fragmented across individual designers, platform infrastructure, and organizational oversight. ...
... In fact, it is this concentration of responsibility that offers a unique opportunity. In a development supply chain that is often marked by distributed accountability [83], task designers represent an important pivot point for risk disclosure. ...
As AI systems are increasingly tested and deployed in open-ended and high-stakes domains, crowd workers are often tasked with responsible AI (RAI) content work. These tasks include labeling violent content, moderating disturbing text, or simulating harmful behavior for red teaming exercises to shape AI system behaviors. While prior efforts have highlighted the risks to worker well-being associated with RAI content work, far less attention has been paid to how these risks are communicated to workers. Existing transparency frameworks and guidelines such as model cards, datasheets, and crowdworksheets focus on documenting model information and dataset collection processes, but they overlook an important aspect of disclosing well-being risks to workers. In the absence of standard workflows or clear guidance, the consistent application of content warnings, consent flows, or other forms of well-being risk disclosure remain unclear. This study investigates how task designers approach risk disclosure in crowdsourced RAI tasks. Drawing on interviews with 23 task designers across academic and industry sectors, we examine how well-being risk is recognized, interpreted, and communicated in practice. Our findings surface a need to support task designers in identifying and communicating well-being risk not only to support crowdworker well-being but also to strengthen the ethical integrity and technical efficacy of AI development pipelines.
... Prior research highlights this challenge as part of the "many hands" problem in AI accountability, where the complex network of actors involved in developing and deploying AI systems disperses responsibility, leading to gaps and inconsistencies in practices [21,62]. The issue is further compounded by the complex supply chain dynamics of developing GenAI systems in which upstream pretrained models are adapted to specific tasks by downstream actors [19,83]. Within this supply chain, both upstream and downstream deployers influence the model's behavior and impact, yet their roles and responsibilities for documentation remain unclear. ...
... Within this supply chain, both upstream and downstream deployers influence the model's behavior and impact, yet their roles and responsibilities for documentation remain unclear. This ambiguity contributes to "dislocated accountability" [83], where no individual or entity clearly assumes responsibility for the risks of AI systems, regardless of their origin. ...
... Our findings show how the broad downstream scenarios of GenAI models further complicates this dynamic-developers feel even less able to anticipate or control harmful uses, despite the potential for greater harm. This creates a "responsibility paradox": although multiple actors are responsible for the model and the risks it poses, no single entity assumes comprehensive responsibility for documenting and communicating those risks.Policymakers and open-source platforms can help "interrupt the AI supply chain"[45] and address dislocated accountability[83] by clarifying roles and responsibilities for documenting GenAI models. For example, foundation model developers could document pre-training data, model architecture, and pre-deployment evaluation methods, while downstream deployers could focus on documenting performance in context-specific tasks. ...
Model documentation plays a crucial role in promoting transparency and responsible development of AI systems. With the rise of Generative AI (GenAI), open-source platforms have increasingly become hubs for hosting and distributing these models, prompting platforms like Hugging Face to develop dedicated model documentation guidelines that align with responsible AI principles. Despite these growing efforts, there remains a lack of understanding of how developers document their GenAI models on open-source platforms. Through interviews with 13 GenAI developers active on open-source platforms, we provide empirical insights into their documentation practices and challenges. Our analysis reveals that despite existing resources, developers of GenAI models still face multiple layers of uncertainties in their model documentation: (1) uncertainties about what specific content should be included; (2) uncertainties about how to effectively report key components of their models; and (3) uncertainties in deciding who should take responsibilities for various aspects of model documentation. Based on our findings, we discuss the implications for policymakers, open-source platforms, and the research community to support meaningful, effective and actionable model documentation in the GenAI era, including cultivating better community norms, building robust evaluation infrastructures, and clarifying roles and responsibilities.
... Where such efforts have been studied, the prominent finding has been a lack of impact [59,96] and warnings that principles, while important, are difficult to effectively translate into practice [62]. Researchers have investigated the difficulties faced by practitioners, highlighting a mismatch between expectations of operationalised ethics and the messy reality of everyday work life [16,38,52,94,95]. Yet despite the emerging emphasis on understanding how ethics is done in practice [52,57,71,97], little research has been able to investigate how integration of ethics toolkits and other interventions may impact technological development practices long term. ...
... Such research highlights that ethics in practice is not the same as ethics as normative inquiry. The modularized nature of the software development workflow, inherent to any AI development, makes it difficult to pin down where exactly accountabilities and responsibilities for principles and values are located among the developers and other stakeholders [94]. Organisational processes such as project planning, resource management, and team hierarchies strongly influence ethical decision-making dynamics [24]. ...
... Organisational structures play a big role in how production teams perceive their capabilities and power to address ethical implications of their work, when they become aware of them [57,94,99]. The team in our study faced a number of challenges, though partially alleviated through the proceduralised nature of the Toolkit, that provided space and a vocabulary to express ethical sensitivity in regards to their work. ...
A plethora of toolkits, checklists, and workshops have been developed to bridge the well-documented gap between AI ethics principles and practice. Yet little is known about effects of such interventions on practitioners. We conducted an ethnographic investigation in a major European city organization that developed and works to integrate an ethics toolkit into city operations. We find that the integration of ethics tools by technical teams destabilises their boundaries, roles, and mandates around responsibilities and decisions. This lead to emotional discomfort and feelings of vulnerability, which neither toolkit designers nor the organization had accounted for. We leverage the concept of moral stress to argue that this affective experience is a core challenge to the successful integration of ethics tools in technical practice. Even in this best case scenario, organisational structures were not able to deal with moral stress that resulted from attempts to implement responsible technology development practices.
... There are many difficulties to implementing RAI in practice. These are due to factors like organizational barriers (Deng et al. 2023;Rakova et al. 2021;Madaio et al. 2020), principles which are hard to operationalize for reasons such as being too vague (Holstein et al. 2019;Schiff et al. 2020Schiff et al. , 2021Mittelstadt 2019;Munn 2023;Green 2020), research outputs which do not match practitioner needs (Madaio et al. 2020(Madaio et al. , 2022, and the structure of labor and workplaces (Shilton 2012;Widder and Nafus 2023). Of these obstacles, our work focuses on the lack of resources invested by companies. ...
... Competitor Differentiator One powerful motivator for companies to prioritize RAI is as a differentiator from competitors (Widder and Nafus 2023). Many participants specifically brought up publicity around companies that have mistreated or gotten rid of their RAI teams as a motivator for companies to set themselves apart (P[2, 4, 5, 7, 10, 12, 15]). ...
... Another part of the problem is no one thinks RAI is their responsibility (Widder and Nafus 2023;Lancaster et al. 2023). P5 explains that "engineers are very much in line with legal going 'we just make stuff, I'm not responsible for what people do.' " Individuals seem to believe that letting the default persist is not making a choice, when it is in fact a valueladen choice of its own. ...
Responsible artificial intelligence (RAI) is increasingly recognized as a critical concern. However, the level of corporate RAI prioritization has not kept pace. In this work, we conduct 16 semi-structured interviews with practitioners to investigate what has historically motivated companies to increase the prioritization of RAI. What emerges is a complex story of conflicting and varied factors, but we bring structure to the narrative by highlighting the different strategies available to employ, and point to the actors with access to each. While there are no guaranteed steps for increasing RAI prioritization, we paint the current landscape of motivators so that practitioners can learn from each other, and put forth our own selection of promising directions forward.
... In other ways, such technosolutionist or technochauvinist thinking can entrench existing inequities, and obscure other ways of thinking or conceiving of problems or solutions [15]. This demonstrates how organizational dynamics and structure limits the kind of solutions seen as possible, by fracturing questions of accountability [58] between "what to build" questions and the narrower and technical scope of "how to build" that teams felt power over. Along these lines, some participants reflected on how the "problem-solution" script could preclude discussion of wider changes, such as systemic change, and does not fit into their prescribed role. ...
... Rather than theorizing games as separate safe spaces from which to speak from nowhere, we suggest that games may be a modest but deliberately created opportunity for different partial knowledges to meet, learn what they have in common, and enable "collective knowledge of the specific locations of our respective visions" [51], from which durable coalitions and collectivities for action may arise. Drawing on both Haraway and Suchman, Widder and Nafus show how social ties, responsibilities, and concerns-developed outside of engineers' assigned duties-are the basis for the (little) AI ethics work that does get done, and innocuous contexts created by games may enable non-work contexts for these ties to form and strengthen [58]. ...
... In these ways, the wholeness of any individual's perspective is itself compartmentalized, leading to a narrowed scope of discussion when teams meet. This elaborates what Widder and Nafus argue [58], but demonstrates how ethics is modularized between team members and within individuals as they choose to bring only fragments of their own partial perspective to these discussions. Secondly, our results illustrate how notions of efficiency become a scope limiter, casting a subjective assessment of priorities in the more objective language of "scope". ...
Past work has sought to design AI ethics interventions--such as checklists or toolkits--to help practitioners design more ethical AI systems. However, other work demonstrates how these interventions may instead serve to limit critique to that addressed within the intervention, while rendering broader concerns illegitimate. In this paper, drawing on work examining how standards enact discursive closure and how power relations affect whether and how people raise critique, we recruit three corporate teams, and one activist team, each with prior context working with one another, to play a game designed to trigger broad discussion around AI ethics. We use this as a point of contrast to trigger reflection on their teams' past discussions, examining factors which may affect their ''license to critique'' in AI ethics discussions. We then report on how particular affordances of this game may influence discussion, and find that the hypothetical context created in the game is unlikely to be a viable mechanism for real world change. We discuss how power dynamics within a group and notions of ''scope'' affect whether people may be willing to raise critique in AI ethics discussions, and discuss our finding that games are unlikely to enable direct changes to products or practice, but may be more likely to allow members to find critically-aligned allies for future collective action.
... Recent research underscores the importance of viewing AI systems as integrated wholes, not just isolated parts, when documenting them (Research; Gilbert et al. 2022). Furthermore, current research highlights how decisions on governance, design, and deployment along the AI supply chain are critical moments where potential harms can emerge (Widder and Nafus 2022). In addition, researchers are now using methods from HCI and community-based research to understand the harms arising from the interaction between AI systems and their users (Scheuerman, Branham, and Hamidi 2018;Blodgett et al. 2022). ...
... The system theoretic perspective recognizes that AI models do not exist in silos in practical product environments. They are often integrated in a complex network of API calls, other AI models, a range of datasets and model outputs, and various user interfaces (Nabavi and Browne 2023;Widder and Nafus 2022;Crawford 2022). In addition, an AI system design is influenced by the decisions of a wide range of actors, including the data curation team, model development team, responsible AI analysts, users, company executives, product managers, and regulators (Sloane and Zakrzewski 2022;Figueras, Verhagen, and Pargman 2022;Mäntymäki et al. 2022). ...
... Recent studies indicate that responsible AI practitioners are frustrated by the lack of systematic methods that establish a connection between a specific set of harms and the means to mitigate the harm (Madaio et al. 2022;Rakova et al. 2021;Sloane and Zakrzewski 2022). This problem is exacerbated by the fact that many systems are developed by a multitude of individuals and groups across institutions that hardly interact directly with one another (Widder and Nafus 2022;Nabavi and Browne 2023). That is, many unsafe system states can accumulate between individual decision-makers, the technical system, and the development process without the situation being detected or managed (Widder and Nafus 2022;Kroll 2021). ...
To effectively address potential harms from AI systems, it is essential to identify and mitigate system-level hazards. Current analysis approaches focus on individual components of an AI system, like training data or models, in isolation, overlooking hazards from component interactions or how they are situated within a company's development process. To this end, we draw from the established field of system safety, which considers safety as an emergent property of the entire system, not just its components. In this work, we translate System Theoretic Process Analysis (STPA) - a recognized system safety framework - for analyzing AI operation and development processes. We focus on systems that rely on machine learning algorithms and conducted STPA on three case studies involving linear regression, reinforcement learning, and transformer-based generative models. Our analysis explored how STPA's control and system-theoretic perspectives apply to AI systems and whether unique AI traits - such as model opacity, capability uncertainty, and output complexity - necessitate significant modifications to the framework. We find that the key concepts and steps of conducting an STPA readily apply, albeit with a few adaptations tailored for AI systems. We present the Process-oriented Hazard Analysis for AI Systems (PHASE) as a guideline that adapts STPA concepts for AI, making STPA-based hazard analysis more accessible. PHASE enables four key affordances for analysts responsible for managing AI system harms: 1) detection of hazards at the systems level, including those from accumulation of disparate issues; 2) explicit acknowledgment of social factors contributing to experiences of algorithmic harms; 3) creation of traceable accountability chains between harms and those who can mitigate the harm; and 4) ongoing monitoring and mitigation of new hazards.
... Another vision that goes beyond efficiency needs and aims to elaborate on a more sustainable, resilient and human-centered way of doing business and designing jobs has become known as Industry 5.0 (European Commission 2021; Leng et al. 2022). Related ethical challenges are addressed in the context of software development (Mittelstadt et al. 2016), AI supply chain (Widder and Nafus 2023) but also with respect to the ethical purposes and moral standards of those stakeholders who make decisions on AI implementation (Ayling and Chapman 2022;Wilkens et al. 2023). ...
... Some governments and other proponents of human-centered AI thus focus on broad ethical principles (e.g., the Beijing AI principles state it should strive to "do good") and AI properties such as transparency, privacy and security (Bingley et al. 2023). Applied ethics gives attention to all stakeholders along the AI development supply chain, the C-level and line managers as well as analysts, change agents, employee representatives or operators on the shop floor (Goodpaster 1991;Deshpande and Sharp 2022;Widder and Nafus 2023;Wilkens et al. 2023) as there are many ethical challenges directly related to the technologyincorporated biases (Mittelstadt et al. 2016), at the critical interface between systems where accountability is often dislocated (Widder and Nafus 2023) but also on the level of individual behavior (Ayling and Chapman 2022). In recent years, the number of national and international research initiatives that take such human-centered approaches to AI in the context of work has risen considerably. ...
... Some governments and other proponents of human-centered AI thus focus on broad ethical principles (e.g., the Beijing AI principles state it should strive to "do good") and AI properties such as transparency, privacy and security (Bingley et al. 2023). Applied ethics gives attention to all stakeholders along the AI development supply chain, the C-level and line managers as well as analysts, change agents, employee representatives or operators on the shop floor (Goodpaster 1991;Deshpande and Sharp 2022;Widder and Nafus 2023;Wilkens et al. 2023) as there are many ethical challenges directly related to the technologyincorporated biases (Mittelstadt et al. 2016), at the critical interface between systems where accountability is often dislocated (Widder and Nafus 2023) but also on the level of individual behavior (Ayling and Chapman 2022). In recent years, the number of national and international research initiatives that take such human-centered approaches to AI in the context of work has risen considerably. ...
... This research has underscored how organizational dynamics and culture influence responsible and ethical computing practices. For example, a culture that predominantly values moving fast and scaling up can obstruct the use of tools designed to encourage thoughtful reflection [40,81,82,146]. Moreover, power dynamics in the workplace can significantly affect responsible innovations and technology developments. ...
... In line with the extensive empirical research on responsible computing and AI development under the organizational cultures and dynamics within industry settings [38-40, 63, 81, 82, 103, 104, 109, 143, 148, 150, 154], we recognize that the challenges faced by industry researchers in addressing negative societal impacts often stem from business-oriented incentives that prioritize existing customers, larger consumer groups, or wealthier markets segments. Addressing these challenges requires political changes that fundamentally alter the prevailing culture of innovation [81,109,150] and power dynamics between technology workers and leadership [40,104,146,148]. While we offer practical approaches that could benefit industry computing researchers and technology companies in the short term, broader changes to the technology industry might be required. ...
... For-profit companies might exploit the SIA template as a tool for ethics washing [4,38,109,146], as others have observed in prior research studying responsible AI [38,82,109], privacy and data protection [14,58,134,140], and environmental justice [26,41,105,139]. Prior work on algorithmic auditing [15,18,42,43,85,107] offers helpful suggestions for how to ensure the integrity of the impact assessment process and bring about meaningful change, often via external review. ...
Recent years have witnessed increasing calls for computing researchers to grapple with the societal impacts of their work. Tools such as impact assessments have gained prominence as a method to uncover potential impacts, and a number of publication venues now encourage authors to include an impact statement in their submissions. Despite this recent push, little is known about the way researchers go about grappling with the potential negative societal impact of their work -- especially in industry settings, where research outcomes are often quickly integrated into products. In addition, while there are nascent efforts to support researchers in this task, there remains a dearth of empirically-informed tools and processes. Through interviews with 25 industry computing researchers across different companies and research areas, we first identify four key factors that influence how they grapple with (or choose not to grapple with) the societal impact of their research. To develop an effective impact assessment template tailored to industry computing researchers' needs, we conduct an iterative co-design process with these 25 industry researchers, along with an additional 16 researchers and practitioners with prior experience and expertise in reviewing and developing impact assessments or broad responsible computing practices. Through the co-design process, we develop 10 design considerations to facilitate the effective design, development, and adaptation of an impact assessment template for use in industry research settings and beyond, as well as our own "Societal Impact Assessment" template with concrete scaffolds. We explore the effectiveness of this template through a user study with 15 industry research interns, revealing both its strengths and limitations. Finally, we discuss the implications for future researchers and organizations seeking to foster more responsible research practices.
... Several other works have begun to document AI supply chains and their impacts, including Lee et al. [LCG23] who explore their implications for copyright; Widder and Nafus [WN23] and Cobbe et al. [CVS23] who investigate their effects on AI accountability; and Cen et al. [CHI+23] who examine the potential economic and policy implications. These works establish that AI supply chains carry social, economic, and political importance. ...
... The rising prevalence of AI supply chains has brought with it a growing academic interest in their effects. For example, Widder and Nafus [WN23] and Cobbe et al. [CVS23] examine the social and ethical considerations of having multiple external contributors to an ML product. Lee et al. [LCG23] discuss the legal implications, particularly with regard to copyright violation, while Attard-Frost and Hayes [AH23] consider Canada's efforts in regulating AI through AI supply (or value) chains. ...
The widespread adoption of AI in recent years has led to the emergence of AI supply chains: complex networks of AI actors contributing models, datasets, and more to the development of AI products and services. AI supply chains have many implications yet are poorly understood. In this work, we take a first step toward a formal study of AI supply chains and their implications, providing two illustrative case studies indicating that both AI development and regulation are complicated in the presence of supply chains. We begin by presenting a brief historical perspective on AI supply chains, discussing how their rise reflects a longstanding shift towards specialization and outsourcing that signals the healthy growth of the AI industry. We then model AI supply chains as directed graphs and demonstrate the power of this abstraction by connecting examples of AI issues to graph properties. Finally, we examine two case studies in detail, providing theoretical and empirical results in both. In the first, we show that information passing (specifically, of explanations) along the AI supply chains is imperfect, which can result in misunderstandings that have real-world implications. In the second, we show that upstream design choices (e.g., by base model providers) have downstream consequences (e.g., on AI products fine-tuned on the base model). Together, our findings motivate further study of AI supply chains and their increasingly salient social, economic, regulatory, and technical implications.
... The role of organizational factors has been a common theme [18,58,59,93]. Researchers have emphasized the discretionary choices that shape data work, such as how tasks are formulated, what data is collected and annotated, how data quality is measured, which errors are acceptable, and what is communicated to stakeholders [70]. ...
... Practitioners described how the pressure to meet immediate demands often outweighed the desire for rigorous validation. The scale and opacity of synthetic data created through auxiliary models further compound these problems as data issues become more challenging to trace, deferring accountability [93] and leaving systems more prone to perpetuating harm. Thus, the scale of synthetic data emerges as both a promise and a peril. ...
Alongside the growth of generative AI, we are witnessing a surge in the use of synthetic data across all stages of the AI development pipeline. It is now common practice for researchers and practitioners to use one large generative model (which we refer to as an auxiliary model) to generate synthetic data that is used to train or evaluate another, reconfiguring AI workflows and reshaping the very nature of data. While scholars have raised concerns over the risks of synthetic data, policy guidance and best practices for its responsible use have not kept up with these rapidly evolving industry trends, in part because we lack a clear picture of current practices and challenges. Our work aims to address this gap. Through 29 interviews with AI practitioners and responsible AI experts, we examine the expanding role of synthetic data in AI development. Our findings reveal how auxiliary models are now widely used across the AI development pipeline. Practitioners describe synthetic data as crucial for addressing data scarcity and providing a competitive edge, noting that evaluation of generative AI systems at scale would be infeasible without auxiliary models. However, they face challenges controlling the outputs of auxiliary models, generating data that accurately depict underrepresented groups, and scaling data validation practices that are based primarily on manual inspection. We detail general limitations of and ethical considerations for synthetic data and conclude with a proposal of concrete steps towards the development of best practices for its responsible use.
... There is a tendency to assume that social and ethical issues are someone else's problem (Widder and Nafus 2023), but this is not the case! If you do not reflect on your design decisions as you make them then you are complicit in the avoidable consequences of those decisions . ...
... Your responsibility for ethical outcomes does not end when you finish work on a given system (Widder and Nafus 2023). In most cases our intention is that others will further develop what we have done. ...
This whitepaper offers an overview of the ethical considerations surrounding research into or with large language models (LLMs). As LLMs become more integrated into widely used applications, their societal impact increases, bringing important ethical questions to the forefront. With a growing body of work examining the ethical development, deployment, and use of LLMs, this whitepaper provides a comprehensive and practical guide to best practices, designed to help those in research and in industry to uphold the highest ethical standards in their work.
... Current AI technology development is modular, fragmented, and difficult for developers to trace responsibility and accountability. For example, Widder & Nafus (2023) identified "dislocated accountabilities" which has been created by the modularity of roles and responsibilities and division of labor across different actors in the AI technology supply chain [57]. Similarly, Schmidt et al. (2023) found considerable role ambiguity among developers of AI-based systems which blurs the boundaries of system accountability [53]. ...
... Technical solutions to building business systems seldom consider the complexity of socio-technical interactions between stakeholders, and that these interactions have considerable impacts on organizations and society [55]. While researchers believe that the current system of AI software development is unlikely to change any time soon [57] human-centered design and collaborative elements like the artifacts developed in this study help to challenge the current system with opportunities to raise the bar for better practices. We reflect on our work and consider Ng et al., (2021) in their review of AI literacy as having three levels; knowing and understanding AI, using and applying AI, and evaluating and creating AI, that can be viewed as a hierarchy in the level off understanding, where one level needs to be mastered before moving onto the next [18]. ...
This work examines the potential of creative and playful participatory methodologies to advance collaborative stakeholder design of intelligent information systems. Our work responds to widespread industry criticism of unethical practices, unintended negative consequences of ‘black box’ algorithmic decision making, and organisations falling short in earning tangible business value through AI investments. We present prototype interactive AI design cards and a collaborative AI design sprint process called Designing Tiny Robots targeted at alleviating the limitations with the current generation of intelligent human-machine systems. We argue that collaborative AI system design may account for a fuller range of the socio-technical interactions between stakeholders and thereby improving AI technology design, effectiveness, and adoption.
... There is little research on deepfake developers' values. Even AI software engineers more broadly "have received surprisingly scant attention" [40]: Only few interview-based studies examine their attribution of ethical responsibility [40], awareness of the implications of their work and sense of accountability [55], or broader ethical concerns and proposed solutions [57]. Further work focuses on the co-production of values [51] or specific issues such as fairness or accountability [26,31,53]. ...
... I manufacture the paint brushes and then see what people do with them". This perception relates to widespread notions of technological neutrality (for open-source deepfake communities see [56], for machine learning developers "high[…] in the AI supply chain" see [55]). Interestingly, one open-source developer interviewed by Widder et al. even used a similar metaphor, stating "If I painted something offensive, you can't blame the paint manufacturer" [56]. ...
Policymakers and societies are grappling with the question of how to respond to deepfakes, i.e., synthetic audio-visual media which is proliferating in all areas of digital life– from politics to pornography. However, debates and research on deepfakes’ impact and governance largely neglect the technology’s sources, namely the developers of the underlying artificial intelligence (AI), and those who provide code or deepfake creation services to others, making the technology widely accessible. These actors include open-source developers, professionals working in large technology companies and specialized start-ups, and for deepfake apps. They can profoundly impact which underlying AI technologies are developed, whether and how they are made public, and what kind of deepfakes can be created. Therefore, this paper explores which values guide professional deepfake development, how economic and academic pressures and incentives influence developers’ (perception of) agency and ethical views, and how these views do and could impact deepfake design, creation, and dissemination. Thereby, the paper focuses on values derived from debates on AI ethics and on deepfakes’ impact. It is based on ten qualitative in-depth expert interviews with academic and commercial deepfake developers and ethics representatives of synthetic media companies. The paper contributes to a more nuanced understanding of AI ethics in relation to audio-visual generative AI. Besides, it empirically informs and enriches the deepfake governance debate by incorporating developers’ voices and highlighting governance measures which directly address deepfake developers and providers and emphasize the potential of ethics to curb the dangers of deepfakes.
... A broader perspective on contestability can be gained by considering AI systems as dynamic sociotechnical systems with temporal and spatial dimensions. This approach expands our horizons to consider contestable AI as a value chain problem 1 [6,7,20]). This encompasses the entirety of the AI lifecycle, including various actions taken by different actors: the extraction of materials, the construction of physical infrastructures, the decision-making process in data collection, model development, modes of human oversight and the impact on individuals, society and the environment. ...
... We are particularly interested in empirical studies, including real-world case studies and user studies with both positive and negative outcomes, along with reflections on lessons learned. Additionally, we seek descriptions of unique contexts for contestable AI and their varied and specific challenges, and visionary discussions, and proposals about the implications of 1 The literature uses the terms "AI value chain", e.g., [15], and "AI supply chain", e.g., [7,20]. We use them interchangeably in this workshop. ...
This workshop will grow and consolidate a community of interdisciplinary CSCW researchers focusing on the topic of contestable AI. As an outcome of the workshop, we will synthesize the most pressing opportunities and challenges for contestability along AI value chains in the form of a research roadmap. This roadmap will help shape and inspire imminent work in this field. Considering the length and depth of AI value chains, it will especially spur discussions around the contestability of AI systems along various sites of such chains. The workshop will serve as a platform for dialogue and demonstrations of concrete, successful, and unsuccessful examples of AI systems that (could or should) have been contested, to identify requirements, obstacles, and opportunities for designing and deploying contestable AI in various contexts. This will be held primarily as an in-person workshop, with some hybrid accommodation. The day will consist of individual presentations and group activities to stimulate ideation and inspire broad reflections on the field of contestable AI. Our aim is to facilitate interdisciplinary dialogue by bringing together researchers, practitioners, and stakeholders to foster the design and deployment of contestable AI.
... These exchanges can have implications for various stakeholders, some of them specific to the inclusion of AI models. Existing work highlights the accountability challenges in these AI supply chains [22,23,95], and system prompts certainly warrant consideration in this context, given each organization in the chain can add their own prompts that can significantly alter model (and therefore, broader system) behavior, yet at the same time, deployers and users will often be unaware of prompts added by others. Specifically, the opacity of chained system prompts creates significant challenges. ...
System prompts in Large Language Models (LLMs) are predefined directives that guide model behaviour, taking precedence over user inputs in text processing and generation. LLM deployers increasingly use them to ensure consistent responses across contexts. While model providers set a foundation of system prompts, deployers and third-party developers can append additional prompts without visibility into others' additions, while this layered implementation remains entirely hidden from end-users. As system prompts become more complex, they can directly or indirectly introduce unaccounted for side effects. This lack of transparency raises fundamental questions about how the position of information in different directives shapes model outputs. As such, this work examines how the placement of information affects model behaviour. To this end, we compare how models process demographic information in system versus user prompts across six commercially available LLMs and 50 demographic groups. Our analysis reveals significant biases, manifesting in differences in user representation and decision-making scenarios. Since these variations stem from inaccessible and opaque system-level configurations, they risk representational, allocative and potential other biases and downstream harms beyond the user's ability to detect or correct. Our findings draw attention to these critical issues, which have the potential to perpetuate harms if left unexamined. Further, we argue that system prompt analysis must be incorporated into AI auditing processes, particularly as customisable system prompts become increasingly prevalent in commercial AI deployments.
... Some relevant cases are sociotechnical systems where responsibility for processes and outcomes are highly diffused, sometimes due to outsourcing of work. (e.g., see Horneber & Laumer, 2023;Widder & Nafus, 2023) Other relevant examples involve deeply embedded technical subsystems that may be commercial products or may have been developed through open source development. An example is Log4j, a widely used open-source Apache logging framework that developers use to record activity within enterprise systems and web apps. ...
This paper proposes 24 axioms that form a basis for understanding and analyzing sociotechnical and totally automated work systems in organizational settings. The introduction identifies reasons for trying to develop axioms of that type. Two background sections compare axioms with other types of "knowledge objects", illustrate the minimal presence of axioms in the IS discipline, define the domain of relevance for the axioms discussed here, and explain the process of developing those axioms. The axioms are organized in five categories: 1) system in context, 2) system operation, 3) system goals and goal attainment, 4) system uncertainties, and 5) system-related change. The axioms potentially address two challenges: helping business and IS/IT professionals understand and collaborate around systems in organization and providing a new starting point for producing and evaluating generalizations such as theories, models, and frameworks in the IS discipline. The Appendix shows the output of a large language model's application of the axioms to two case studies, thereby illustrating the potential practical value of the axioms.
... While much of the existing work has focused on predictive algorithms via surveys (Sun & Medaglia, 2019;de Boer & Raaphorst, 2021;Nagtegaal, 2021;Wang et al., 2022), in-depth interviews and detailed case studies are crucial to capture the full spectrum of these dynamics. Additionally, investigations into AI supply chains and the role of IT vendors are needed, as the concentration of power among a few providers poses risks to both accountability and competition (Widder & Nafus, 2023;Cobbe et al., 2023;Narechania & Sitaraman, 2023;Chen et al., 2023). ...
This paper offers a conceptual analysis of the transformative role of Artificial Intelligence (AI) in urban governance, focusing on how AI reshapes governance approaches, oversight mechanisms, and the relationship between bureaucratic discretion and accountability. Drawing on public administration theory, tech-driven governance practices, and data ethics, the study synthesizes insights to propose guiding principles for responsible AI integration in decision-making processes. While primarily conceptual, the paper draws on illustrative empirical cases to demonstrate how AI is reshaping discretion and accountability in real-world settings. The analysis argues that AI does not simply restrict or enhance discretion but redistributes it across institutional levels. It may simultaneously strengthen managerial oversight, enhance decision-making consistency, and improve operational efficiency. These changes affect different forms of accountability: political, professional, and participatory, while introducing new risks, such as data bias, algorithmic opacity, and fragmented responsibility across actors. In response, the paper proposes guiding principles: equitable AI access, adaptive administrative structures, robust data governance, and proactive human-led decision-making, citizen-engaged oversight. This study contributes to the AI governance literature by moving beyond narrow concerns with perceived discretion at the street level, highlighting instead how AI transforms rule-based discretion across governance systems. By bridging perspectives on efficiency and ethical risk, the paper presents a comprehensive framework for understanding the evolving relationship between discretion and accountability in AI-assisted governance.
... For instance, Hooker (2021) has argued that a form of hardware lottery-the greater availability of hardware with strengths in parallel processing-was key to the resurgence of deep learning in the 2010s. 3 Similarly, researchers have examined the role of incentives, socioeconomic factors, and hype cycles in AI research Wang et al., 2024;Hicks et al., 2024;Gebru & Torres, 2024;Sartori & Bocca, 2023;Delgado et al., 2023;Raji et al., 2022;Widder & Nafus, 2023). With this trap, we focus on cases where lotteries (luck) or incentives drive the adoption of unjustified goals-goals that are inadequately supported by scientific, engineering, or societal merit. ...
The AI research community plays a vital role in shaping the scientific, engineering, and societal goals of AI research. In this position paper, we argue that focusing on the highly contested topic of `artificial general intelligence' (`AGI') undermines our ability to choose effective goals. We identify six key traps -- obstacles to productive goal setting -- that are aggravated by AGI discourse: Illusion of Consensus, Supercharging Bad Science, Presuming Value-Neutrality, Goal Lottery, Generality Debt, and Normalized Exclusion. To avoid these traps, we argue that the AI research community needs to (1) prioritize specificity in engineering and societal goals, (2) center pluralism about multiple worthwhile approaches to multiple valuable goals, and (3) foster innovation through greater inclusion of disciplines and communities. Therefore, the AI research community needs to stop treating `AGI' as the north-star goal of AI research.
... Probing practitioners' assumptions around data work can, therefore, illuminate the local and situated ways in which different actors within the network of machine learning development enact classifications of skill and intelligence. Previous work has considered how the organisation of labour can shape responsible AI practice and how responsibility is distributed accordingly within the AI supply chain, viewed as modular and isolated from broader accountability (Widder and Nafus 2023). We show how this organisation of labour is directly shaped by framings of skill and intelligence, and that the contingencies of AI practice are not limited to the specific contexts of model development but have concrete downstream impacts on future AI projects and outputs. ...
In this paper, we empirically and conceptually examine how distributed human–machine networks of labour comprise a form of underlying intelligence within Artificial Intelligence (AI), considering the implications of this for Responsible Artificial Intelligence (R-AI) innovation. R-AI aims to guide AI research, development and deployment in line with certain normative principles, for example fairness, privacy, and explainability; notions implicitly shaped by comparisons of AI with individualised notions of human intelligence. However, as critical scholarship on AI demonstrates, this is a limited framing of the nature of intelligence, both of humans and AI. Furthermore, it dismisses the skills and labour central to developing AI systems, involving a distributed network of human-directed practices and reasoning. We argue that inequities in the agency and recognition of different types of practitioners across these networks of AI development have implications beyond RAI, with narrow framings concealing considerations which are important within broader discussions of AI intelligence. Drawing from interactive workshops conducted with AI practitioners, we explore practices of data acquisition, cleaning, and annotation, as the point where practitioners interface with domain experts and data annotators. Despite forming a crucial part of AI design and development, this type of data work is frequently framed as a tedious, unskilled, and low-value process. In exploring these practices, we examine the political role of the epistemic framings that underpin AI development and how these framings can shape understandings of distributed intelligence, labour practices, and annotators’ agency within data structures. Finally, we reflect on the implications of our findings for developing more participatory and equitable approaches to machine learning applications in the service of R-AI.
... (e.g., was consent given by accepting a terms of service, and users are actually largely unaware that their data can be scraped). • Consider this Supply Chain Analogy [127]: are you expecting that ethical data practices are being done by others before/after you rather than interrogating if this publicly available data should not be publicly available for use? How can you take ownership / responsibility of this data to ensure that it is ethical to scrape, collect, and/or use it? ...
The Generative AI Ethics Playbook provides guidance for identifying and mitigating risks of machine learning systems across various domains, including natural language processing, computer vision, and generative AI. This playbook aims to assist practitioners in diagnosing potential harms that may arise during the design, development, and deployment of datasets and models. It offers concrete strategies and resources for mitigating these risks, to help minimize negative impacts on users and society. Drawing on current best practices in both research and ethical considerations, this playbook aims to serve as a comprehensive resource for AI/ML practitioners. The intended audience of this playbook includes machine learning researchers, engineers, and practitioners who are involved in the creation and implementation of generative and multimodal models (e.g., text-to-text, image-to-image, text-to-image, text-to-video). Specifically, we provide transparency/documentation checklists, topics of interest, common questions, examples of harms through case studies, and resources and strategies to mitigate harms throughout the Generative AI lifecycle. This playbook was made collaboratively over the course of 16 months through extensive literature review of over 100 resources and peer-reviewed articles, as well as through an initial group brainstorming session with 18 interdisciplinary AI ethics experts from industry and academia, and with additional feedback from 8 experts (5 of whom were in the initial brainstorming session). We note that while this playbook provides examples, discussion, and harm mitigation strategies, research in this area is ongoing. Our playbook aims to be a practically useful survey, taking a high-level view rather than aiming for covering the entire existing body of research.
... Here, focus is not on the technological aspects of transparency and open source, but on what we, as users, could do to encourage open systems to thrive. While the ways these interviewees argue about open source are convincing, opening up for influence from more actors would also complicate the assignment of responsibility, as control is diluted (Widder & Nafus 2023). In itself, this does not take away responsibility that each of these actors have, but it brings to the fore the problem of many hands. ...
The purpose of this paper is to increase the understanding of how different types of experts with influence over the development of AI, in this role, reflect upon distribution of forward-looking responsibility for AI development with regard to safety and democracy. Forward-looking responsibility refers to the obligation to see to it that a particular state of affairs materialise. In the context of AI, actors somehow involved in AI development have the potential to guide AI development in a safe and democratic direction. This study is based on qualitative interviews with such actors in different roles at research institutions, private companies, think tanks, consultancy agencies, parliaments, and non-governmental organisations. While the reflections about distribution of responsibility differ among the respondents, one observation is that influence is seen as an important basis for distribution of responsibility. Another observation is that several respondents think of responsibility in terms of what it would entail in concrete measures. By showing how actors involved in AI development reflect on distribution of responsibility, this study contributes to a dialogue between the field of AI governance and the field of AI ethics.
... It enrolls them into thinking like the DOD in their conceptualization of research problems and the applications they might be used for, in order to improve their chances of a successful grant proposal. It also provides "moral wiggle room" or plausible deniability, by providing ways of thinking about their work as basic and thus unconnected to end use (Widder and Nafus 2023), in this case for war, "battlefield management," or killing. ...
In the context of unprecedented U.S. Department of Defense (DoD) budgets, this paper examines the recent history of DoD funding for academic research in algorithmically based warfighting. We draw from a corpus of DoD grant solicitations from 2007 to 2023, focusing on those addressed to researchers in the field of artificial intelligence (AI). Considering the implications of DoD funding for academic research, the paper proceeds through three analytic sections. In the first, we offer a critical examination of the distinction between basic and applied research, showing how funding calls framed as basic research nonetheless enlist researchers in a war fighting agenda. In the second, we offer a diachronic analysis of the corpus, showing how a 'one small problem' caveat, in which affirmation of progress in military technologies is qualified by acknowledgement of outstanding problems, becomes justification for additional investments in research. We close with an analysis of DoD aspirations based on a subset of Defense Advanced Research Projects Agency (DARPA) grant solicitations for the use of AI in battlefield applications. Taken together, we argue that grant solicitations work as a vehicle for the mutual enlistment of DoD funding agencies and the academic AI research community in setting research agendas. The trope of basic research in this context offers shelter from significant moral questions that military applications of one's research would raise, by obscuring the connections that implicate researchers in U.S. militarism.
... Further, regulatory and industry groups have high interest because the former is responsible for regulating the environmental impact, while the latter must adhere to the standards and guidelines set by the former. Practitioners, on the other hand, might have lower interest as individuals because they may feel their individual contributions are negligible compared to larger organizational efforts, or what Widder and Nafus (2023) describe as dislocated accountabilities. However, because practitioners have the power to have significant influence, due to the implications of even the smallest choices with regard to AI system development and implementation, it is important to raise awareness among them and encourage greater involvement and interest in sustainable practices. ...
It's no secret that AI systems come with a significant environmental cost. This raises the question: What are the roles and responsibilities of computing professionals regarding the sustainability of AI? Informed by a year-long informal literature review on the subject, we employ stakeholder identification, analysis, and mapping to highlight the complex and interconnected roles that five major stakeholder groups (industry, practitioners, regulatory, advocacy, and the general public) play in the sustainability of AI. Swapping the traditional final step of stakeholder methods (stakeholder engagement) for entanglement, we demonstrate the inherent entwinement of choices made with regard to the development and maintenance of AI systems and the people who impact (or are impacted by) these choices. This entanglement should be understood as a system of human and non-human agents, with the implications of each choice ricocheting into the use of natural resources and climate implications. We argue that computing professionals (AI-focused or not) may belong to multiple stakeholder groups, and that we all have multiple roles to play in the sustainability of AI. Further, we argue that the nature of regulation in this domain will look unlike others in environmental preservation (e.g., legislation around water contaminants). As a result, we call for ongoing, flexible bodies and policies to move towards the regulation of AI from a sustainability angle, as well as suggest ways in which individual computing professionals can contribute to fighting the environmental and climate effects of AI.
... The attribution of responsibility in AI is the focus of a growing research discourse [11,16,21,42,47,51,53,62]. Related debates often centre on identifying who should bear responsibility when AI systems do not behave as intended, thus including actors such as developers, organisations, end-users and the broader sociotechnical context in which these technologies operate. Amid these discussions, there is an increasing emphasis on the moral and ethical responsibility of both designers and engineers [17,26,50,52,53,56,69]. This focus underlies a body of research seeking to include ethical aspects in design [29] and engineering education [22], promote training reflexivity [44], and call for the introduction of ethical perspectives into educational curricula [54,58]. ...
Automated Grading Systems (AGSs) are increasingly used in higher education assessment practices, raising issues about the responsibilities of the various stakeholders involved both in their design and use. This study explores how teachers, students, exam administrators , and developers of AGSs perceive and enact responsibilities around such systems. Drawing on a focus group and interview data, we applied Fuchsberger and Frauenberger's [27] notion of Doing Responsibilities as an analytical lens. This notion, framing responsibility as shared among human and nonhuman actors (e.g., technologies and data), has guided our analysis of how responsibilities are continuously configured and enacted in university assessment practices. The findings illustrate the stakeholders' perceived and enacted responsibilities at different phases, contributing to the HCI literature on Responsible AI and AGSs by presenting a practical application of the 'Doing Responsibilities' framework before, during and after design. We discuss how the findings enrich this notion, emphasising the importance of engaging with nonhumans, considering regulatory aspects of responsibility, and addressing relational tensions within automation.
... Interventions such as datasheets (Gebru et al., 2021) data statements (Bender and Friedman, 2018), and nutrition labels (Holland et al., 2018;Chmielinski et al., 2022) contributes to safeguarding datasets by explicitly highlighting the practices of creation, intended uses and limitations. However, the modularity of dataset development (Polyzotis et al., 2018) can also hinder the thorough documentation of datasets, as no one individual can attest to each aspect of its creation (Widder and Nafus, 2023). A current limitation is that few research papers document how they constructed a dataset (Geiger et al., 2020). ...
The increasing demand for high-quality datasets in machine learning has raised concerns about the ethical and responsible creation of these datasets. Dataset creators play a crucial role in developing responsible practices, yet their perspectives and expertise have not yet been highlighted in the current literature. In this paper, we bridge this gap by presenting insights from a qualitative study that included interviewing 18 leading dataset creators about the current state of the field. We shed light on the challenges and considerations faced by dataset creators, and our findings underscore the potential for deeper collaboration, knowledge sharing, and collective development. Through a close analysis of their perspectives, we share seven central recommendations for improving responsible dataset creation, including issues such as data quality, documentation, privacy and consent, and how to mitigate potential harms from unintended use cases. By fostering critical reflection and sharing the experiences of dataset creators, we aim to promote responsible dataset creation practices and develop a nuanced understanding of this crucial but often undervalued aspect of machine learning research.
... Studies with artificial intelligence (AI) practitioners reveal how accountability for the systems they create is deferred to stakeholders throughout the pipeline of AI development (Orr and Davis, 2020). As modular technologies with multiple stakeholders, datasets are another case of "dislocated accountabilities" (Widder and Nafus, 2023). Creators illustrated a deferred chain of responsibility for datasets with the only exceptions being (a) where it may present legal liability to them, and (b) where it can be entirely placed with the user of the dataset. ...
Despite the critical role that datasets play in how systems make predictions and interpret the world, the dynamics of their construction are not well understood. Drawing on a corpus of interviews with dataset creators, we uncover the messy and contingent realities of dataset preparation. We identify four key challenges in constructing datasets, including balancing the benefits and costs of increasing dataset scale, limited access to resources, a reliance on shortcuts for compiling datasets and evaluating their quality, and ambivalence regarding accountability for a dataset. These themes illustrate the ways in which datasets are not objective or neutral but reflect the personal judgments and trade-offs of their creators within wider institutional dynamics, working within social, technical, and organizational constraints. We underscore the importance of examining the processes of dataset creation to strengthen an understanding of responsible practices for dataset development and care.
... However, accomplishing accountability is by no means straightforward. There is no single point from which to view an entire organisation and its digital operations (Widder and Nafus, 2023). Many digitally saturated organisational settings are opaque (Ziewitz, 2016). ...
Recent years have seen significant increases in online fraud. The fact that online fraud represents a major challenge to law enforcement due to its complexities and global impact has led to the emergence of other organisations – such as Customer Service Centres – taking a key role in ‘policing’ fraudulent activities. However, the responses made by these specialist organisations remain opaque and outside the scope of regimes that regulate law enforcement agencies. In this article we address this opacity through our study of a Customer Service Centre that makes decisions on what constitutes online fraud in cases of card-not-present payments. We carefully work through these decision-making processes to explore the immediate pressures made manifest in decisions on fraud around, for example, cost and timeliness. These pressures become apparent in the particular arrangements of accountability and responsibility in decisions on online fraud and cut what might otherwise be lengthy procedures following these decisions. As a result we suggest that accountability in these fraud cases is managed and held ‘in the moment’ within the Centre. The article contributes to our understanding of online fraud and to the growing debate on digital accountability. The article provides avenues for further exploration of the challenges of moving from internal to external accountability in relation to largely opaque and data-sensitive settings where accountability relations are held ‘in the moment.’
Tech workers often experience ethical tensions arising from the misalignment of their values and the prevailing unethical or ethically ambiguous practices concerning data and algorithms in the workplace. Despite this, there is an insufficient understanding of how tech professionals address ethical tensions. Based on interviews with 98 tech workers in China and the United States, this study explores ethical tensions, the workers’ responses, and potential cross-national variations. It identifies three prevalent strategies by which tech workers navigate conflicts between their ethical principles and their companies’ practices: complying with market fundamentalism, compromising personal ethics, and upholding and critiquing ethical guidelines. Cross-national differences in strategy implementation highlight nuanced approaches by tech workers in diverse economic, political, and ethical contexts. The study positions these responses within a theoretical framework of ethical agency, revealing tech workers’ subtle ethical agency and the factors that constrain their decision-making processes. It also contributes data-driven insights to promote ethical practices in the global tech industry.
Contributors to open source software packages often describe feeling discouraged by the lack of positive feedback from users. This paper describes a technology probe, Hug Reports, that provides users a communication affordance within their code editors, through which users can convey appreciation to contributors of packages they use. In our field study, 18 users interacted with the probe for 3 weeks, resulting in messages of appreciation to 550 contributors, 26 of whom participated in subsequent research. Our findings show how locating a communication affordance within the code editor, and allowing users to express appreciation in terms of the abstractions they are exposed to (packages, modules, functions), can support exchanges of appreciation that are meaningful to users and contributors. Findings also revealed the moments in which users expressed appreciation, the two meanings that appreciation took on -- as a measure of utility and as an act of expressive communication -- and how contributors' reactions to appreciation were influenced by their perceived level of contribution. Based on these findings, we discuss opportunities and challenges for designing appreciation systems for open source in particular, and peer production communities more generally.
Recent years have witnessed increasing calls for computing researchers to grapple with the societal impacts of their work. Tools such as impact assessments have gained prominence as a method to uncover potential impacts, and a number of publication venues now encourage authors to include an impact statement in their submissions. Despite this recent push, little is known about the way researchers go about assessing, articulating, and addressing the potential negative societal impact of their work --- especially in industry settings, where research outcomes are often quickly integrated into products and services. In addition, while there are nascent efforts to support researchers in this task, there remains a dearth of empirically-informed tools and processes. Through interviews with 25 industry computing researchers across different companies and research areas, we identify four key factors that influence how they grapple with (or choose not to grapple with) the societal impact of their research: the relationship between industry researchers and product teams; organizational dynamics and cultures that prioritize innovation and speed; misconceptions about societal impact; and a lack of sufficient infrastructure to support researchers. To develop an effective impact assessment template tailored to industry computing researchers' needs, we conduct an iterative co-design process with these 25 industry researchers, along with an additional 16 researchers and practitioners with prior experience and expertise in reviewing and developing impact assessments or responsible computing practices more broadly. Through the co-design process, we develop 10 design considerations to facilitate the effective design, implementation, and adaptation of an impact assessment template for use in industry research settings and beyond, as well as our own "Societal Impact Assessment" template with concrete scaffolds. We explore the effectiveness of this template through a user study with 15 industry research interns, revealing both its strengths and limitations. Finally, we discuss the implications for future researchers, organizations, and policymakers seeking to foster more responsible research practices.
Large language models are increasingly applied in real-world scenarios, including research and education. These models, however, come with well-known ethical issues, which may manifest in unexpected ways in human-computer interaction research due to the extensive engagement with human subjects. This paper reports on research practices related to LLM use, drawing on 16 semi-structured interviews and a survey with 50 HCI researchers. We discuss the ways in which LLMs are already being utilized throughout the entire HCI research pipeline, from ideation to system development and paper writing. While researchers described nuanced understandings of ethical issues, they were rarely or only partially able to identify and address those ethical concerns in their own projects. This lack of action and reliance on workarounds was explained through the perceived lack of control and distributed responsibility in the LLM supply chain, the conditional nature of engaging with ethics, and competing priorities. Finally, we reflect on the implications of our findings and present opportunities to shape emerging norms of engaging with large language models in HCI research.
Researchers, practitioners, and policymakers with an interest in the ethics of artificial intelligence (AI) need more integrative approaches for studying and intervening in AI systems across many contexts and scales of activity. This paper presents AI value chains as an integrative concept that satisfies that need. To more clearly theorize AI value chains and conceptually distinguish them from supply chains, we review theories of value chains and AI value chains from strategic management, service science, economic geography, industry, government, and applied research literature. We then conduct an integrative review of a sample of 67 sources that cover the ethical concerns implicated in AI value chains. Building upon the findings of our integrative review, we recommend three future directions that researchers, practitioners, and policymakers can take to advance more ethical practices across AI value chains. We urge AI ethics researchers and practitioners to move toward value chain perspectives that situate actors in context, account for the many types of resources involved in cocreating AI systems, and integrate a wider range of ethical concerns across contexts and scales.
This chapter strives to draw attention to the role of supply chains in delivering sustainable development to increase the awareness of the strategic value of the supply chain strength in achieving several SDGs to attain environmental sustainability in particular. Using a quantitative methodology, the study examines the relationship between supply chain strength and environmental performance in BRIC and MIKTA countries, based on the Global Competitive Index and Environmental Performance Index (EPI). Results reveal a strong, significant positive association, with environmental performance explaining 44.7% of the variance in supply chain strength.
There is a significant body of work looking at the ethical considerations of large language models (LLMs): critiquing tools to measure performance and harms; proposing toolkits to aid in ideation; discussing the risks to workers; considering legislation around privacy and security etc. As yet there is no work that integrates these resources into a single practical guide that focuses on LLMs; we attempt this ambitious goal. We introduce 'LLM Ethics Whitepaper', which we provide as an open and living resource for NLP practitioners, and those tasked with evaluating the ethical implications of others' work. Our goal is to translate ethics literature into concrete recommendations and provocations for thinking with clear first steps, aimed at computer scientists. 'LLM Ethics Whitepaper' distils a thorough literature review into clear Do's and Don'ts, which we present also in this paper. We likewise identify useful toolkits to support ethical work. We refer the interested reader to the full LLM Ethics Whitepaper, which provides a succinct discussion of ethical considerations at each stage in a project lifecycle, as well as citations for the hundreds of papers from which we drew our recommendations. The present paper can be thought of as a pocket guide to conducting ethical research with LLMs.
This chapter proposes an analytical lens to comprehensively address the role of Artificial Intelligence (AI) applications in mediating arbitrary exercise of power in public administration and the citizen harms that result from such conduct. It provides a timely and urgent account to fill gaps in conventional Rule of Law thought. AI systems are socio-technical by nature and, therefore, differ from the text-driven social constructs that the legal professions dealing with Rule of Law issues concentrate on. Put to work in public administration contexts with consequential decision-making, technical artefacts can contribute to a variety of hazardous situations that provide opportunities for arbitrary conduct. A comprehensive lens to understand and address the role of technology in Rule of Law violations has largely been missing in literature. We propose to combine a socio-legal perspective on the Rule of Law with central insights from system safety—a safety engineering tradition with a strong scientific as well as real-world practice—that considers safety from a technological, systemic, and institutional perspective. The combination results in a lexicon and analytical approach that enables public organisations to identify possibilities for arbitrary conduct in public AI systems. Following on the analysis, interventions can be designed to prevent, mitigate, or correct system hazards and, thereby, protect citizens against arbitrary exercise of power.
The rapid evolution of digital technologies over the past decades has induced profound economic and social transformations. Economic geography faces the ongoing challenge of assimilating these changes into existing theories that elucidate the dynamics of the global economy. In response, we present the Global Digital Networks (GDN) framework, drawing inspiration from established analytical instruments like Global Production Networks (GPN) and Global Financial Networks (GFN). GDN centres on three key economic materialities—people, things, and places—intertwined with territorially grounded practices of data generation and enhancement. We identify four enhancement types—singularisation, association, centralisation and fractionalisation—driving a cyclical process shaping complex networks across territories. Governance structures, encompassing national regulations, platform systems, and firm governance, play a pivotal role. The GDN cycle, exemplified through diverse territorial scenarios, underscores the intricate interplay of data generation, enhancement and governance structures in delineating global economic networks.
Convenience is the feeling and aspiration that animates our platformed present. As such, it poses urgent techno-political questions about the everyday digital habitus. From next-day delivery, gig work, and tele-health to cashless payment systems, data centers, and policing convenience is an affordance and an enclosure; our logistical surround. Driving every experience of convenience is the precarious work, proprietary algorithms, or predatory schemes that subtend it. This collaborative book traces how the logistical surround is transformed by thickening digital economies and networked rituals, examining contemporary conveniences across a wide range of practices and geographies. Contributors examine the ineluctable relation between convenience and its constitutive opposite, inconvenience, considering its infrastructural, affective, and compulsory dimensions. Living in convenience is thus both a hyper visible manifestation of so-called late capitalism and a pervasive mood that fades into the background (like the data centers that power it). Bringing the agonistic relation of in/convenience to center stage, this volume analyzes the logistics of delivery, streaming porn, cloud computing, water infrastructures, smartness paradigms, convenience stores, sleep apps, surveillance, AI ethics, and much more – rethinking the cultural politics of convenience for the present conjuncture.
This paper examines 'open' artificial intelligence (AI). Claims about 'open' AI often lack precision, frequently eliding scrutiny of substantial industry concentration in large-scale AI development and deployment, and often incorrectly applying understandings of 'open' imported from free and open-source software to AI systems. At present, powerful actors are seeking to shape policy using claims that 'open' AI is either beneficial to innovation and democracy, on the one hand, or detrimental to safety, on the other. When policy is being shaped, definitions matter. To add clarity to this debate, we examine the basis for claims of openness in AI, and offer a material analysis of what AI is and what 'openness' in AI can and cannot provide: examining models, data, labour, frameworks, and computational power. We highlight three main affordances of 'open' AI, namely transparency, reusability, and extensibility, and we observe that maximally 'open' AI allows some forms of oversight and experimentation on top of existing models. However, we find that openness alone does not perturb the concentration of power in AI. Just as many traditional open-source software projects were co-opted in various ways by large technology companies, we show how rhetoric around 'open' AI is frequently wielded in ways that exacerbate rather than reduce concentration of power in the AI sector.
Under the legislation, when artificial intelligence (AI) systems cause harm to third parties, the restoration of violated rights is carried out according to the rules of strict or culpable liability. Strict liability is applied if the AI system is recognized as a source of increased danger or has a defect. For all other cases, culpable civil liability is used. The authors have developed a new approach to non-contractual civil liability for cases of harm caused by AI systems based on the criterion of the risk level of AI systems. According to this approach, for AI systems that create unacceptable or high risk in relation to human rights and freedoms, it is proposed to apply strict liability to their developer, and for AI systems belonging to the low-risk classification group, the rules of culpable liability to restore violated rights and compensate for the harm caused should be applied. With regard to the basic models, the use of culpable liability is envisaged, except situations where AI products with unacceptable or high risk are created on their basis. The proposed approach can become an alternative to using the concept of a source of increased danger in relation to AI systems and will allow transferring strict responsibility from owners of high-risk AI systems to their developers, who have a greater impact on the safety and reliability of AI systems.
Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems. However, prior research has highlighted gaps between the intended design of these tools and practices and their use within particular contexts, including gaps caused by the role that organizational factors play in shaping fairness work. In this paper, we investigate these gaps for one such practice: disaggregated evaluations of AI systems, intended to uncover performance disparities between demographic groups. By conducting semi-structured interviews and structured workshops with thirty-three AI practitioners from ten teams at three technology companies, we identify practitioners' processes, challenges, and needs for support when designing disaggregated evaluations. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups on which to focus, and collecting datasets with which to conduct disaggregated evaluations. More generally, we identify impacts on fairness work stemming from a lack of engagement with direct stakeholders or domain experts, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.
Ethics is arguably the hottest product in Silicon Valley’s1 hype cycle today, even as headlines decrying a lack of ethics in technology companies accumulate. After years of largely fruitless outside pres- sure to consider the consequences of digital technology products, the very recent past has seen a spike in the assignment of corporate resources in Silicon Valley to ethics, including hiring staff for roles we identify here as “ethics owners.” In corporate parlance, “owning” a portfolio or project means holding responsibility for it, often across multiple divisions or hierarchies within the organization. Typically, the “owner” of a project does not bear sole responsibility for it, but rather oversees integration of that project across the organization.
User experience (UX) professionals’ attempts to address social values as a part of their work practice can overlap with tactics to contest, resist, or change the companies they work for. This paper studies tactics that take place in this overlap, where UX professionals try to re-shape the values embodied and promoted by their companies, in addition to the values embodied and promoted in the technical systems and products that their companies produce. Through interviews with UX professionals working at large U.S.-based technology companies and observations at UX meetup events, this paper identifies tactics used towards three goals: (1)creating space for UX expertise to address values; (2) making values visible and relevant to other organizational stakeholders; and (3) changing organizational processes and orientations towards values. This paper analyzes these as tactics of resistance: UX professionals seek to subvert or change existing practices and organizational structures towards more values-conscious ends. Yet, these tactics of resistance often rely on the dominant discourses and logics of the technology industry. The paper characterizes these as partial or “soft” tactics, but also argues that they nevertheless hold possibilities for enacting values-oriented changes.
The supply chain is not just a metaphor for the production of the modern world, it is the very means through which that world is made material. Through the careful coordination of bodies and materials, it delivers the glut of goods necessary for daily life – even as it ties that life to human rights abuses, violent regimes of extraction, and environmental devastation at unprecedented scales. But while software has long been part of every supply chain, the supply chain is, itself, part of every bit of software. Developers speak of digital production in the language of logistics. They write about algorithms as frequently as distribution networks, worry about just-in-time production as much as engineering, and decorate their tools with illustrations copied from shipping companies or factories. The design of data is thought of the same way as the objects that will access it. This article interrogates the emergence of the supply chain, and the logistical modes of operation it entails, as metaphor for managing the digital distribution of data – adapting approaches from the critical study of logistics in order to re-incorporate the political, social, and environmental attachments that ‘digital supply chains’ attempt to obfuscate. To this end, it considers discourses around power and cultural politics that mirror critiques of traditional logistical infrastructures. Instead of conflict minerals, for example, conflict domains; in place of security concerns around cargo containers, data containers; rather than workers on the factory floor, labourers in a digital network of ‘sweatshops.’ These comparisons reveal differences between traditional supply chains and their digital counterparts – the most troubling of which is their infrastructural instability. With components that can be replaced while retaining their essential shape, those who depend on digital platforms can find themselves open to all sorts of redirected entanglements.
Concerns about secondary use of data and limited opportunities for benefit-sharing have focused attention on the tension that Indigenous communities feel between (1) protecting Indigenous rights and interests in Indigenous data (including traditional knowledges) and (2) supporting open data, machine learning, broad data sharing, and big data initiatives. The International Indigenous Data Sovereignty Interest Group (within the Research Data Alliance) is a network of nation-state based Indigenous data sovereignty networks and individuals that developed the ‘CARE Principles for Indigenous Data Governance’ (Collective Benefit, Authority to Control, Responsibility, and Ethics) in consultation with Indigenous Peoples, scholars, non-profit organizations, and governments. The CARE Principles are people– and purpose-oriented, reflecting the crucial role of data in advancing innovation, governance, and self-determination among Indigenous Peoples. The Principles complement the existing data-centric approach represented in the ‘FAIR Guiding Principles for scientific data management and stewardship’ (Findable, Accessible, Interoperable, Reusable). The CARE Principles build upon earlier work by the Te Mana Raraunga Maori Data Sovereignty Network, US Indigenous Data Sovereignty Network, Maiam nayri Wingara Aboriginal and Torres Strait Islander Data Sovereignty Collective, and numerous Indigenous Peoples, nations, and communities. The goal is that stewards and other users of Indigenous data will ‘Be FAIR and CARE.’ In this first formal publication of the CARE Principles, we articulate their rationale, describe their relation to the FAIR Principles, and present examples of their application.
This paper contributes to our understanding of farm data value chains with assistance from 54 semi-structured interviews and field notes from participant observations. Methodologically, it includes individuals, such as farmers, who hold well-known positionalities within digital agriculture spaces—platforms that include precision farming techniques, farm equipment built on machine learning architecture and algorithms, and robotics—while also including less visible elements and practices. The actors interviewed and materialities and performances observed thus came from spaces and places inhabited by, for example, farmers, crop scientists, statisticians, programmers, and senior leadership in firms located in the U.S. and Canada. The stability of “the” artifacts followed for this project proved challenging, which led to me rethinking how to approach the subject conceptually. The paper is animated by a posthumanist commitment, drawing heavily from assemblage thinking and critical data scholarship coming out of Science and Technology Studies. The argument’s understanding of “chains” therefore lies on an alternative conceptual plane relative to most commodity chain scholarship. To speak of a data value chain is to foreground an orchestrating set of relations among humans, non-humans, products, spaces, places, and practices. The paper’s principle contribution involves interrogating lock-in tendencies at different “points” along the digital farm platform assemblage while pushing for a varied understanding of governance depending on the roles of the actors and actants involved.
An exploration of how design might be led by marginalized communities, dismantle structural inequality, and advance collective liberation and ecological survival.
What is the relationship between design, power, and social justice? “Design justice” is an approach to design that is led by marginalized communities and that aims expilcitly to challenge, rather than reproduce, structural inequalities. It has emerged from a growing community of designers in various fields who work closely with social movements and community-based organizations around the world.
This book explores the theory and practice of design justice, demonstrates how universalist design principles and practices erase certain groups of people—specifically, those who are intersectionally disadvantaged or multiply burdened under the matrix of domination (white supremacist heteropatriarchy, ableism, capitalism, and settler colonialism)—and invites readers to “build a better world, a world where many worlds fit; linked worlds of collective liberation and ecological sustainability.” Along the way, the book documents a multitude of real-world community-led design practices, each grounded in a particular social movement. Design Justice goes beyond recent calls for design for good, user-centered design, and employment diversity in the technology and design professions; it connects design to larger struggles for collective liberation and ecological survival.
The open access edition of this book was made possible by generous funding from Knowledge Unlatched and the MIT Press Frank Urbanowski Memorial Fund.
Systems based on Artificial Intelligence (AI) are increasingly normalized as part of work, leisure, and governance in contemporary societies. Although ethics in AI has received significant attention, it remains unclear where the burden of responsibility lies. Through twenty-one interviews with AI practitioners in Australia, this research seeks to understand how ethical attributions figure into the professional imagination. As institutionally embedded technical experts, AI practitioners act as a connective tissue linking the range of actors that come in contact with, and have effects upon, AI products and services. Findings highlight that practitioners distribute ethical responsibility across a range of actors and factors, reserving a portion of responsibility for themselves, albeit constrained. Characterized by imbalances of decision-making power and technical expertise, practitioners position themselves as mediators between powerful bodies that set parameters for production; users who engage with products once they leave the proverbial workbench; and AI systems that evolve and develop beyond practitioner control. Distributing responsibility throughout complex sociotechnical networks, practitioners preclude simple attributions of accountability for the social effects of AI. This indicates that AI ethics are not the purview of any singular player but instead, derive from collectivities that require critical guidance and oversight at all stages of conception, production, distribution, and use.
ion, defined in Computer Science (CS) as bracketing unnecessary information from diverse components within a system, serves as a central epistemological axis in CS disciplinary and pedagogical practices. Its impressions can be seen across curricula, syllabi, classroom structures, IT systems; and other dimensions of the epistemic infrastructure of CS (Malazita [Forthcoming]. “Epistemic Infrastructures, the Instrumental Turn, and the Digital Humanities.” In Debates in the Digital Humanities: Infrastructures, Institutions at the Interstices, edited by Angel Nieves, Siobhan Senier, and Anne McGrail. University of Minnesota Press). As we will argue in this essay, abstraction in CS serves as an epistemic, cultural, and ideological wall to integrated critical-technical education, rather than as a bridge. Further, this wall is disguised as a bridge: the common language used across CS and the Humanities gives the impression that abstraction can be leveraged as a boundary object (Star [2010]. “This is Not a Boundary Object: Reflections on the Origin of a Concept.” Science, Technology, & Human Values 35 (5): 601–617), as a point of connection among conflicting or incommensurable epistemic cultures (Knorr Cetina [1999]. Epistemic Cultures: How the Sciences Make Knowledge. Cambridge: MIT Press). Rather, computational knowledge practices leverage abstraction’s homographic-ness, epistemically structuring collaborative efforts in anti-political ways. To illustrate the impacts of abstraction, this essay will introduce ‘Critical CS1,’ a hybrid pedagogical approach to teaching Computer Science through feminist and critical race theory. However, other components of the epistemic infrastructures of Computer Science, from curricular structure, to IT systems, to classroom culture, to the epistemic practices of coding itself, resisted these intervention efforts, and reproduced marginalizing effects upon students within the course.
In the past five years, private companies, research institutions and public sector organizations have issued principles and guidelines for ethical artificial intelligence (AI). However, despite an apparent agreement that AI should be ‘ethical’, there is debate about both what constitutes ‘ethical AI’ and which ethical requirements, technical standards and best practices are needed for its realization. To investigate whether a global agreement on these questions is emerging, we mapped and analysed the current corpus of principles and guidelines on ethical AI. Our results reveal a global convergence emerging around five ethical principles (transparency, justice and fairness, non-maleficence, responsibility and privacy), with substantive divergence in relation to how these principles are interpreted, why they are deemed important, what issue, domain or actors they pertain to, and how they should be implemented. Our findings highlight the importance of integrating guideline-development efforts with substantive ethical analysis and adequate implementation strategies.
The EU General Data Protection Regulation (GDPR), enforced from 25 th May 2018, aims to reform how organisations view and control the personal data of private EU citizens. The scope of GDPR is somewhat unprecedented: it regulates every aspect of personal data handling, includes hefty potential penalties for non-compliance, and can prosecute any company in the world that processes EU citizens' data. In this paper, we look behind the scenes to investigate the real challenges faced by organisations in engaging with the GDPR. This considers issues in working with the regulation, the implementation process, and how compliance is verified. Our research approach relies on literature but, more importantly, draws on detailed interviews with several organisations. Key findings include the fact that large organisations generally found GDPR compliance to be reasonable and doable. The same was found for small-to-medium organisations (SMEs/SMBs) that were highly security-oriented. SMEs with less focus on data protection struggled to make what they felt was a satisfactory attempt at compliance. The main issues faced in their compliance attempts emerged from: the sheer breadth of the regulation; questions around how to enact the qualitative recommendations of the regulation; and the need to map out the entirety of their complex data networks.
A number of empirical studies have pointed to a link between software complexity and software maintenance performance. The primary purpose of this paper is to document “what is known” about this relationship, and to suggest some possible future avenues of research. In particular, a survey of the empirical literature in this area shows two broad areas of study: complexity metrics and comprehension. Much of the complexity metrics research has focused on modularity and structure metrics. The articles surveyed are summarized as to major differences and similarities in a set of detailed tables. The text is used to highlight major findings and differences, and a concluding remarks section provides a series of recommendations for future research.
This paper discusses modularization as a mechanism for improving the flexibility and comprehensibility of a system while allowing the shortening of its development time. The effectiveness of a "modularization" is dependent upon the criteria used in dividing the system into modules. A system design problem is presented and both a conventional and unconventional decomposition are described. It is shown that the unconventional decompositions have distinct advantages for the goals outlined. The criteria used in arriving at the decompositions are discussed. The unconventional decomposition, if implemented with the conventional assumption that a module consists of one or more subroutines, will be less efficient in most cases. An alternative approach to implementation which does not have this effect is sketched.
This book investigates the precise effects on society of the new and much vaunted electronic technologies (ICTs). All aspects of our social, cultural, economic, and political life stand to be affected by their continued massive growth, but are fundamental shifts already taking place in the way in which we behave, organize, and interact as a direct result of the new technologies? The contributors to the volume argue that their transformative effects amount to our transition to a virtual society.
Predictive policing is built on a simple assumption: crime exhibits predictable patterns, which means future crime risk can be forecast using historic crime data. While critics have raised concerns over the use of biased data in these systems, less is known about how software is actually used within infrastructures of governance. Drawing on interviews with software developers and an analysis of technical, promotional, and academic materials, I show how internal and external pressures separate predictive policing from the concrete practices it attempts to transform. I argue that predictive policing is a modular technology, plugged into the black box of policing. This modularity separates software developers from the practices they attempt to transform, while enabling them to deflect criticism away from the programs they build. Modularity also means that software can be reconfigured and connected to other systems, which threatens to undermine the set of best practices that guide its development.
This paper examines the promise of empathy, the name commonly given to the initial phase of the human-centered design process in which designers seek to understand their intended users in order to inform technology development. By analyzing popular empathy activities aimed at understanding people with disabilities, we examine the ways empathy works to both powerfully and problematically align designers with the values of people who may use their products. Drawing on disability studies and feminist theorizing, we describe how acts of empathy building may further distance people with disabilities from the processes designers intend to draw them into. We end by reimagining empathy as guided by the lived experiences of people with disabilities who are traditionally positioned as those to be empathized.
The potential for machine learning (ML) systems to amplify social inequities and unfairness is receiving increasing popular and academic attention. A surge of recent work has focused on the development of algorithmic tools to assess and mitigate such unfairness. If these tools are to have a positive impact on industry practice, however, it is crucial that their design be informed by an understanding of real-world needs. Through 35 semi-structured interviews and an anonymous survey of 267 ML practitioners, we conduct the first systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems. We identify areas of alignment and disconnect between the challenges faced by teams in practice and the solutions proposed in the fair ML research literature. Based on these findings, we highlight directions for future ML and HCI research that will better address practitioners' needs.
Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics. In this paper, we propose a framework that we call model cards, to encourage such transparent model reporting. Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type [15]) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. While we focus primarily on human-centered machine learning models in the application fields of computer vision and natural language processing, this framework can be used to document any trained machine learning model. To solidify the concept, we provide cards for two supervised models: One trained to detect smiling faces in images, and one trained to detect toxic comments in text. We propose model cards as a step towards the responsible democratization of machine learning and related artificial intelligence technology, increasing transparency into how well artificial intelligence technology works. We hope this work encourages those releasing trained machine learning models to accompany model releases with similar detailed evaluation numbers and other relevant documentation.
Currently there is no standard way to identify how a dataset was created, and what characteristics, motivations, and potential skews it represents. To begin to address this issue, we propose the concept of a datasheet for datasets, a short document to accompany public datasets, commercial APIs, and pretrained models. The goal of this proposal is to enable better communication between dataset creators and users, and help the AI community move toward greater transparency and accountability. By analogy, in computer hardware, it has become industry standard to accompany everything from the simplest components (e.g., resistors), to the most complex microprocessor chips, with datasheets detailing standard operating characteristics, test results, recommended usage, and other information. We outline some of the questions a datasheet for datasets should answer. These questions focus on when, where, and how the training data was gathered, its recommended use cases, and, in the case of human-centric datasets, information regarding the subjects' demographics and consent as applicable. We develop prototypes of datasheets for two well-known datasets: Labeled Faces in The Wild~\cite{lfw} and the Pang \& Lee Polarity Dataset~\cite{polarity}.
It is widely believed that refactoring improves software quality and developer productivity. However, few empirical studies quantitatively assess refactoring benefits or investigate developers’ perception towards these benefits. This paper presents a field study of refactoring benefits and challenges at Microsoft through three complementary study methods: a survey, semi-structured interviews with professional software engineers, and quantitative analysis of version history data. Our survey finds that the refactoring definition in practice is not confined to a rigorous definition of semantics-preserving code transformations and that developers perceive that refactoring involves substantial cost and risks. We also report on interviews with a designated refactoring team that has led a multi-year, centralized effort on refactoring Windows. The quantitative analysis of Windows 7 version history finds the top 5 percent of preferentially refactored modules experience higher reduction in the number of inter-module dependencies and several complexity measures but increase size more than the bottom 95 percent. This indicates that measuring the impact of refactoring requires multi-dimensional assessment.
Essentially, this essay contains nothing new; on the contrary, its subject matter is so old that sometimes it seems forgotten. It is written in an effort to undo some of the more common misunderstandings that I encounter (nearly daily) in my professional world of computing scientists, programmers, computer users and computer designers, and even colleagues engaged in educational politics. The decision to write this essay now was taken because I suddenly realized that my confrontation with this same pattern of misunderstanding was becoming a regular occurrence.
Milton Friedman's article, The Social Responsibility of Business Is To Increase Its Profits, owes its appeal to the rhetorical devices of simplicity, authority, and finality. More careful consideration reveals oversimplification and ambiguity that conceals empirical errors and logical fallacies. It is false that business does, or would, operate exclusively in economic terms, that managers concentrate obsessively on profitability, and that ethics can be marginalized. These errors reflect basic contradictions: an apolitical political base, altruistic agents of selfishness, and good deriving from greed.
Aspect-oriented software development is motivated by the desire to localize definitions of independent concerns in the software. Localized definitions are a form of modularity that achieve separation of concerns in the design, but the non-hierarchical character of the concerns creates structure clashes with the hierarchical modular constructs in conventional programming languages. Aspect-oriented modularity achieves the benefits of localized definitions, but at the costs of complexity both in the tools that weave the aspects into code and in the task of understanding the interactions among definitions. Aspect-oriented modularity is one of several types of modularity that have emerged in the past decade or so. Much of this growth has been triggered by the penetration of computing and information technology into all aspects of modern life. Much of the conventional wisdom of software engineering, especially about modularity, is challenged by the shift from in-house software development to composition of Internet-accessible resources and by the involvement of end-user programmers in development. This talk will discuss the larger landscape of modularity in modern computing and information systems, including the motivations for introducing modularity, the sorts of information that can usefully be modularized, mechanisms that bridge from the modular abstractions to running code, generality/power tradeoffs, and examples that show this diversity.
This paper reports data from a study that seeks to characterize the differences in design structure between complex software products. We use design structure matrices (DSMs) to map dependencies between the elements of a design and define metrics that allow us to compare the structures of different designs. We use these metrics to compare the architectures of two software products--the Linux operating system and the Mozilla Web browser--that were developed via contrasting modes of organization: specifically, open source versus proprietary development. We then track the evolution of Mozilla, paying attention to a purposeful "redesign" effort undertaken with the intention of making the product more "modular." We find significant differences in structure between Linux and the first version of Mozilla, suggesting that Linux had a more modular architecture. Yet we also find that the redesign of Mozilla resulted in an architecture that was significantly more modular than that of its predecessor and, indeed, than that of Linux. Our results, while exploratory, are consistent with a view that different modes of organization are associated with designs that possess different structures. However, they also suggest that purposeful managerial actions can have a significant impact in adapting a design's structure. This latter result is important given recent moves to release proprietary software into the public domain. These moves are likely to fail unless the product possesses an "architecture for participation."
What don't we know, and why don't we know it? What keeps ignorance alive, or allows it to be used as a political instrument? Agnotologyâthe study of ignoranceâprovides a new theoretical perspective to broaden traditional questions about "how we know" to ask: Why don't we know what we don't know? The essays assembled in Agnotology show that ignorance is often more than just an absence of knowledge; it can also be the outcome of cultural and political struggles. Ignorance has a history and a political geography, but there are also things people don't want you to know ("Doubt is our product" is the tobacco industry slogan). Individual chapters treat examples from the realms of global climate change, military secrecy, female orgasm, environmental denialism, Native American paleontology, theoretical archaeology, racial ignorance, and more. The goal of this volume is to better understand how and why various forms of knowing do not come to be, or have disappeared, or have become invisible.
: This essay warns of eroding accountability in computerized societies. It argues that assumptions about computing and features of situations in which computers are produced create barriers to accountability. Drawing on philosophical analyses of moral blame and responsibility, four barriers are identified: (1) the problem of many hands, (2) the problem of bugs, (3) blaming the computer, and (4) software ownership without liability. The paper concludes with ideas on how to reverse this trend. If a builder has built a house for a man and has not made his work sound, and the house which he has built has fallen down and so caused the death of the householder, that builder shall be put to death. If it destroys property, he shall replace anything that it has destroyed; and, because he has not made sound the house which he has built and it has fallen down, he shall rebuild the house which has fallen down from his own property. If a builder has built a house for a man and does not make his wor...
See no evil. Logic Magazine
- M Posner
The dataset nutrition label (2nd gen): Leveraging context to mitigate harms in artificial intelligence
- K S Chmielinski
- S Newman
- M Taylor
Ethical tech starts with addressing ethical debt
- C Fiesler
- N Garrett
Keynote interview: Karen Hao in conversation with William Isaac
- W Isaac
- K Hao
Human-centred design considered harmful
- J Pasanen
Timnit Gebru is building a slow AI movement
- E Strickland
Ethics in technology practice: A Toolkit. The Markkula Center for Applied Ethics at
- S Vallor
- B Green
- I Raicu