Conference PaperPDF Available

Notes on bias in the socio-material realization of AI technologies

Authors:
1
Notes on bias in the socio-material realization of AI technologies
Hans Radder (H.Radder@vu.nl)
(These notes are a revised version of an extended abstract submitted to, and presented at, the
conference on Bias in AI and Neuroscience, 17-19 June 2019, Radboud University Nijmegen)
1. A comprehensive account of technology
Artificial intelligence (AI) is not merely a matter of the application of abstract, disembodied
algorithms. An adequate interpretation and evaluation of the epistemic, social and moral
significance of AI should include its material and social realization in technological systems.
Elsewhere (Radder 2009) I have developed a comprehensive account of technology. It
consists of two parts. First, it defines and explains a (type of) technology as ‘a (type of)
artifactual, functional system with a certain degree of stability and reproducibility’. This first
part is a theoretical analysis. As such, it makes a conditional claim: it explains what needs to
be done if we want to realize particular technologies. But it does not tell you whether the
actual realization of these technologies will be feasible and desirable. Therefore, this analysis
should be complemented by a second part that examines the socio-material feasibility and the
normative desirability of technologies.
Take, for example, contact lens technology. The lens itself is clearly an artifactual object, not
a natural one. Its usual function is to improve human eyesight. Its functioning is supposed to
be stable (depending on the type of lens, for a day, a week, a month, or longer). Many lenses
of a particular type exist, so they are reproducible; the lens is not an isolated object but always
part of a system. What we take as our system depends on our analytic focus. It may be
relatively limited (in the case of lens and eye), but also quite broad (lens, eye, cleansing fluid,
opticians selling these fluids, factories producing such fluids, and so on). Finally, successfully
realizing a working contact lens requires its embedding in a suitable socio-material
environment: on the material side, observing sufficient hygiene, avoiding places with too
much dirt and dust in the air; and on the social side, safety regulations for lens and liquids,
health care certificates for professional opticians, legal arrangements in cases of malpractices,
and so on.
This account immediately entails a variety of conditions for the feasibility and desirability of
contact lens technology. Its feasibility requires a control of both the system and the relevant
parts of its environment. Successful use requires a measure of discipline: observing hygiene,
for instance, cannot be expected from young children or dementia patients. For individuals the
desirability of the technology depends on whether or not potential users see the advantages of
wearing contact lenses as more important than the required discipline and costs. At a social
scale, the desirability of the technology will depend on considerations of economic
profitability, environmental safety, societal costs, and the like.
2. AI technologies and some general biases in interpreting their significance
In general terms, it is easy to see how this analysis applies to AI technologies. Successful AI
technologies are also artifactual, functional systems with a certain degree of stability and
2
reproducibility.
1
System and/or environment include computers, users, factories, providers
and their servers, commercial companies, regulatory bodies, educational facilities, technology
policies, hackers, cybercriminals, critical social stakeholders, and so on and so on. A
corresponding wide range of questions can and should be posed concerning the feasibility and
desirability of successfully realizing particular AI technologies.
Below, I will examine a few examples of AI technologies and their possible biases in
(somewhat) more detail. But first, there is the question of what we mean by the notion of bias.
First, bias is not the same as error or mistake.
2
The latter may be structural but also
occasional. In contrast, bias seems to be inherently structural. Second, identifying something
as a bias requires having some knowledge of its causes and of the way it might be corrected.
We may, for instance, call a particular approach biased when it uses a one-sided assumption
and (intentionally or unintentionally) disregards relevant assumptions that are more
comprehensive. Third, biased actions or views have consequences that are undesirable in a
normative sense. They may go against epistemic, technological, social or moral norms.
Therefore, criticisms of biases in AI should include reflection on the nature and the grounds
of such norms.
3
Finally, bias in AI’ may refer to built-in features of the AI technologies
themselves or to views about their broader meaning and significance.
Interpreting AI technologies in the way I have done enables us to identify some quite general
biases concerning AI that can be found frequently in both academic and popular discourses. It
is, for instance, often claimed that ‘because of the huge impacts of internet and new social
media, we now live in a virtual world’. This claim is based on a particularly one-sided, and
hence biased, view of AI technologies, which seems to presuppose that the many material and
social elements of AI systems and their environments are falling readymade out of the sky.
However, without the extensive material and social production of these elements, there will be
no internet and no social media at all. The fact that nowadays a substantial part of these
production processes takes place in non-western, cheap-labour countries does not mean that
they do not exist. A second example of general bias concerning AI is the claim that AI systems
are autonomous. Thus, the recent Programmakrant of the Internationale School voor
Wijsbegeerte (https://isvw.nl/programmakrant-2019/, p. 3) claims: ‘All experts agree on this
one thing: in the future an artificially intelligent system will be able to act autonomously and
freely’. The above account of technology, however, implies that no technology is
autonomous: no technology can work successfully without a variety of individual and
collective human contributions.
3. A more detailed example: recognizing audio profiles
1
Thus, AI is not ‘a mathematical model’ (as Homi Shlod, and quite a few others, claim in
Conference programme (2019, 7).
2
‘In computer science bias is defined as the amount of error that results from using a
simplified model of a real-world phenomenon’ (Melissa McCradden, in Conference
programme 2019, 72). The latter phenomenon is called ‘the ground truth’, which would be
fully replicating the world. However, the great problem of this notion is that only the world
itself fully replicates it.
3
In this ‘extended abstract’, I cannot include such reflection in any detail. For this reason,
quite a few of my claims will be qualified, in the sense of pointing to possible biases in and
concerning AI technologies.
3
The basic claim of AI believers in deep learning systems is that most, or even all, cognitive
achievements by human beings can be successfully modeled by deep neural networks
(DNNs). In this section, I discuss this claim by analyzing a DNN that is said to be able to
observationally discriminate between sonar echoes originating from either an undersea rock
or an undersea mine.
4
Consider first the account provided by Paul Churchland. He emphasizes the difficulty of the
task, resulting from the fact that echoes of different kinds of object are hardly distinguishable
by the human ear, while echoes of the same kind show considerable variation. The procedure
is described by him as follows:
We begin by recording fifty different mine echoes and fifty different rock echoes, a
fair sample of each. We then digitize the power profile of each echo with a frequency
analyzer, and feed the resulting vector into the bank of input units. ... We want the
output units to respond with appropriate activation levels (specifically {1,0} for a
mine; {0,1} for a rock) when fed an echo of either kind. (Churchland 1992, 345-346)
One of the networks put to this task is a three-layered network, having thirteen input units,
seven hidden units and two output units. After the learning process has been finished, the
network proves to be able to identify new rock and mine echoes in a remarkably reliable way.
Churchland stresses the autonomous and spontaneous working of the network.
Here we have a binary discrimination between a pair of diffuse and very hard-to-
define acoustic properties. Indeed, we never did define them! It is the network that has
generated an appropriate internal characterization of each type of sound, fueled only
by examples. (Churchland 1992, 346)
However, this is a very limited, and hence biased, account of this observational process. A
less one-sided analysis shows that at least the following steps are involved:
(a) Acquire ship, crew, instrumentation, technicians and scientists; sail to the chosen location
at sea.
(b) Set up and test the required apparatus for normal operation; position the ‘rocks’ and
‘mines’ on the sea floor; record the sonar echoes. Note that these rocks and mines
are artifacts, not real rocks or mines, which might be more or less covered with plants.
(c) Process the recorded sound profiles; this includes quantification, selection, discretization,
normalization and deconceptualization of the auditory profiles.
(d) Predefine the dimension of the solution space: there should be two kinds of patterns in the
profiles (not more nor less than two).
(e) Establish an external norm for improvement and correctness of the output during the
learning and testing stage.
(f) Establish a criterion for degree of success of the final output: how good is good enough?
(g) Reconceptualize the dimensionless output numbers (as ‘mine’ resp. ‘rock’).
4
See Radder (2006, chap. 5). The fact that this is a somewhat older DNN does not matter.
Current DNNs work basically in the same way, even if they may be more complex in terms of
number of ‘data’ and number of units, hidden layers and processing algorithms.
4
This analysis shows that in realizing an AI technology that can make human-like observations
much more is involved than just the processing of input activation through an automated
neural network. It illustrates my claim that no technology can work successfully without a
host of crucial human contributions. The view of AI believers that these AI systems are
autonomous agents proves to be strongly biased.
4. Four basic issues concerning AI technologies and their possible biases
Of course, much more theoretical analysis and critique and many more examples and case
studies of bias in AI are possible and required. This applies in particular to the many
uses/abuses of AI technologies for commercial interests, criminal purposes, large-scale
surveillance, warfare, and so on (see, e.g., O’Neil 2016). For reasons of space, I have to limit
myself to brief remarks on four basic sources of bias in AI.
(a) The term ‘data’ means ‘givens’, which suggests that they are unmediated and (therefore)
neutral. This suggestion, however, is patently false. As many philosophers have shown, ‘data’
are the result, the output, of empirical processes. As such they are pre-structured, in two ways.
First, through conceptual interpretation, as already argued convincingly in 1956 by Wilfrid
Sellars in his classic critique of the ‘myth of the given’ (Sellars 1997[1956]). A second pre-
structuring takes place through the specifics of the work that needs to be done to produce the
relevant ‘data’ in a stable and reproducible way (Radder 1996 and 2006).
5
Because of this
pre-structuring, the ‘data’ used as inputs for AI technologies may include several kinds of
bias.
(i) There may be bias in the selection of the ‘data’. Long ago, the computer programme
BACON was alleged to have autonomously rediscovered several scientific laws, for instance
Kepler’s laws on the motion of the planets (see Langley et al. 1987). One of the questionable
features of this claim was that it was based on preprocessed observational results from which
the noise had already been removed (see the Symposium on ‘Computer discovery and the
sociology of scientific knowledge’ 1989). A more recent example is the preprocessing that
often occurs in AI systems that are claimed to be able to identify people from their
photographs. For instance, by using as ‘data’ only pictures with centrally placed faces and no
other ‘distracting’ objects (Collins 2018, chap. 6).
(ii) In these cases, the ‘data’ have been selectively preprocessed. This fact alone does not
necessarily imply a bias, especially since we always make, and have to make, selections. It all
depends on what is claimed to have been shown. Thus, a more clearly biased view is the claim
that an AI system can generally distinguish straight and gay men, while it has been shown to
work only on pictures of (straight and) openly self-identifying gay men (Sullivan 2018, 15).
Even more biased would be the conclusion that the relevant studies have proven that being or
not being gay is a completely natural phenomenon, independent of society and culture.
(iii) The location and the language of the ‘data’ may be biased. As Sabina Leonelli (2016, 56)
points out ‘most widely used data infrastructures [in biology] are grounded in Anglo-
American science and funding structures ... and use the English language’. In a similar way,
5
Actually, my view is that the use of the term ‘data’ should be forbidden and violations be
sanctioned with a substantial fine. However, because this proposal might not find universal
agreement, I have decided to qualify the term ‘data’ by always adding quotation marks.
5
the biased primacy given to English-language sources by search engines like Google is
obvious.
(iv)A general type of human contribution to AI systems that aim to identify images of objects
or people is the (implicit or explicit) addition of information on what is meant to be
foreground and what background in the ‘data’. As in the example in the previous section (see
point (d)), the solution space is predefined: focus on the specified foreground and leave out
everything in the background.
(v) Finally, there will necessarily be ignorance of the quality of the empirical process that
produced the data in all those cases where they are generated in an open, uncontrolled
environment. This will happen, for instance, when large volumes of ‘data’ are collected
through tracking people’s smart phones in unknown circumstances. In these cases, it will be
unclear how stable and reproducible the acquired data (or their statistical aggregates) are. It is
hard to see how this practice can be methodologically justified, given the big stir that came to
be known as the replication crisis in science. Most scientists see the lack of accurate
knowledge and strict control of the ‘data’-generating procedures as the main cause of this
crisis (see, e.g., Earp and Trafimow 2015).
(b) A second issue concerns the quality of the final output of AI technologies. The relevant
normative question is: how good is good enough?
6
This question remains important even for
cases of AI technologies that perform better than humans. Suppose several such technologies
are available, but some of them are better than others while the better ones are considerably
more costly. In such a case, the question how good is good enough, given the relevant
purposes, needs to be addressed and answered by human beings. Clearly, the answers to this
question may differ widely (for instance, between a medical AI system that yields crucial
diagnoses, a household system that tells you when the milk in your fridge is past its date, and
a commercial system that allows allocating specific advertisements to individual people).
Underlying the neglect of this normative question is often a disregard of economic issues by
fervent AI believers. How much money and effort are real people or real governments willing
to spend on the development and use of specific AI technologies?
(c) A further major issue concerns AI understanding of human language. In his recent book,
Harry Collins discusses this question in great detail. He develops and defends the following
two claims.
No computer will be fluent in a natural language, pass a severe Turing test and have full
human-like intelligence unless it is fully embedded in normal human society.
No computer will be fully embedded in human society as a result of incremental
progress based on current techniques. (Collins 2018, 2)
He supports these claims by many concrete examples of the contextual specificity of human
language usage. Particularly relevant are the ones of misspellings, for instance the sentence: ‘I
am going to misspell wierd to prove a point’ (Collins 2018, 3).
7
According to all ‘intelligent’
6
Hardly any AI systems are 100% successful. For instance, the accuracy of commercially
available systems aimed at detecting text-based harmful speech remains around 70-80% (see
Antonialli et al. in Conference programme (2019, 115).
7
A comparable example can be found in the paper by Antonialli et al. in Conference
programme (2019).
6
spell checkers this sentence includes a mistake, while we humans know that this is not the
case. The philosophically interesting feature of this example is that it explicitly interweaves
object language and metalanguage.
What is missing in Collins’ account, however, is the point of the possible and actual
transformations of our normal human society’. Therefore, he does not address an alternative
to his first claim, saying that ‘no computer will be fluent in a natural language and pass a
severe Turing test unless our normal society has been transformed to fully fit the
requirements of the AI technologies’. In the case of the spelling example, this transformation
would imply a redefinition of the use of sentences that mix object language and metalanguage
as illegitimate, as showing a lack of fluency in our (transformed) natural language.
8
(d) Finally, the alternative conception of AI as the rule-guided manipulation of formal
symbols suffers from a similar bias as the DNN approach discussed in section 3 (see Radder
2006, chap. 12). Again, the problem is the one-sided focus on formal systems. It simply takes
for granted the applicability of the formal systems to their social and material realizations and
thus lacks an adequate account of the relationship between the formal symbols and the
concrete socio-material realities. What is neglected by this symbol manipulation approach is
the necessary transformation and control of the worlds in which AI technologies need to be
embedded in order to function in a stable and reproducible way.
9
Literature
Churchland, P.M. (1992). A deeper unity: Some Feyerabendian themes in
neurocomputational form. In Cognitive models of science, edited by R.N. Giere, 341-
363. Minneapolis: University of Minnesota Press.
Collins, H. (2018). Artificial intelligence: Against humanity’s surrender to computers.
Cambridge: Polity Press.
Conference programme (2019). Bias in AI and neuroscience: Full program (Radboud
University, Nijmegen.
Earp, B.D. and Trafimow, D. (2015). Replication, falsification, and the crisis of confidence
in social psychology. Frontiers in psychology 6 (article 621), 1-11. Digitally available
at https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00621/full.
Langley, P., Simon, H., Bradshaw, G. and Zytkow, J. (1987). Scientific discovery:
Computational explorations of the creative mind. Cambridge, MA: MIT Press.
Leonelli, S. (2016). Data-centric biology: A philosophical study. Chicago: University of
Chicago Press.
O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and
threatens democracy. UK: Penguin Books.
Radder, H. (1996). In and about the world (Albany, N.Y.: State University of New York
Press).
Radder, H. (2006). The world observed/The world conceived. Pittsburgh: University of
Pittsburgh Press.
8
Thus, this redefinition follows the advice of logicians like Bertrand Russell and Alfred
Tarski to keep apart object language and metalanguage in order to avoid certain paradoxes in
the logical reconstruction of language.
9
For a brief analysis of computer chess along these lines, see Radder (1996, 180-183).
7
Radder, H. (2009). Why technologies are inherently normative. In Philosophy of technology
and engineering sciences, edited by A. Meijers, 887-921. Amsterdam: Elsevier.
Sellars, W. (1997[1956]). Empiricism and the philosophy of mind. Cambridge, MA: Harvard
University Press.
Sullivan, E. (2019). Understanding from machine learning models. Paper presented at VU
meeting on Philosophy of Science and Technology (February 15).
Symposium on ‘Computer discovery and the sociology of scientific knowledge’ (1989).
Social studies of science 19 (4), 563-695.
ResearchGate has not been able to resolve any citations for this publication.
Article
Neural network models of cognitive processes provide us with an entirely new conception of perceptual recognition, theoretical knowledge, learning and conceptual change. This brain- based perspective on cognition bears directly on some long-standing issues in the philosophy of science. In particular, it appears to vindicate five of the more central themes of Paul Feyerabend’s philosophy of science.
Chapter
This chapter focuses on the philosophy of technology. Any systematic philosophical discussion of the normativity of technology should be based on a plausible account of the very idea of 'technology'. Explaining and defending the claim that technologies are inherently normative requires, first, a plausible account of the notions of 'technology' and 'normativity' is essential. The distinct elements of this characterization of technologies can be described as a system. A system is any aggregate (or collective) of mutually interacting material entities within a certain region of space and time. Thus, a system possesses not only a spatial but also a temporal dimension, which allows to see processes as systems. Although the notion of a system may also be used in more substantial ways, this unassuming definition is appropriate in the present context. It may also be phrased by saying that technologies have a systemic character, because they result from bringing, and keeping, together two or more material entities.
Article
By the late 60s, every good materialist expected that epistemological theory would one day make explanatory contact, perhaps even a reductive contact, with a proper theory of brain function. Not even the most optimistic of us, however, expected this to happen in less than fifty years, and most would have guessed a great deal longer. And yet the time has arrived. Experimental neuroscience has revealed enough of the brain’s microphysical organization, and mathematical analysis and computer simulation have revealed enough of its functional significance, that we can now address epistemological issues directly. Indeed, we are in a position to reconstruct, in neurocomputational terms, issues in the philosophy of science specifically. This is my aim in what follows.
Artificial intelligence: Against humanity's surrender to computers
  • H Collins
Collins, H. (2018). Artificial intelligence: Against humanity's surrender to computers. Cambridge: Polity Press. Conference programme (2019). Bias in AI and neuroscience: Full program (Radboud University, Nijmegen.
Replication, falsification, and the crisis of confidence in social psychology'
  • B D Earp
  • D Trafimow
Earp, B.D. and Trafimow, D. (2015). 'Replication, falsification, and the crisis of confidence in social psychology'. Frontiers in psychology 6 (article 621), 1-11. Digitally available at https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00621/full.
Weapons of math destruction: How big data increases inequality and threatens democracy
  • C O'neil
O'Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. UK: Penguin Books.