Aurélie Herbelot’s research while affiliated with University of Trento and other places

The development of Large Language Models (LLMs) as a component of systems such as ChatGPT foregrounds a range of issues which can only be analysed through novel interdisciplinary approaches. Our pilot project `Exploring novel figurative language to conceptualise Large Language Models’, funded by Cambridge Language Sciences, is aimed at helping both specialists and non-specialists gain a more precise understanding of the technology and its implications. In this poster, we use `slop' as a metaphor to highlight one aspect of LLMs, but situate the issue in a broader context. We use ‘slop’ to mean text delivered to a reader which is of little or no value to them (or is even harmful) OR is so verbose or convoluted that the value is hidden. Examples of slop include: over-general instructions, unnecessary terms and conditions and spam email. The term `slop' is sometimes used specifically for AI-generated content but in our usage it predates machine-generated text. Slop arises when desiderata other than communication with readers determine text production or delivery. Systems incorporating LLMs may become `supersloppers': tools for the creation and delivery of more and more pointless text. Because so much slop already exists, and because it is often repetitious, maximizing the text on which LLMs are trained results in systems which excel at production of slop. It is useful to think of slop as a category because it draws attention to specific ways in which the arenas we are examining are far removed from the basic setting of human conversation.

In this paper, we derive a notion of word meaning in context that characterizes meaning as both intensional and conceptual. We introduce a framework for specifying local as well as global constraints on word meaning in context, together with their interactions, thus modelling a wide range of lexical shifts and ambiguities observed in utterance interpretation. We represent sentence meaning as a situation description system, a probabilistic model which takes utterance understanding to be the mental process of describing to oneself one or more situations that would account for an observed utterance. We show how the system can be implemented in practice, and apply it to examples containing various contextualisation phenomena.

Empirical insights into language processing have a philosophical relevance that extends well beyond philosophical questions about language. This chapter will discuss this wider relevance: We will consider how experimental philosophers can examine language processing in order to address questions in several different areas of philosophy. To do so, we will present the emerging research program of experimental argument analysis (EAA) that examines how automatic language processing shapes verbal reasoning – including philosophical arguments. The evidential strand of experimental philosophy uses mainly questionnaire-based methods to assess the evidentiary value of intuitive judgments that are adduced as evidence for philosophical theories and as premises for philosophical arguments. Extending this prominent strand of experimental philosophy, EAA underpins such assessments, extends the scope of the assessments, and expands the range of the empirical methods employed: EAA examines how automatic inferences that are continually made in language comprehension and production shape verbal reasoning, and draws on findings about comprehension biases that affect the contextualisation of such default inferences, in order to explain and expose fallacies. It deploys findings to assess premises and inferences from premises to conclusions, in philosophical arguments. To do so, it adapts methods from psycholinguistics and recruits methods from computational linguistics.

This paper presents a novel framework for evaluating Neural Language Models' linguistic abilities using a constructionist approach. Not only is the usage-based model in line with the underlying stochastic philosophy of neural architectures, but it also allows the linguist to keep meaning as a determinant factor in the analysis. We outline the framework and present two possible scenarios for its application.

Neural language models have seen a dramatic increase in size in the last years. While many still advocate that 'bigger is better', work in model distillation has shown that the number of parameters used by very large networks is actually more than what is required for state-of-the-art performance. This prompts an obvious question: can we build smaller models from scratch, rather than going through the inefficient process of training at scale and subsequently reducing model size? In this paper, we investigate the behaviour of a biologically inspired algorithm, based on the fruit fly's olfactory system. This algorithm has shown good performance in the past on the task of learning word embeddings. We now put it to the test on the task of semantic hashing. Specifically, we compare the fruit fly to a standard binary network on the task of generating locality-sensitive hashes for text documents, measuring both task performance and energy consumption. Our results indicate that the two algorithms have complementary strengths while showing similar electricity usage.

Empirical insights into language processing have a philosophical relevance that extends well beyond philosophical questions about language. This chapter will discuss this wider relevance: We will consider how experimental philosophers can examine language processing in order to address questions in several different areas of philosophy. To do so, we will present the emerging research program of experimental argument analysis (EAA) that examines how automatic language processing shapes verbal reasoning – including philosophical arguments. The evidential strand of experimental philosophy uses mainly questionnaire-based methods to assess the evidentiary value of intuitive judgments that are adduced as evidence for philosophical theories and as premises for philosophical arguments. Extending this prominent strand of experimental philosophy, EAA underpins such assessments, extends the scope of the assessments, and expands the range of the empirical methods employed: EAA examines how automatic inferences that are continually made in language comprehension and production shape verbal reasoning, and draws on findings about comprehension biases that affect the contextualisation of such default inferences, in order to explain and expose fallacies. It deploys findings to assess premises and inferences from premises to conclusions, in philosophical arguments. To do so, it adapts methods from psycholinguistics and recruits methods from computational linguistics.

It has been shown that Recurrent Artificial Neural Networks automatically acquire some grammatical knowledge in the course of performing linguistic prediction tasks. The extent to which such networks can actually learn grammar is still an object of investigation. However, being mostly data-driven, they provide a natural testbed for usage-based theories of language acquisition. This mini-review gives an overview of the state of the field, focusing on the influence of the theoretical framework in the interpretation of results.

Philosophers are often credited with particularly well-developed conceptual skills. The ‘expertise objection’ to experimental philosophy builds on this assumption to challenge inferences from findings about laypeople to conclusions about philosophers. We draw on psycholinguistics to develop and assess this objection. We examine whether philosophers are less or differently susceptible than laypersons to cognitive biases that affect how people understand verbal case descriptions and judge the cases described. We examine two possible sources of difference: Philosophers could be better at deploying concepts, and this could make them less susceptible to comprehension biases (‘linguistic expertise objection’). Alternatively, exposure to different patterns of linguistic usage could render philosophers vulnerable to a fundamental comprehension bias, the linguistic salience bias, at different points (‘linguistic usage objection’). Together, these objections mount a novel ‘master argument’ against experimental philosophy. To develop and empirically assess this argument, we employ corpus analysis and distributional semantic analysis and elicit plausibility ratings from academic philosophers and psychology undergraduates. Our findings suggest philosophers are better at deploying concepts than laypeople but are susceptible to the linguistic salience bias to a similar extent and at similar points. We identify methodological consequences for experimental philosophy and for philosophical thought experiments.

In this theoretical paper, we consider the notion of semantic competence and its relation to general language understanding—one of the most sough-after goals of Artificial Intelligence. We come back to three main accounts of competence involving (a) lexical knowledge; (b) truth-theoretic reference; and (c) causal chains in language use. We argue that all three are needed to reach a notion of meaning in artificial agents and suggest that they can be combined in a single formalisation, where competence develops from exposure to observable performance data. We introduce a theoretical framework which translates set theory into vector-space semantics by applying distributional techniques to a corpus of utterances associated with truth values. The resulting meaning space naturally satisfies the requirements of a causal theory of competence, but it can also be regarded as some ‘ideal’ model of the world, allowing for extensions and standard lexical relations to be retrieved.

In human semantic cognition, proper names (names which refer to individual entities) are harder to learn and retrieve than common nouns. This seems to be the case for machine learning algorithms too, but the linguistic and distributional reasons for this behaviour have not been investigated in depth so far. To tackle this issue, we show that the semantic distinction between proper names and common nouns is reflected in their linguistic distributions by employing an original task for distributional semantics, the Doppelg\"anger test, an extensive set of models, and a new dataset, the Novel Aficionados dataset. The results indicate that the distributional representations of different individual entities are less clearly distinguishable from each other than those of common nouns, an outcome which intriguingly mirrors human cognition.