Project

Forschungslogiken in the text-based Digital Humanities: Analysing evaluative practices after the machine learning turn

Goal: We analyse evaluation in German-language online (a) comments to recipes and (b) lay book reviews.
We approach these evaluative practices from three perspectives, (1) Close-reading/heuristics/qualitative; (2) Corpus linguistics; (3) Machine learning/sentiment analysis, in order to reflect on the impact of theoretical assumptions and empirical methodologies on research questions, results and interpretation.

Updates
0 new
2
Recommendations
0 new
2
Followers
0 new
5
Reads
0 new
95

Project log

Thomas C. Messerli
added 2 research items
Prominent among the social developments that the web 2.0 has facilitated is digital social reading (DSR): on many platforms there are functionalities for creating book reviews, ‘inline’ commenting on book texts, online story writing (often in the form of fanfiction), informal book discussions, book vlogs, and more. In this article, we argue that DSR offers unique possibilities for research into literature, reading, the impact of reading and literary communication. We also claim that in this context computational tools are especially relevant, making DSR a field particularly suitable for the application of Digital Humanities methods. We draw up an initial categorization of research aspects of DSR and briefly examine literature for each category. We distinguish between studies on DSR that use it as a lens to study wider processes of literary exchange as opposed to studies for which the DSR culture is a phenomenon interesting in its own right. Via seven examples of DSR research, we discuss the chosen approaches and their connection to research questions in literary studies.
The aim of the special issue is to present theory-conscious studies of selected facets of digital cultures of evaluation across a broad range of discourses on the social web. At the same time, bringing together papers from linguistics, literary studies, and digital humanities, it is our aim to take stock of common denominators, but also differences, in method and theory, instilling discussion for a cross-disciplinary data-driven and data-intensive inquiry of valuation and evaluation within “data humanities”. The types of discourse covered will include online lay book reviews on platforms such as Goodreads and LovelyBooks, as well as reviews of non-literary “products“ such as cooking recipes, restaurant reviews, and concerts.
Thomas C. Messerli
added a research item
While interdisciplinary research on metaphor is abundant (Eggs, 2000; Semino & Demjén, 2017; Veale et al., 2016), it is still scarce in Digital Humanities. At the intersection of literary studies, corpus stylistics, and digital humanities, we present an exploratory quantitative metaphor analysis of a corpus of German language lay book reviews. Using a deliberately simple methodological approach that operates on seed words for conceptual sources and targets we investigate how reading experiences of literary texts are metaphorically presented by reviewers. We explore a corpus of approx. 1.3 mill. book reviews for metaphors used to conceptualize the target domain READING EXPERIENCE. In line with conceptual metaphor theory, metaphors in language are understood as closely linked to human thought processes and experiences (Lakoff & Johnson, 1980, pp. 4–6; Shutova, 2017). They are mappings from typically more basic experiential source domains (LIFE) to more abstract target domains (READING EXPERIENCE), indicated by indirectly used lexis (the words come, end, and road in “we've come to the end of our road”, VUAMC, Steen et al., 2010). Starting from findings on literature reviews in English (Stockwell, 2009; Nuttall & Harrison, 2018) and on reviews in German (Köhler, 1999), we analyze metaphor patterns in social reading networks, with a particular focus on the mapping READING EXPERIENCE IS MOTION. The main aim at this stage is to draw up a first typology of mappings.
Thomas C. Messerli
added a research item
Prominent among the social developments that the web 2.0 has facilitated is digital social reading (DSR): on many platforms there are functionalities for creating book reviews, 'inline' commenting on book texts, online story writing (often in the form of fanfiction), informal book discussions, book vlogs, and more. In this article we argue that DSR offers unique possibilities for research into literature, reading, the impact of reading and literary communication. We also claim that in this context computational tools are especially relevant, making DSR a field particularly suitable for the application of Digital Humanities methods. We draw up an initial categorization of research aspects of DSR and briefly examine literature for each category. We distinguish between studies on DSR that use it as a lens to study wider processes of literary exchange as opposed to studies for which the DSR culture is a phenomenon interesting in its own right. Via seven examples of DSR research we discuss the chosen approaches and their connection to research questions in literary studies.
Thomas C. Messerli
added an update
The Basel/Zurich research cluster of SNSF “Digital Lives” projects is organizing a three-day
conference on “Digital Practices. Reading, writing and evaluation on the web” at the
University of Basel, Switzerland. The conference will take place from June 3rd to June 5th
2020.
Deadline for submission: January 17th, 2020
Notifications of acceptance: February 28th, 2020
Further details in the Flyer (PDF).
 
Thomas C. Messerli
added a research item
Our contribution reports on a research project that addresses distinct research methodologies as well as the epistemic dimension of textual Digital Humanities. In a case study that combines machine learning applications of sentiment detection with more traditional manual and semi-automatic text analysis, such as close reading and annotation, we study evaluative language use in web 2.0 discourses as an example of text production in the context of digitization. Our practical aim in these case studies is to examine how available dictionaries, algorithms, and machine learning procedures work on German textual data extracted from the social web. Addressing the question of how users convey their evaluation of books, we find and generate sets of answers to the question itself using traditional humanities and machine learning approaches, and on the other hand shed light on the role of the distinct research epistemologies as well as our own research practices and attitudes which frame or even shape our findings. One of the central aims for our multidisciplinary mixed-method research design is thus a thorough reflection of the premises underlying our own research practice as different types of actors in the field of textual humanities. Scrutiny will be directed at built-in biases of training corpora used for machine-learning, but also at researchers’ guiding schemata for scientific/scholarly action: these are mutually determined by and determine reality and humanity, showing in research practice – as types of research questions, argumentation, interpretation, and methodology. Our methods consist in a range of tools for reflective thinking, including meta-annotation of the (semi-)automatic and manual tags and memos for documenting observations. On this level, we will address the following questions: 1. Theoretical frameworks: What understanding of language and communication motivate the different approaches? What assumptions are made about the world, reality and humanity? 2. Operationalization: What, if any, linguistic and literary categories and theories are being operationalized? What covert and overt decisions inform existing analytical practices? 3. Results and interpretation: What rationale is used to discuss and interpret results? How do interpretations of results tie in with the respective theoretical frameworks? What open questions are foregrounded? What role and importance are given to the interpretation of data within the overall research projects? How is subjectivity discussed? 4. Research practice: What explicit and implicit rationale motivates the employment of particular research processes? Using the empirical research on web 2.0 discourses throughout its stages for our enquiry, we examine the underlying epistemologies that our team brings to the table, extrapolating from here to our field of textual DH generally. Our goal is not to find the best method, but to reveal underlying worldviews and epistemologies that guide the analysis through principled reflective thinking.
Thomas C. Messerli
added an update
Thomas C. Messerli
added a project goal
We analyse evaluation in German-language online (a) comments to recipes and (b) lay book reviews.
We approach these evaluative practices from three perspectives, (1) Close-reading/heuristics/qualitative; (2) Corpus linguistics; (3) Machine learning/sentiment analysis, in order to reflect on the impact of theoretical assumptions and empirical methodologies on research questions, results and interpretation.