What’s in the brain that ink may character ....: A Quantitative Narrative Analysis of Shakespeare’s 154 Sonnets for Use in Neurocognitive Poetics

Full text

!
P
!
1!
What’s in the brain that ink may character ….: A Quantitative 1!
Narrative Analysis of Shakespeare’s 154 Sonnets for Use in 2!
Neurocognitive Poetics 3!
Arthur M. Jacobs 1, 2, 3, Sarah Schuster 1,4, Shuwei Xue8 , & Jana Lüdtke1, 2 4!
5!
1) Department of Experimental and Neurocognitive Psychology, Freie Universität Berlin, 6!
Germany 7!
2) Dahlem Institute for Neuroimaging of Emotion (D.I.N.E.), Berlin, Germany 8!
3) Center for Cognitive Neuroscience Berlin (CCNB), Berlin, Germany 9!
4) Universität Salzburg, Centre for Cognitive Neuroscience, Hellbrunner Strasse 34, 5020 10!
Salzburg, Austria 11!
12!
13!
14!
Correspondence: Arthur M. Jacobs 15!
Department of Experimental and Neurocognitive Psychology, Freie Universität Berlin, 16!
Habelschwerdter Allee 45 , D-14195 Berlin, Germany. 17!
Email: ajacobs@zedat.fu-berlin.de 18!
19!
20!
21!
22!
Short title. A Quantitative Narrative Analysis of Shakespeare’s 154 Sonnets 23!
24!
Keywords. Neurocognitive Poetics, Quantitative Narrative Analysis, Coh-Metrix, SEANCE, 25!
TAACO, machine learning, Digital humanities, Sonnets. 26!
27!
28!
29!
30!

!
P
!
2!
Abstract 1!
Shakespeare’s sonnets count among the most aesthetically successful or popular pieces of 2!
verbal art and have been the object of countless essays by literary critics and of theoretical – as 3!
opposed to empirical – scientific studies. To pave the ground for future empirical studies in 4!
Neurocognitive Poetics, we ask 11 questions about potentially relevant properties of this 5!
outstanding corpus of western poetry and answer them by help of Quantitative Narrative 6!
Analysis (QNA) tools. Using both tools for cognitive and affective-aesthetic analysis, in the 7!
first two parts of this paper we quantify aspects of the sonnets’ readability and 8!
comprehensibility, surprisal, emotion and mood potential, as well as indices of their thematic 9!
richness, symbolic imagery, and semantic association potential. In the final part, we first 10!
demonstrate how the results of these QNAs can be used for generating testable predictions for 11!
empirical studies of literature. Second, we show how they can be combined with computational 12!
modeling for identifying those of the many quantifiable sonnet features that play a potential 13!
key role in their reception. Finally, we feed the QNA data into a machine learning algorithm 14!
which successfully classifies the 154 sonnets into two main categories, i.e. the „young man“ 15!
and „dark lady“ poems. 16!
17!
18!
19!
20!
21!
22!
23!
24!

!
P
!
3!
Introduction 1!
Although Shakespeare’s works must count among the most successful and popular pieces of 2!
verbal art and have been the object of countless essays by literary critics and of theoretical – 3!
as opposed to empirical – scientific studies (e.g., Jakobson & Jones, 1970), they still seem full 4!
of surprises even for eminent experts. Thus, in the preface of his book on the language of 5!
Shakespeare “Think of my words”, Crystal (2008, preface), states: “Everytime I do even the 6!
most menial search of my Shakespeare database, I discover something I have never noticed 7!
before”. 8!
Shakespeare’s sonnets first appeared in 1609, a collection of about 18000 words (17515 9!
according to our counts) that have changed the world and the way our mindbrains feel and 10!
think about it (Schrott & Jacobs, 2011). The majority of the sonnets (1-126), termed fair youth 11!
or young man sonnets, are addressed to a young man, with whom the poet is said to have had 12!
an intense relationship. In sonnets 1-17 the poet tries to convince the young man to marry and 13!
have children (e.g., beautiful children that will look just like their father, ensuring his 14!
immortality). Many of the remaining sonnets in the young man sequence focus on the power of 15!
poetry and pure love to defeat death and "all oblivious enmity". Sonnets 127 - 154, termed the 16!
dark lady or mistress sonnets, are said to speak to a promiscuous and scheming woman. Both 17!
the poet and his fair youth have become obsessed with the raven-haired temptress in these 18!
sonnets, and the poet's whole being is at odds with his insatiable sickly appetite (147.4). The 19!
tone is distressing, with language of sensual feasting, uncontrollable urges, and sinful 20!
consumption (cf. http://www.shakespeare-online.com/sonnets/sonnetintroduction.html). This 21!
sequence is sometimes considered a proto-sketch for Shakespeare’s drama Othello, although 22!
the true actors in lyric are words, not characters in conflict: The drama in the sonnets is thus 23!
produced by new linguistic strategies and internal changes in topic or syntactic structure 24!
(Vendler, 1997). 25!
26!
General features of sonnets 27!
English or Shakespearean sonnets (from the italian word sonetto meaning a small song or 28!
lyric) typically are decasyllabic 14-liners in iambic pentameter. Besides a clear surface 29!
structure of three (isomorphic) quatrains and one (anomalous) couplet, and a typical – but not 30!
absolute – rhyme scheme (abab cdcd efef gg) sonnets feature structural coherence, logical 31!
development and unit of play. According to Vendler (1997) a sonnet presents a conundrum and 32!
unfolds itself in a developing dynamic of feeling and thought marked by a unifying play of 33!
mind and language. 34!
35!
The sonnet’s versification encourages the greater use of monosyllabic words (of which English 36!
is much richer than, e.g., German) and it allows metrical variation to be introduced more 37!
easily. Sonnets can be said to have a comparatively volatile thought structure with changes in 38!
rhyme-sound from quatrain to quatrain encouraging new turns of thought, and a step-by-step 39!
movement towards the definite closure provided by the couplet (Wainwright, 2011). Thus, a 40!
sonnet’s structure appears good for argument (e.g., Shakespeare’s sonnet 138) or polemics 41!
(e.g., Milton), sequences exploring different aspects of a single motif, e.g. love. A sonnet also 42!

!
P
!
4!
is a system in motion: Its four parts can be set in a number of logical relations (e.g., successive 1!
and equal, hierarchical, contrastive or contradictory, successively “louder” or “softer) and play 2!
with changes of agency or speech act, rhetorical address, grammatical form, discursive texture 3!
each producing its own emotional dynamic moves - within the speaker’s mind and heart - and 4!
poetic effects in readers mindbrains (Vendler, 1997). Following Vendler, the dynamic can be 5!
assimilated to a narrowing down (funnel-shape) movement from quatrain/Q1 (e.g., wide 6!
epistemological field) to Q2 (e.g., queries, contradicts, subverts position in Q1) to Q3 (e.g., 7!
subtlest, most comprehensive/truthful position and solution) to the final couplet/C 8!
(summarizing, ironic or expansive coda - restating semantically the body of the sonnet, i.e., 9!
Q1 to 3 - with a crucial tonal difference and an often a self-ironizing turn to the proverbial or 10!
idiomatic, e.g., sonnet 94). The so-called couplet tie are the significant, usually thematically 11!
central words from the body (Q1-3) repeated in the couplet. In addition, many of the sonnets 12!
also exhibit the two-part (octave-sestet) structure of Petrarchan sonnets, i.e. the first eight lines 13!
logically or metaphorically stand against the last six, e.g. as a problem-solution, question-14!
answer or generalization-application dynamic. 15!
16!
In a way this dynamic parallels the narrowing of the „text world“ of a reader during the 17!
incremental reading act in the sense that the number of potential events, characters or new text 18!
world referents (e.g., entities, attributes, relations) decreases towards the end of a text. As 19!
argued by Steen (2004), this can have notable effects on the way readers process poetic text 20!
elements and metaphors in particular. 21!
22!
On the one hand, sonnets are comparable to narrative in that practically each sonnet “tells a 23!
little story”. This is an advantage, since it allows to supplement qualitative and typological text 24!
analyses (e.g., Jakobson & Lévi-Strauss, 1962, Jakobson & Jones, 1970; Meireles, 2005) by 25!
quantitative narrative analysis (QNA) which requires a minimum of text length and structural 26!
variability to provide reliable results. On the other hand, sonnets differ from narratives in form 27!
and content structure, since they exhibit the potential for lots of new beginnings, fresh angles, 28!
different tones (intimate, meditative, comic, polemical) and do not need a narrative’s thread 29!
(Wainwright, 2011). Their nice juxtaposition of the strict regularity and continuity of form 30!
against the other likely changes in subject, mood or style make them ideal candidates for 31!
evoking affective and aesthetic reader responses, and thus for scientific studies of literary 32!
experience (Delmonte, 2016; Jacobs, 2015c, 2016a). If the poet himself indeed learned to find 33!
strategies to enact feeling in form and replicate human (affective) responses in a unique 34!
richness and virtuosity of linguistic forms throughout the composition of the 154 sonnets 35!
(Vendler, 1997), then readers may well sequentially and incrementally acquire new insights 36!
into their own feelings. Thus, the principle of interest to sustain rereadings of sonnets are 37!
believed to be their discourse variety and fertility in structural complexity (cf. Simonton, 38!
1989). 39!
40!
In sum, sonnets offer a rich structure at all textual processing levels and thus a great potential 41!
for multilevel poetic effects (cf. Jacobs et al., 2016a). Due to their short length, sonnets are 42!
relatively easy to manage for both the writer and the reader. On the other hand, they also are 43!

!
P
!
5!
long enough to contain and induce alternations in moods (e.g., mood empathy changes; Jacobs 1!
et al., 2016a; Lüdtke et al., 2014) and can be considered a great repository of moods induced 2!
by treating the plot elegiacally, sardonically, ironically and tragically (Vendler, 1997). In sum, 3!
they seem to be ideal candidates for empirical studies in neurocognitive poetics (Jacobs, 4!
2015a,b). 5!
6!
Quantitative Narrative Analysis (QNA) of Shakespeare’s sonnets and dramas 7!
Shakespeare’s works were also among the first to have been analysed through the use of 8!
computer tools that empower researchers to quickly analyze large bodies of literary texts on 9!
many characteristics of language and discourse, thus offering predictions about their aesthetic 10!
success, artistic worth or comprehensibility (e.g., Delmonte, 2014; Graesser et al., 2004, 2010, 11!
2011; Simonton, 1989, 1990). In his seminal QNA study of Shakespeare’s sonnets, Simonton 12!
(1989) discovered that the sonnets with superior aesthetic success (as assessed by an archival 13!
popularity measure) had the following distinctive features: (1) treat specific themes, (2) display 14!
considerable thematic richness in the number of issues discussed, (3) exhibit greater linguistic 15!
complexity as gauged by such objective measures as the type-token ratio (i.e., the ratio of 16!
different words to total words as an index of lexical variability/verbal complexity) and 17!
adjective-verb quotient (i.e., the proportion of adjectives to verbs as an altemative gauge of 18!
linguistic complexity), and (4) feature more primary process imagery (as assessed by 19!
Martindale's, 1975, Regressive Imagery Dictionary/RID). 20!
21!
Simonton identified a few supremely popular sonnets standing out from the universe of 154 22!
(sonnets 29, 30, 73, and 116). He also identified 24 topics or specific themes, such art, beauty 23!
and love, with variable frequencies of occurrence in the 154 sonnets: Love in its various 24!
facettes (e.g., the intensity and power of love, its increase or decrease, its constructive or 25!
destructive force, friendly, tender, or altruistic love, fraternal love, love in relation to virtue and 26!
happiness, the sacrifices of love) was by far the dominant topic occurring in more than 100 27!
sonnets. Simonton assumed that the quantifiable features type-token ratio, number of unique 28!
words, adjective-verb quotient, broken lines and run-on lines – and perhaps thematic richness – 29!
determine a sonnet's arousal potential and thus its aesthetic value (via complexity, novelty, 30!
surprise, and other collative properties, Berlyne, 1971; Cupchik, 1986; Marin et al., 2016). 31!
32!
In a more recent QNA of Shakespeare’s sonnets using a novel tool called SPARSAR that 33!
allows both a broader and deeper form and content analysis than Simonton’s, Delmonte (2016) 34!
challenges Simonton’s claims that unique words and type-token ratio characterize better 35!
sonnets or that the most popular sonnets have a majority of concrete or primary process related 36!
concepts. Using semantic classes from WordNet (Fellbaum, 1988), Delmonte claims that the 37!
superior sonnets according to his own web-based search all contain a majority of abstract 38!
concepts, as opposed to primary process concepts1. 39!
40!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1!Unopportunely, Delmonte’s paper does not list the specific concepts.!

!
P
!
6!
The present study 1!
The aim of our study was to continue the QNA efforts of helping readers, critics and empirical 2!
researchers of Shakespeare’s sonnets in their private or public analyses of why and how these 3!
brilliant pieces of verbal art can induce significant cognitive, affective and aesthetic responses. 4!
More particurlarly, we aim at providing further QNA-based hypotheses and predictions for 5!
empirical studies in the emerging field of neurocognitive poetics concerning the readability, 6!
comprehensibility and affective-aesthetic potential of literary texts (Jacobs, 2015b; cf. also 7!
Burke, 2015; Nicklas & Jacobs, 2016). As argued recently by Jacobs (2015b) and Willems and 8!
Jacobs (2016), neurocognitive poetics studies using natural and ecologically valid materials – 9!
like the sonnets – can usefully inform and constrain models and theories in a number of 10!
domains, like emotion and language (e.g., Koelsch et al., 2015; Lindquist et al., 2015: 11!
Panksepp, 2008), emotion and literature (Miall, 1989; Oatley, 1994), affective word 12!
recognition and reading (Bestgen, 1994; Briesemeister et al., 2014, 2015; Jacobs et al., 2015, 13!
Jacobs et al., 2016b; Hofmann & Jacobs, 2014; Hsu et al., 2015; Kuhlmann et al., 2016; 14!
Lüdtke & Jacobs, 2015), empathy and mental simulation (e.g., Goldman, 2006; Oatley, 2016), 15!
immersion and transportation (Green & Brock, 2000; Hsu et al., 2014; Jacobs & Schrott, 2015; 16!
Ryan, 2001; Schrott & Jacobs, 2011), literary imagery (Kuzmičová, 2014), foregrounding 17!
(Miall & Kuiken, 1994; Van Peer, 1986), emotion and language development (Jacobs & 18!
Kinder, 2015; Miall & Dissanayake, 2003, Sylvester et al., 2016), self-construction and life 19!
narrativity (e.g., Habermas & de Silveira, 2008; Pleh, 2003), general aesthetics (e.g., Chatterjee 20!
& Vartanian, 2014; Jacobsen, 2006; Kintsch, 2012; Leder et al., 2004, 2015, Leder & Nadal, 21!
2014; Marin, 2015; Pelowski et al., 2016), cultural adaptation (Hutcheon, 2011; Nicklas & 22!
Jacobs, 2016), creativity (Beaty et al., 2016; De Beaugrande, 1979), cognitive poetics (e.g., 23!
Stockwell, 2009; Tsur, 1998, Turner & Pöppel, 1983), or literary reading (Burke, 2011, 2015; 24!
Jacobs, 2011, 2015a,b; Schrott & Jacobs, 2011) and its effects on well-being (e.g., O’Sullivan 25!
et al., 2015). 26!
27!
These QNA-based predictions will be specified in the following sections guided by specific 28!
questions addressed via use of tools like Coh-Metrix (CM in short; Graesser et al., 2004), 29!
TAACO (Crossley et al., 2015), SEANCE (Crossley et al., 2016), or RID (Martindale, 1975). 30!
The paper is structured into four parts presenting cognitive and affective-aesthetic QNAs of the 31!
sonnets before discussing how the data of these QNAs can be used in empirical investigations 32!
of neurocognitive poetics and computational modeling. 33!
34!
Part I. Cognitive QNAs 35!
36!
Question 1. How do sonnets differ in structural surface descriptors? 37!
A wealth of structural surface descriptors has been shown to influence language processing and 38!
reading at about all empirically investigated levels, from basic word recognition to judgments 39!
of essay quality or poetry reception (e.g., Crossley et al., 2014; Jacobs et al., 2015, 2016a). The 40!
data in Table 1 provide an overview of a set of relevant surface variables characterising 41!
sonnets 1 – 126 and 127 – 154 separately. The two wordclouds shown in Figure 1 visualize the 42!
most important content words in the young man (left) vs. dark lady (right) poems. According 43!

!
P
!
7!
to our own QNAs, altogether, the ten most freqently used words in the sonnets are: THE, OF, 1!
I, MY, TO, IN, AND, THY, THAT, THOU, WITH, and IS. Important key words among the 2!
100 most frequently occurring words are (in order of frequency): LOVE, BEAUTY, SWEET, 3!
EYE(S), HEART, TIME, and WORLD. There are 2480 words that occur only once in the 4!
sonnets, i.e. about 14%. Only 262 words are used 10 times or more. The vocabulary comprises 5!
roughly 4000 different words (39572). 6!
7!
Figure 1 here 8!
9!
At the poem level, the sonnet with the highest number of words!(130)!is the „betrayal“ sonnet 10!
42 (That thou hast her, it is not all my grief,..), while the “downhearted” sonnet 66 (Tired with 11!
all these, for restful death I cry,..) achieves its poetic effects with as little as 89 words arranged 12!
in the most repetitive fashion of all 154 sonnets. Sonnet 148 (O me, what eyes hath Love put in 13!
my head,..) with its many “O’s” and “I’s” – and which can be considered a rewrite of sonnet 14!
137 (Thou blind fool, Love, what dost thou to mine eyes,..) – features the highest number of 15!
content words (86/123). In contrast, sonnet 62 (Sin of self-love possesseth all mine eye..) not 16!
only features an odd dramatic scenario (Vendler, 1997, ch. 62), but also has the smallest 17!
number of content words (56/107). Sonnet 43 (When most I wink, then do mine eyes best see..) 18!
with its many alliterations has the highest average familiarity index according to CM (589) and 19!
plays with redundancy, as evidenced for example by a high “couplet glue”, i.e. seven words 20!
used in the poem’s body reappear in the couplet (Vendler, 1997, ch. 43). Sonnet 63 (Against 21!
my love shall be, as I am now..) with the “very deft couplet” (Vendler, 1997, ch. 63) features 22!
the highest CM value for content word imageability3 (469). 23!
24!
Table 1 summarizes a few exemplary surface descriptor statistics (QNA tools like CM, 25!
TAACO or SEANCE offer hundreds of them) separately for the „young man“ and „dark lady“ 26!
sonnets showing that these two categories descriptively can differ on some of these (which 27!
may or not be due to the inequal number of sonnets in each category). The 154 sonnets thus 28!
offer a rich variety of structural differences that can be exploited in experimental designs of 29!
neurocognitive poetics studies (see Part III). 30!
31!
32!
33!
34!
35!
36!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2!The exact numbers can vary slightly according to how spelling corrections were applied for better application of the QNA
tools.!
3!CM’s WRDIMGc index estimates how easy it is to construct a mental image of a word (it is highly correlated with word
concreteness). Examples of low imagery words are reason (285), dogma (327), and overtone (268) compared to words with
high imagery such as bracelet (606) and hammer (618).
!
!

!
P
!
8!
Table 1. Selected structural surface descriptors (mean values with standard deviations in 1!
brackets) 2!
Variable
Sonnets 1 –
126 („young
man“)
Sonnets 127 – 154
(„dark lady“)
TAACO number words
113 (5.8)
116 (6.4)
TAACO type-token ratio
0.71 (0.4)
0.68 (0.05)
TAACO content words
69 (5.6)
71 (6.08)
TAACO number function words
44 (6.7)
44 (6.9)
TAACO pronoun noun ratio
0.38 (0.19)
0.39 (0.22)
CM familiarity
565 (9.3)
566 (8.6)
CM imageability
418 (26.6)
417 (31.1)
3!
4!
Question 2. Which sonnets have the highest comprehensibility, i.e. are easiest to read and 5!
understand? 6!
7!
CM readability analysis 8!
Readability formulas have been developped as a quantitative tool for estimating text difficulty 9!
or comprehensibility. Among the >40 readability formulas, the most common are the Flesch 10!
Reading Ease Score and the Flesch Kincaid Grade Level (Klare, 1974-1975). However, both 11!
indices are based on the average sentence length (words/sentence) and word length 12!
(syllables/word) and – apart from other shortcomings – hence do not really appear sensitive for 13!
the present analyses due to the lack of sufficient variation in these two measures across the 14!
firmly structured sonnets (i.e., 14 lines, 10 syllables/line): The average number of words per 15!
line is 8.12 +- .43 and the average number of syllables/word is 1.25 +- .07. 16!
17!
A more recent, complex and potentially sensitive alternative is the CML2 readability index 18!
(McNamara et al., 2014). Although this is a second language readability score it is used here as 19!
a tentative simple composite measure providing a single number to assess text difficulty. 20!
21!
(1) CML2 readability index = -45.032 + (52.230 * content word overlap) + (61.306 * 22!
sentence syntax similarity) + (22.205 * CELEX mean log minimum frequency for 23!
content words) 24!
25!
The universe of 154 sonnets provides approximately normally distributed data on this measure 26!
of readability, as shown in Figure 2 (the hypothesis that the data are from a normal distribution 27!
could not be rejected as shown by the results of the W test). 28!
29!
Figure 2 here 30!
31!
Here we use this index to tentatively answer the question which sonnet theoretically is hardest 32!
or easiest to read (at least in a second language, which surely applies to millions of English as 33!

!
P
!
9!
L2 readers of sonnets). According to our CM analyses these are sonnets 1 (and 107) and 138, 1!
respectively. 2!
3!
4!
Sonnet 1 5!
From fairest creatures we desire increase, 6!
That thereby beauty’s rose might never die, 7!
But as the riper should by time decease, 8!
His tender heir might bear his memory: 9!
But thou, contracted to thine own bright eyes, 10!
Feed’st thy light’s flame with self-substantial fuel, 11!
Making a famine where abundance lies, 12!
Thyself thy foe, to thy sweet self too cruel. 13!
Thou that art now the world’s fresh ornament 14!
And only herald to the gaudy spring, 15!
Within thine own bud buriest thy content 16!
And, tender churl, makest waste in niggarding. 17!
Pity the world, or else this glutton be, 18!
To eat the world’s due, by the grave and thee. 19!
20!
Sonnet 138 21!
When my love swears that she is made of truth 22!
I do believe her, though I know she lies, 23!
That she might think me some untutor’d youth, 24!
Unlearned in the world’s false subtleties. 25!
Thus vainly thinking that she thinks me young, 26!
Although she knows my days are past the best, 27!
Simply I credit her false speaking tongue: 28!
On both sides thus is simple truth suppress’d. 29!
But wherefore says she not she is unjust? 30!
And wherefore say not I that I am old? 31!
O, love’s best habit is in seeming trust, 32!
And age in love loves not to have years told: 33!
Therefore I lie with her and she with me, 34!
And in our faults by lies we flatter’d be. 35!
36!
According to Vendler (1997, ch. 1) sonnet 1 may have been deliberately composed late, as a 37!
„preface“ or index to the others, standing out from the rest by two features: i) its sheer 38!
abundance of values, images, and concepts important in the sequence which are called into 39!
play and ii) the number of significant words brought to our attention. Self-evidently good 40!
values and salient images enumerated by Vendler include: beauty or sweetness, and rose or 41!
famine. As evidence for her view that Shakespeare’s mind works by contrastive taxonomy, 42!
Vendler cites the pairs of opposite concepts in sonnet 1: increase vs. decrease, ripening vs. 43!
dying, or immortality vs. memory. „Making an aesthetic investment in profusion“, sonnet 1 44!
also introduces catachresis, that is metaphors from incompatible categories applied to the same 45!
object (i.e., the young man as „a candle which refuses to bud forth“), which according to 46!
Vendler should vigorously call attention to itself (at least, if detected by the expert reader’s 47!
mind) and, by the cognitive dissonance it produces, should press readers into reflection. 48!
Another outstanding feature Vendler mentions is the greater than norm number of speech-acts 49!
in sonnet 1, especially in the vocative Q2 with its many direct addresses (e.g., thou, thyself, 50!

!
P
!
10!
thy). Sonnet 138 is said to depend wholly on reported discourse and to either represent a 1!
„depraved picture of cynical partners“ or a „sophisticated rendition of the way all lovers flatter 2!
each other“ (Vendler, 1997, ch. 138). It thus could be assimilated to an optical illusion like the 3!
Necker cube – a bistable figure –, a stylistic device often used in Petrarchan love sonnets 4!
(Schrott & Jacobs, 2011). 5!
6!
Linewise analysis 7!
Figure 3 compares the CML2 indices for both poems linewise. Although a linewise analysis 8!
seems a bit tricky given the small number of words involved, the data in Figure 3 fit nicely 9!
with the results of the poemwise analysis of Figure 2 in showing that – descriptively – sonnet 10!
138 has a higher CML2 index than sonnet 1 for all but two lines: four (His tender heir might 11!
bear his memory vs. Unlearned in the world’s false subtleties) and seven (Making a famine 12!
where abundance lies vs. Simply I credit her false speaking tongue). In contrast to most of the 13!
other lines, face validity suggests that these two indeed seem equally well readable. Overall, 14!
comparing the CML2 of the two sonnets linewise has enough face validity to think that this 15!
fine-grained analysis – although tentative – has encouraging potential for generating 16!
hypotheses concerning reader response measures at the level of poem lines (see Parrt III 17!
below). 18!
19!
CM easability analysis 20!
A both broader and deeper analysis using the eight text easability principal component z-scores 21!
of CM (cf. Graesser & McNamara, 2010; Graesser et al., 2011; McNamara et al., 2010) further 22!
illustrates the contrast between the presumably easiest to read sonnet 138 vs. the most difficult 23!
sonnet 1. 24!
25!
Figure 4 here 26!
27!
Descriptively, the differences between the two sonnets are: 1) sonnet 138 has a higher 28!
narrativity4 score (1.35) and thus is theoretically closer to everyday oral conversation than 29!
sonnet 1 (-2.15), which may have some face validity; 2) sonnet 138 is syntactically5 a bit less 30!
simple than sonnet 1 (0.029 vs. 0.413); 3) sonnet 138 possesses less concrete words6 (-0.59); 4) 31!
sonnet 138 features a higher referential cohesion score7 than sonnet 1 (1.96 vs. -.076), i.e. it 32!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
4 Narrative text tells a story, with characters, events, places, and things that are familiar to the reader. Narrative is closely
affiliated with everyday, oral conversation. This robust component is highly affiliated with word familiarity, world
knowledge, and oral language. Non-narrative texts on less familiar topics lie at the opposite end of the continuum.
5 This component reflects the degree to which the sentences in the text contain fewer words and use simpler, familiar
syntactic structures, which are less challenging to process.
6 Texts that contain content words that are concrete, meaningful, and evoke mental images are easier to process and
understand. Abstract words represent concepts that are difficult to represent visually. Texts that contain more abstract words
are more challenging to understand.
7 A text with high referential cohesion contains words and ideas that overlap across sentences and the entire text, forming
explicit threads that connect the text for the reader. Low cohesion text is typically more difficult to process because there are
fewer connections that tie the ideas together for the reader.!

!
P
!
11!
should typically be easier to process because there are fewer connections that tie the ideas 1!
evoked in the poem together for the reader; 5) sonnet’s 138 higher deep cohesion score8 (1.65 2!
vs. 0.48) suggests that it helps the reader to form a more coherent and deeper understanding of 3!
the causal events, processes, and actions in the poem; 6) sonnet’s 138 higher verbal cohesion 4!
score9 (2.01 vs. -0.11) suggests that it enhances situation model building as compared to sonnet 5!
1; 7) sonnet 138 also has a relatively greater connectivity score10 than sonnet 1 (-2.33 vs. -6!
5.56), and thus is likely to facilitate readers’ deeper understanding of the relations in the poem; 7!
finally, 8) the slightly higher temporality score11 of sonnet 138 (2.3 vs. 1.55) theoretically also 8!
would facilitate its comprehension as compared to sonnet 1. 9!
10!
In sum, descriptive differences in 6/8 indices are consistent in suggesting that sonnet 138 is 11!
easier to read and understand than sonnet 1. However, sonnet 1 may have a slightly simpler 12!
syntax and feature some words with a higher concreteness value than sonnet 138 which 13!
hypothetically makes it easier to understand than sonnet 138 on 2/8 dimensions. Lacking any 14!
inference statistics, these descriptive analyses allow no conclusions but serve an illustrative 15!
and heuristic, hypothesis-generating purpose demonstrating how CM can be used to compare 16!
the readability of two or more poems at a more sophisticated level than the traditional 17!
readability scores. 18!
Even though CM was designed to primarily analyze longer text book materials rather than 19!
short poems, it already was succesfully applied to the language of Shakespeare (Graesser et al., 20!
2011) and – given the paucity of specialised QNA alternatives for the structural description of 21!
poetry (Jacobs, 2015b) – the data in Figures 2 - 5 can be submitted to empirical testing in 22!
rating, eye movement or neuroimaging studies (see Part III). 23!
24!
Figure 5 here 25!
26!
Figure 5 shows the distributions of the five most commonly used standardized easibility scores 27!
for all 154 sonnets (i.e., narrativity, syntactic simplicity, word concreteness, and referential and 28!
deep cohesion; Graesser et al., 2011). The approximately normally distributed data (two 29!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
8!This dimension reflects the degree to which the text contains causal and intentional connectives when there are causal and
logical relationships within the text. These connectives help the reader to form a more coherent and deeper understanding of
the causal events, processes, and actions in the text. !
9!This component reflects the degree to which there are overlapping verbs in the text. When there are repeated verbs, the text
likely includes a more coherent event structure that will facilitate and enhance situation model understanding. This
component score is likely to be more relevant for texts intended for younger readers and for narrative texts (McNamara,
Graesser, &Louwerse, 2012).
10 This component reflects the degree to which the text contains explicit adversative, additive, and comparative connectives
to express relations in the text. Thus, it reflects the number of logical relations in the text that are explicitly conveyed. This
score is likely to be related to the reader’s deeper understanding of the various relations in the text.
11 Texts that contain more cues about temporality and that have more consistent temporality (i.e., tense, aspect) are easier to
process and understand. In addition, temporal cohesion contributes to the reader’s situation model level understanding of the
events in the text.
!

!
P
!
12!
distributions are normal, three fail the W test) facilitate the use of these scores in statistical 1!
effect analyses such as linear mixed or regression models and demonstrate the potential of this 2!
sonnet corpus for empirical studies in neurocognitive poetics (Jacobs, 2015b). 3!
4!
Table 2 gives an overview of the three sonnets easiest vs. hardest to process, respectively, for 5!
each of the five dimensions. According to Table 2 then, sonnet 138 mainly is easier to read 6!
because it has a considerably higher narrativity score than sonnet 1. 7!
8!
Table 2. Three easiest and hardest to read sonnets according to five CM easability scores 9!
10!
Narrativity
Syntactic
Simplicity
Word
Concreteness
Referential
Cohesion
Deep
Cohesion
easiest
42, 138, 149
145, 125, 10
63, 153, 20
47, 134, 136
51, 52, 22
hardest
1, 77, 95
80, 67, 73
105, 90, 115
125, 65, 85
31, 144, 35
11!
12!
Question 3. Which sonnets have the highest surprisal value? 13!
Surprisal, the most common quantification of words’ information content (Frank, 2013) is 14!
known to be a co-determinant of reading speed and eye movement parameters correlating 15!
positively with reading time (e.g., Frank 2013; Smith & Levy 2013). Moreover, the amplitude 16!
of the N400 event-related potential (ERP) component was found to correlate with word 17!
surprisal values (Frank et al., 2015). When words come unexpected to the reader – which is 18!
part of the attraction of poetry – their surprisal value is higher than when they can be 19!
anticipated by context and skilled knowledge of lexis and grammar. For each of the 154 20!
sonnets we computed two indices of their surprisal value12. 21!
22!
Figure 6a shows the distributions of these two different, normally distributed mean surprisal 23!
values for the sonnet corpus: The upper panel shows surprisal values based on the Subtlex 24!
database (Brysbaert & New, 2009), the lower panel shows surprisal values based on a 25!
Shakespeare corpus (http://shakespeare.mit.edu/). The context effect of Shakespeare’s verbal art 26!
can easily be seen in the difference between the two means (3.7 vs. 3.1): not surprisingly, 27!
Shakespeare’s words are notably more surprising when matched against a modern database 28!
than when taxed within their own verbal neighborhood. 29!
30!
Figure 6a-c here 31!
32!
It was interesting to see whether our Shakespeare corpus surprisal measure correlates with 33!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
12!The surprisal values were estimated by means of a corpus-based trigram model – the values for each word being
computed by the SRILM package (see Willems et al. 2015): 1) a corpus consisting of the works of Shakespeare
(http://shakespeare.mit.edu/) excluding his sonnets (encompassing 1433958 sentences), and 2) a contemporary corpus of
spoken sentences (SUBTLEX; encompassing 6043188 sentences). The trigram model already was successfully applied to
experimental data (EEG: Frank et al. 2015; reading time: Smith & Levy 2013).
!

!
P
!
13!
CM’s readability (CML2) and easability scores: All correlations were significant but one (deep 1!
cohesion). Thus, CML2 readability, narrativity and referential cohesion all significantly 2!
decreased with increasing surprisal: F(1,152) = 63.42, p<.0001, R2 = .29; F(1,152) = 42.87, 3!
p<.0001, R2 = .22; F(1,152) = 49.77, p<.0001, R2 = .25, respectively. Interestingly, syntactic 4!
simplicity and word concreteness increased with increasing surprisal: F(1,152) = 11.52, 5!
p<.0009, R2 = .07; F(1,152) = 15.31, p<.0001, R2 = .091, respectively. The higher the surprisal 6!
value of a sonnet, the less easy to read, the less „narrative“ and the less (referentially) coherent 7!
it seems. However, at least for this corpus, sonnets with a higher surprisal also tend to feature 8!
a simpler syntax and more concrete words (or words with high concreteness values). Even if 9!
the effects were small, this finding seems interesting material for further research: Perhaps, in 10!
some cases the poet chose to trade-off a critical amount of poetic surprisal for fewer or less 11!
abstract words and simpler grammar in order not to make the poem too hard to comprehend. 12!
Examples for poems that are high on both syntactic simplicity and surprisal are sonnets 60, 66 13!
and 125. Examples for poems that are high on both word concreteness and surprisal are 14!
sonnets 66, 153 and 154. 15!
16!
The surprisal value of 4.25 (Subtlex) for the theoretically hardest to read sonnet 1 is 17!
significantly higher than that of sonnet 138 (3.4; F(1,26) =8.9, p<.006, R2 = .26) thus 18!
confirming the results summarized in Figure 4. Overall, the line with highest surprisal is line 19!
12 in sonnet 1 (5.85: And, tender churl, makest waste in niggarding), the one with the lowest is 20!
line two from sonnet 138 (2.43: I do believe her, though I know she lies). Figure 6b and c zoom 21!
into the individual lines and words of sonnet 1 to reveal their line- and wordwise surprisal 22!
values for even more fine-grained hypotheses, e.g., concerning eye movement or ERP 23!
parameters (see Part III). 24!
25!
Before we take an „emotional turn“ in our analyses, a short summary of the cognitive QNAs of 26!
Part I seems in order. Thanks to their approximately normally distributed features describing 27!
the readability, comprehensibility and surprisal, the 154 sonnets offer a rich playground for 28!
generating and testing hypotheses about reader responses in neurocognitive poetics studies 29!
(Jacobs, 2015b) at different levels of inquiry: the metalevel of poem category, i.e. „young 30!
man“ vs. „dark lady“ poems, the poem level, e.g. poems with low vs. high comprehensibility, 31!
and even the line- or wordwise levels, e.g., lines/words with low vs. high surprisal. Together 32!
with the abundance of qualitative (or quasi-quantitative) content analyses and hermeneutic 33!
interpretations of Shakespeare’s sonnets by critics and scholars (e.g., De Beaugrande, 1979; 34!
Jakobson & Jones, 1970; Vendler, 1997) the present QNA results should motivate a series of 35!
empirical studies using combined qualitative-quantitative, multimethod, multilevel designs 36!
(Jacobs et al., 2016a) providing new insights into the complexities of „brain and poetry“ 37!
(Schrott & Jacobs, 2011) by allowing to i) better manage (i.e., manipulate, match or control) a 38!
great number of potentially relevant stimulus variables for more natural and ecologically valid 39!
experiments (Willems, 2015; Willems & Jacobs, 2016); ii) disentangle effects of surface and 40!
form vs. deep structure and content features, or iii) cognitive vs. affective variables (e.g., 41!
Jacobs et al., 2016a,b; Menninghaus et al., 2014, 2015). 42!
43!

!
P
!
14!
Complementing the previous one, Part II looks at quantifiable affective-aesthetic variables of 1!
sonnets that theoretically and empirically are at least as relevant for reader responses to poetry 2!
as the cognitive ones (e.g., Jacobs, 2015a,b; Lüdtke et al., 2014; Schrott & Jacobs, 2011). 3!
4!
Part II: Affective-aesthetic QNAs 5!
6!
Question 4. Which sonnets have the highest emotion potential? 7!
The recently growing interest in emotional word and text processing (see Citron, 2012; Jacobs 8!
et al., 2015, 2016a, for review) has been made possible by databases allowing to quantitatively 9!
estimate affective word features, such as the Berlin Affective Word List (BAWL, 10!
Briesemeister et al., 2011; Võ et al., 2006, 2009; Jacobs et al., 2015), the Affective Norms for 11!
English Words (ANEW; Bradley and Lang, 1999), the Affective Norms for German Sentiment 12!
Terms (ANGST; Schmidtke et al., 2014), or by computational algorithms (Westbury et al., 13!
2014). Several recent studies demonstrate how such tools can be used to predict and interpret 14!
reader responses to poetry (e.g., Aryani et al., 2016; Jacobs et al., 2016b; Ullrich et al., 2016) 15!
and prose (Altmann et al., 2012, 2014; Hsu et al., 2015), which are theoretically predicted by 16!
the Panksepp-Jakobson hypothesis of the Neurocognitive Poetics Model (NCPM) of literary 17!
reading (Jacobs, 2011, 2015a,b). 18!
19!
A sophisticated comprehensive tool for English texts that complements cognitive QNA tools 20!
like CM is SEANCE (Crossley et al., 2016). Here we computed the 20 SEANCE component 21!
scores for each sonnet and summed them to determine the sonnets with the theoretically 22!
highest emotion potential. These 20 scores are: Negative adjectives, Social order, Action, 23!
Positive adjectives, Joy, Affect for friends and family, Fear and disgust, Politeness, Polarity 24!
nouns, Polarity verbs, Virtue adverbs, Positive nouns, Respect, Trust verbs, Failure, Well 25!
being, Economy, Certainty, Positive verbs, and Objects (for details, see Crossley et al., 2016). 26!
 27!
Figure 7 here 28!
29!
Figure 7 shows the distribution of this composite feature. The three sonnets with the highest 30!
and lowest emotion potentials according to our SEANCE composite score, respectively, are: 31!
140, 151, 144 and 3, 90, 53. 32!
33!
Sonnet 140 34!
Be wise as thou art cruel; do not press 35!
My tongue-tied patience with too much disdain; 36!
Lest sorrow lend me words and words express 37!
The manner of my pity-wanting pain. 38!
If I might teach thee wit, better it were, 39!
Though not to love, yet, love, to tell me so; 40!
As testy sick men, when their deaths be near, 41!
No news but health from their physicians know; 42!
For if I should despair, I should grow mad, 43!
And in my madness might speak ill of thee: 44!
Now this ill-wresting world is grown so bad, 45!
Mad slanderers by mad ears believed be, 46!

!
P
!
15!
That I may not be so, nor thou belied, 1!
Bear thine eyes straight, though thy proud heart go wide. 2!
3!
Sonnet 3 4!
Look in thy glass, and tell the face thou viewest 5!
Now is the time that face should form another; 6!
Whose fresh repair if now thou not renewest, 7!
Thou dost beguile the world, unbless some mother. 8!
For where is she so fair whose unear’d womb 9!
Disdains the tillage of thy husbandry? 10!
Or who is he so fond will be the tomb 11!
Of his self-love, to stop posterity? 12!
Thou art thy mother’s glass, and she in thee 13!
Calls back the lovely April of her prime: 14!
So thou through windows of thine age shall see 15!
Despite of wrinkles this thy golden time. 16!
But if thou live, remember’d not to be, 17!
Die single, and thine image dies with thee. 18!
19!
Sonnet 140 indeed features many emotion-laden words like CRUEL, PITY or DESPAIR and 20!
in Q3 a „pathological picture of the world in which both speaker and audience are conceded to 21!
be mad“ (Vendler, 1997, ch. 140) is drawn. The three top emotion potential sonnets all belong 22!
to the „dark lady“ category. 23!
24!
As a first cross-validation check, we computed the correlation across all 154 sonnets between 25!
the simple emotion potential measure proposed in Jacobs (2015b), i.e. the product between the 26!
absolute mean values for valence and arousal of each word in a text (as computed from the 27!
database of Warriner et al., 2013) and the SEANCE composite score. The correlation was 28!
small but significant: F(1,153) = 14.75, p < .0002, R2 = .09. In a second cross-validation check, 29!
we computed the correlation of the SEANCE score with Martindale’s RID measure 30!
EMOTION (Martindale, 1975; cf. Simonton, 1989). The correlation was smaller than for the 31!
previous measure but still significant: F(1,153) = 4.9, p < .028, R2 = .03. 32!
33!
Figure 8 here 34!
35!
Given Simonton’s claim that sonnets with superior aesthetic success feature more primary 36!
process imagery, in Figure 8 we show the distributions of the three RID indices (Primary and 37!
Secondary process, Emotion) for all sonnets. Table 3 gives an overview of the three sonnets 38!
with the highest and lowest values, respectively, for each of the three RID indices and their 39!
sum total. Interestingly, the theoretically easiest-to-read sonnet 138 also features a high 40!
secondary process score. 41!
42!
Table 3. Sonnets with lowest/highest RID values. 43!
44!
Primary
process
Secondary
process
Emotion
Sum RID
highest
153, 154, 73
57, 49, 138
40, 25, 30
153, 2, 43

!
P
!
16!
lowest
36, 149, 4
99, 7, 94
12, 81, 43
74, 134, 94
1!
Question 5. Which sonnets have the highest positive vs. negative mood potential? 2!
Three indices can be used in a straightforward way to predict the potential of a sonnet to 3!
induce either a positive or negative mood (Aryani et al., 2016): Mean word valence, valence 4!
span, and word valence sum. Since these measures are only moderately correlated (all r < .44), 5!
they can lead to divergent predictions. In a second analysis we computed two component 6!
scores of SEANCE to estimate the potential of a sonnet to induce either a negative or positive 7!
mood: A „negative mood score“ was obtained by summing up the scores for components one 8!
and seven (negative adjectives and fear and disgust), and a „positive mood score“ by summing 9!
up the scores for components four, five, 12 and 19 (positive adjectives, positive nouns, positive 10!
verbs and joy). Figure 9 shows the distribution of these five indices. 11!
12!
Figure 9 here 13!
14!
Table 4 gives an overview of the three sonnets with the highest and lowest values, respectively, 15!
for each of the five mood indices. The results provide a heterogeneous picture calling for 16!
empirical investigation, since each index makes different predictions as to which three sonnets 17!
induce a positive vs. negative mood. As has been shown empirically by Lüdtke et al. (2014), 18!
and Jacobs et al. (2016a), also other factors than these five indices play a role in mood 19!
induction through poetry, but the results of Aryani et al. (2016) suggest that whether a poem is 20!
rated as „sad“ or „friendly“ is clearly affected by word valence. To what exent such ratings 21!
reflect the perception (in the poem) and/or genuine feeling of a sad vs. joyful mood13 is an 22!
open issue for future research that can use the QNA data produced in this paper. We checked 23!
whether the „young man“ sonnets differed significantly from the „dark lady“ sonnets in their 24!
positive vs. negative mood potential. There was no difference for the former, but the latter 25!
indeed was significantly greater for the „dark lady“ sonnets: 1.1 > 0.33, F(1,152) = 11.15, p < 26!
.001, R2 = .07. Thus whether perceived and/or felt mood, in an empirical study the „dark lady“ 27!
sonnets should produce higher response measures of negative mood than the „young man“ 28!
sonnets. 29!
30!
Table 4. Sonnets with lowest/highest scores for various estimates of mood potential 31!
32!
Valence mean
Valence Span
Valence Sum
SEANCE
negative
SEANCE
positive
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
13!In music psychology there is a debate between a cognitivist and emotivist position accounting for the „sad music
paradoxon“, i.e. the phenomenon that people like sad music (Taruffi, 2016). The first position states that people do not
experience genuine sadness at all, but merely recognize the sadness depicted by the music (e.g., Kivy, 1990), while the
second claims that sad music induces an emotion similar to „real“ sadness, although it is not clear to what extent they overlap
(Levinson, 1997). Regarding sad (vs. joyful) poetry one can argue in a similar vein. The observation that similarly to music
(Krumhansl, 1997) poetry also shows measurable peripheral-physiological effects (Jacobs et al., 2016a) can be taken to
suggest that it evokes „real“ feelings at least to some extent.

!
P
!
17!
highest
153, 154, 73
57, 49, 138
40, 25, 30
129, 140, 120
128, 26, 136
lowest
36, 149, 4
99, 7, 94
12, 81, 43
74, 134, 94
44, 90, 133
1!
Question 6. Which sonnets have the greatest thematic richness? 2!
As outlined above, thematic richness is one of Simonton’s (1989) key sonnet features for 3!
superior aesthetic success. Moreover, recent empirical research on poetry reception supports 4!
the notion that the motif or topic of a poem is important for reader responses (e.g., Lüdtke et 5!
al., 2014; Jacobs et al., 2016a). As stated by the latter authors (p. 97): „Knowing, inferring, or 6!
guessing the overall motif of a poem might therefore be especially important – as a kind of 7!
orienting metameaning active in working memory – for interpreting hidden multiple meanings 8!
and unexpected meaning twists typical for abstract or obscure poetic texts (Shimron, 1980; 9!
Yaron, 2002, 2008), or for the couplet at the end of Shakespearean sonnets. Here we used 10!
Simonton’s (1989) 24 different topics to compute a thematic richness index (TRI = sum of all 11!
of topics per sonnet). Figure 10 summarizes the distribution of this TRI across the 154 sonnets. 12!
13!
Figure 10 here 14!
15!
According to this analysis, the majority of sonnets (68) has two motifs or topics, combining, 16!
e.g., love and poetry. Nineteen sonnets highlight none of the topics listed by Simonton, 17!
whereas six sonnets (14, 15, 25, 65, 76, and 82) play with as many as six different motifs. 18!
Thus, in Simonton’s terms (1989, Table 1, p. 704) sonnet 14 features the topics i) beauty 1b, 19!
i.e. beauty and truth, the beautiful as an object of contemplation; ii) family 6a, i.e. the desire 20!
for offspring; iii) immortality 6a, i.e. immortality through offspring, the perpetuation of the 21!
species; iv) love 1e, i.e. the intensity and power of love, its increase or decrease, its 22!
constructive or destructive force; v) love 2b, i.e. friendly, tender, or altruistic love; fratermal 23!
love, and vi) time 7, i.e. the temporal course of the passions emotional attitudes toward time 24!
and mutability. 25!
26!
Which of the other variables of the present QNA significantly correlate with the TRI? Among 27!
the more than 40 variables, for which this was the case (p<.05), for 17 at least 5% of the 28!
variance was accounted for by TRI. Among those, the most relevant for the present purposes 29!
were: Mean and summed word valence, number of posititive words, and positive noun 30!
component (all SEANCE) which all correlated positively with TRI, and number of negative 31!
words, negative adjectives, and negative mood score which correlated negatively (all R2 > .05). 32!
These results allow to state the following theoretical claim to be empirically tested in future 33!
studies: The higher a sonnet’s TRI, the higher the likelihood that readers will like it and rate it 34!
as perceiving/inducing a positive mood. Thus, perhaps, the aesthetic success of sonnets is 35!
mediated by their mood perception/induction potential? 36!
37!
Question 7. Which sonnets have the highest symbolic imagery index (SII)? 38!
The sonnets contain a wealth of recurrent images, archetypes, archetypal patterns and personal 39!
myths „through which the imaginary of the writer and that of the reader bind, generating 40!
meaning“ (Meireles, 2005, p. 5). Music (sound) and painting (imagery) have been 41!

!
P
!
18!
characterized as perhaps the most distinctive features of poetry (Schrott & Jacobs, 2011), but 1!
little is know about which kind of imagery prevails in poetry, what its neural correlates are, 2!
and how it can reliably and validly be measured (Jacobs, 2016a). Determining the type and 3!
occurrence of imagery in poetry thus may constitute a first step towards tackling these issues. 4!
Based on the work of Meireles, here we computed a typological „symbolic imagery index/SII“ 5!
by coding each sonnet for the occurrence of the following eight types of recurrent 6!
symbolic/archetypical images: TIME (i.e., words expressing symbols for time like CLOCK, 7!
MOMENT or HOURS), SOLAR (i.e., images conveying solar symbols as seen in words like 8!
DAY, SUN, or STARS), WATER (e.g., words like LIQUID, TEARS), NOCTURNAL (e.g., 9!
DARKNESS, MOON), SEASON (e.g., SUMMER, APRIL), NATURE (i.e., only the word 10!
NATURE itself), IMMORTALITY (e.g., ETERNAL, SOUL, BODY), and COLOR (e.g., 11!
BLACK, SCARLET). Much as for the TRI analyses, we use this list in a heuristic fashion – 12!
without any claims regarding its completeness, validity or poetic effectiveness – for purposes 13!
of comparison and cross-validation with the other tools used here (for a discussion of the 14!
symbolic imagery in the sonnets see Vendler, 1997, or Meireles, 2005). 15!
16!
Figure 11 here 17!
18!
Figure 11 shows the distribution of the SII, which is simply the sum of the eight indices listed 19!
above. Thus, 15 sonnets have an SII of 0 being void of the symbols in our list. At the other 20!
extreme, sonnet 65 features 75% (6/8) of the eight symbols (TIME, SOLAR, WATER, 21!
SEASON, IMMORTALITY, COLOR) and 23 sonnets have more than two. The great majority 22!
(105), however, focuses on one or two symbols or archetypes according to Meireles’ (2005) 23!
typology. 24!
25!
Sonnet 65 26!
Since brass, nor stone, nor earth, nor boundless sea, 27!
But sad mortality o’er-sways their power,28!
How with this rage shall beauty hold a plea,29!
Whose action is no stronger than a flower? 30!
O, how shall summer’s honey breath hold out 31!
Against the wreckful siege of battering days, 32!
When rocks impregnable are not so stout,33!
Nor gates of steel so strong, but Time decays? 34!
O fearful meditation! where, alack, 35!
Shall Time’s best jewel from Time’s chest lie hid? 36!
Or what strong hand can hold his swift foot back? 37!
Or who his spoil of beauty can forbid? 38!
O, none, unless this miracle have might,39!
That in black ink my love may still shine bright. 40!
41!
Which variables of the QNA tools used here correlate with the SII? This was the case for seven 42!
variables. The strongest significant (positive) correlation was found for RID’s primary process 43!
score: F(1,152) = 22.93, p < .0001, R2 = .13, thus cross-validating Meireles’ typology. The 44!
other significantly correlated variables that accounted for at least 5% of variance in SII were 45!
(in order of R2): CM’s word concreteness score (positive, R2 = .12), TAACO content types 46!
(positive, R2 = .085), summed word valence (positive, R2 = .083), TAACO type-token ratio 47!

!
P
!
19!
and repeated content lemmas and pronouns (both positive, R2 = .06), and TRI (positive, R2 = 1!
.06). 2!
3!
To summarize, sonnets that are rich in symbolic imagery like 55, 12, 14, 18, 27, 56, 61, 63, 65, 4!
68, or 98 should feature more concrete and unique content words, as well as words associated 5!
with oral or sexual needs like BREAD or LUST, with sensations (e.g., SHARP), defensive 6!
symbols (e.g., PILGRIM), regressive knowledge (e.g., SECRET), or icarian imagery (e.g., 7!
VALLEY) according to RID. They also should have an overall more positive valence, 8!
supported by words expressing joy, anticipation or surprise like HAPPY or MAGICAL, a 9!
greater lexical diversity and more repetitions of content lemmas and pronouns. In line with 10!
this, their TRI should be higher. On the other hand, they should feature less adjectives 11!
expressing fear or disgust, less emotion words (e.g., AFRAID, HARSH ), and less words 12!
affiliated with everyday, oral conversation (due to small but significant negative correlations 13!
with SEANCE’s fear and disgust component, RID’s emotion score, and CM’s narrativity 14!
score). 15!
16!
Question 8. Which sonnets have the highest semantic association potential (SAP)? 17!
The EAT is an index of the number of associations from the Edinburgh Associative Thesaurus 18!
(EAT, Kiss et al., 1973). The number of semantic associates of a word is a factor that has 19!
various effects on both behavioral and neuronal measures in word and text processing, as well 20!
as in memory tasks (see Hofmann & Jacobs, 2014, for review). For example, recent 21!
computational and neurocognitive studies suggest that the affective evaluation of words and 22!
texts is co-determined by their semantic associations (Kuhlmann et al., 2016; Hofmann & 23!
Jacobs, 2014; Recchia & Louwerse, 2015; Westbury et al., 2014). Here we were interested in 24!
the variance across the 154 sonnets concerning their “SAP”. 25!
26!
Figure 12 here 27!
28!
Figure 12 shows the distribution of the SAP. It is based on the EAT summed – across the 14 29!
lines of a sonnet – relative number – i.e., divided by the total number of words/sonnet – of 30!
unique words associated with a given target word. The sonnets had a mean SAP of 39; the 31!
three sonnets with the highest SAP were: 124, 127, and 129 (all > 42); with an SAP of 36, 32!
sonnet 123 was at the low end of the distribution. As expected from the above cited studies, 33!
SAP weakly but significantly correlated with the valence (sum) of the sonnets: F(1,152) = 4.7, 34!
p < .03, R2 = .03, allowing the tentative hypothesis that sonnets rich in SAP will produce 35!
higher liking ratings. Interestingly, SAP also marginally significantly correlated with the 36!
computational estimate of the sonnets’ theoretical emotion potential, i.e. the product of word 37!
valence and arousal, as computed by the algorithm of Westbury et al. (2014): F(1,152) = 3.7, p 38!
< .055, R2 = .02. If replicated with other poetry corpora, this would be confirmatory evidence 39!
for Westbury et al.’s (2014) claim that valence consists of four dimensions (potency, 40!
happiness, approachability, and association with anger) that can be computed by help of the 41!
open-source co-occurrence model HiDEx (Shaoul & Westbury, 2010). 42!
43!

!
P
!
20!
Overall, the affective-aesthetic indices discussed in Questions 4-8 appear sensitive enough to 1!
be used for generating and testing hypotheses concerning emotional reader responses in 2!
neurocognitive poetics studies on sonnet reception. Question 9 now deals with the issue 3!
whether together with the cognitive indices they also are sentitive to more or less subtle 4!
changes in form and content across different sonnet parts. 5!
6!
Question 9. Can QNA capture sonnet dynamics? 7!
In the Introduction, sonnets were described as systems in motion with a thematic-semantic 8!
narrowing down movement from Q1 to Q4 and C (Vendler, 1997). Moreover, according to 9!
Vendler, the sequence of images, for example, should have a notable effect on its 10!
interpretation. Simonton (1990, p.261) also found effects of sonnet part by showing that „as we 11!
ascend from the mediocre sonnets to those that have likely earned a permanent position in 12!
literary history, the probability of encountering a unique word in either the third quatrain or the 13!
final couplet decreases”. 14!
15!
Here we wanted to see whether the present QNA tools also can detect traces of such dynamics, 16!
e.g. can tools like CM, RID, or SEANCE capture aspects that reveal a thematic diminution 17!
from Q1 to C, or changes in comprehensibility or emotion potential from the octave to the 18!
sestet? 19!
20!
Regarding composite indices of ease of comprehension, we found significant differences 21!
between the four parts of a sonnet (Q1,2,3 and C) for the CML2 readability index and several 22!
of the five CM easability indices. Using sonnet part as the independent variable in several one-23!
way ANOVAs the following picture emerged: The final couplets had significantly higher 24!
CML2 readability scores than the quatrains which did not differ from each other (means: 6.7 25!
vs. 1.4, 1.0, 2.4, respectively; see Appendix Table A1 for details). The couplets also had 26!
significantly higher narrativity, syntactic simplicity, referential and deep cohesion but lower 27!
surprisal scores than the body parts of the sonnet (see Appendix). The only other significant 28!
effect of sonnet part on CM easability scores was that, on average, Q2s had a higher syntactic 29!
simplicity than Q1s. 30!
31!
Regarding the octave-sestet contrast, sestets were systematically easier to read than octaves 32!
(means: 3.8 vs. 1.2, respectively) and had significantly higher narrativity and deep cohesion 33!
scores but lower surprisal values (see Appendix Table A1). 34!
35!
Complementing this cognitive QNA by an affective one using RID, the simple emotion 36!
potential measure mentioned above (Jacobs, 2015b), and the 20 SEANCE components, the 37!
only significant effects we observed was that, on average, sestets had an overall higher emotion 38!
potential, affect for friends and family and positive verbs score than the octaves (see 39!
Appendix). 40!
41!
A final analysis looked at the assumption mentioned in the Introduction that the number of new 42!
text world referents decreases towards the end of a text. As evidenced by Figure 13 this was 43!

!
P
!
21!
definitely the case: F(1,2152) = 516.33, p < .0001, R2 = .19. While Q1s feature 87% new 1!
words on average, this value drops to 76% for Q2s and 68% for Q3s reaching a plateau for Cs 2!
with 59% (all ps<.0001). This QNA discovery sheds new light on Vendler’s (1997) „funnel-3!
shape“ movement assumption mentioned in the Introduction, supports Steen’s (2004) view of a 4!
narrowing down of the reader’s text world, and thus has interesting implications for future 5!
empirical studies, such as the hypothesis derived from Steen (2004, p. 1304) that style figures 6!
(e.g., metaphors) should be easier to recognize towards the end of poems. 7!
8!
Figure 13 here 9!
10!
In sum, the cognitive and affective indices used here are sensitive – albeit to different degrees 11!
– to sequential changes in form and content across sonnet parts. The present results thus 12!
motivate more work testing Vendler’s (1997), Steen’s (2004) or further assumptions that will 13!
now be discussed in Part III. 14!
15!
Part III. Hypotheses for Neurocognitive Poetics Studies 16!
17!
Question 10. What can the present QNAs be used for? 18!
The most difficult question comes last, of course. What good is all the effort spent in applying 19!
QNA tools to sonnets? Despite his extensive analyses including phonetic, poetic and syntactic-20!
semantic relational levels in poems, Delmonte (2016, p. 93) concludes his work on a 21!
challenging note: “From the data reported above, it is hard to understand what criteria would 22!
be best choice for the individuation of most popular sonnets. It seems clear, however, that 23!
neither themes nor readability indices are sufficient by themselves to identify them all. Nor do 24!
evaluations based on semantic/pragmatic criteria derived from existing lexica help in the final 25!
classification. We surmise that an evaluation of how much popular a poem can be should also 26!
take into account cultural issues which have not been tackled by this study ….In particular, the 27!
contribution of rhetoric devices, like similes and metaphors, is hard to compute consistently for 28!
all sonnets: Shakespeare’s best virtue was his subtlety in generating a great quantity of 29!
secondary meanings from simple juxtaposition of terms and images. So eventually, what 30!
SPARSAR can do is help practitioners in that direction without giving a final complete result, 31!
but leave the user to combine different schemes, graphs, tables and other data together in the 32!
puzzle constituted by poetry that aims at excellence and lasts forever, like the one we have 33!
been commenting in this article”. 34!
35!
In a somewhat more optimistic vein, Simonton (1989, p. 703) advanced that „Our 36!
understanding of artistic creativity would be enlarged if we knew which of these four 37!
alternative measures optimally predicted aesthetic success“. Similarly optimistic, Graesser et 38!
al. (2011, p. 31) conclude their QNA analysis of three dramas by Shakespeare using the CM 39!
tool (Graesser et al., 2004) with: “In closing, we believe that there is so much to be learned 40!
from computer analyses of literature. Computers may never understand and fully appreciate 41!
Shakespeare. But humans don’t either. Meanwhile we can learn from computer analyses just as 42!
we learn from the insights of literary scholars. A computational science of literature is a 43!

!
P
!
22!
worthy player in the interdisciplinary arena”. 1!
2!
We leave it to interested readers to form their own opinion and meanwhile propose a few 3!
potentially useful applications in the following sections. For us, a first straightforward use of 4!
the present QNA results is in empirical studies on neurocognitive poetics that require 5!
quantitative variables for their stimulus selection and/or statistical data analyses testing specific 6!
predictions. As argued in Jacobs (2015b), the dynamically developing but still very recent field 7!
of neurocognitive poetics needs extended and refined text-analytical tools. These are necessary 8!
for both model development and for inspiring experimental designs that use more natural and 9!
ecologically valid stimuli and tasks, as well as a combination of direct/indirect and 10!
online/offline measures aiming at a higher overall validity. All these are part and parcel of the 11!
neurocognitive poetics perspective (Bohrn et al., 2012a,b, 2013; Chen et al., 2016; Dixon & 12!
Bortolussi, 2015; Jacobs, 2011, 2015a,b,c, 2016a; Lehne et al., 2015; Liu et al., 2015; 13!
O’Sullivan et al., 2015; Vaughan-Evans et al., 2016; Wallentin et al., 2011; Willems et al., 14!
2015; Willems & Jacobs, 2016; Zeman et al., 2013). 15!
16!
Next, we discuss some example predictions straightforwardly emerging from the above QNAs. 17!
18!
Predictions based on present results 19!
The present QNA data allow to formulate nested hypotheses at three levels of detail: poem 20!
category (young man vs. dark lady), across poem contrasts (poem X vs. poem Y), and within-21!
poem contrasts (Q1-3 vs. C, octave vs. sestet, line- or wordwise). We will give examples for all 22!
of them in the hope to encourage further empirical research in line with the goals set in a recent 23!
review paper on the scientific study of literary experience and response (Jacobs, 2015c; 24!
2016a). 25!
26!
Poem category 27!
At this supra-poem level, we found several descriptive differences between the young man and 28!
dark lady poems that lead to testable hypotheses. An example of interest in the light of 29!
Simonton’s (1989, 1990) analyses concerns the TRI index which, on average, was significantly 30!
higher for the young man sonnets than the dark lady ones, suggesting a global thematic-31!
semantic narrowing: means = 2.9+-0.12 vs. 1.3+-0.25, F(1,152) = 32.6, p < .0001, R2 = .1814. 32!
Thus, in line with our considerations regarding Question 6 above, we can hypothesize that on 33!
average – and all other things being equal – readers will be more inclined to like one or more 34!
randomly selected young man sonnet(s) more than dark lady one(s) and rate it as 35!
perceiving/inducing a more positive mood15. Other indirect on- or offline measures (e.g., eye 36!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
14!We first established equal variances for the two different sample sizes (O’Brien F(1,152) = 1.75, p = .18;
Brown-Forsythe F(1,152) = 2.44, p = .12), and also checked by bootstrapping (N = 100) that the confidence
intervals of the mean difference (-1.6+-0.95, CIu = -1.05, CIl = -2.2) did not differ from those of the bootstrapping
(CIu = -1.04, CIl = -2.16). Finally, we computed a nonparametric test which confirmed the results of the one-way
ANOVA: Wilcoxon/Kruskal-Wallis: S = 1106,5, Z = -5.25, p<.0001.
15!Before developping and testing any hypotheses we recommend to augment such QNA-based statistical
analyses by qualitative content analyses done by experts, e.g. literary scholars who could use the Abstractness

!
P
!
23!
tracking, neuroimaging, free recall, response times) could be used to cross-validate the direct 1!
offline measures (rating data; Dixon & Bortolussi, 2015; Jacobs, 2016a): neuroimaging data 2!
indicating a higher activation of neural networks associated with aesthetic liking (e.g., 3!
orbitofrontal cortex; Brown et al., 2011; Jacobs et al., 2016b) for young man sonnet(s) would 4!
be a case in point. 5!
6!
Across-poem contrasts 7!
A simple hypothesis based on the QNA data summarized in Figures 2 to 4 is that overall 8!
reading time is greater for sonnet 1 than for sonnet 138. More specific hypotheses regarding 9!
eye tracking studies can be derived from Figures 3 and 4, e.g., that sonnet 138 with its multiple 10!
higher cohesion scores should produce longer residuals of mean first-pass fixation times on 11!
text parts important for coherence building (e.g., conjunctions) than sonnet 1 (cf. Louwerse, 12!
2001). These hypotheses can be generalised, of course, by stating them „parametrically“, e.g. 13!
the higher the CML2 score of a sonnet, the shorter should be its reading time, mean gaze 14!
durations etc. The data in Figure 5 and Table 2 allow more specific hypotheses, such as that, 15!
say, narrativity ratings are higher for sonnet 42 than for sonnet 1, or that comprehensibility 16!
ratings are higher for sonnets 51 and 52 than for 31 and 144. Eye tracking experiments could 17!
also test the hypothesis that sonnets 145 and 125 produce a smaller likelihood of regressive 18!
saccades and longer gaze durations (associated with syntactic complexity, if other relevant 19!
variables are controlled for) than sonnets 80 or 67. 20!
21!
If reading speed or related eye movement parameters were the response measure of choice, the 22!
data of Figure 6 also are of interest. They allow to hypothesize that reading time also (co-23!
)varies significantly with the sonnets’ surprisal value, as should do the N400 amplitude, if ERP 24!
were the response measure. The German poet Durs Grünbein (1996) called for a poetry full of 25!
images rich in ‘‘factor N400’’, which he considered to be an index of the foregrounding 26!
potential of metaphors, speculating that such metaphors cause ‘‘neurolinguistic clashes’’ (cf. 27!
Jacobs, 2015b). According to the NCPM (Jacobs, 2011, 2015a,b), sonnets/lines/words with 28!
higher surprisal – and thus foregrounding – potential should more likely produce higher liking 29!
ratings, smaller saccades and longer fixation durations than sonnets low on surprisal. Data 30!
from a recent eye tracking study using short literary stories support these predictions (Van den 31!
Hoven et al., 2016) and it will be intriguing to see whether they also hold for sonnet reading. 32!
Regarding potential neuroimaging studies on sonnet reception, the surprisal data in Figure 6 33!
can be used to predict selective activation in the left inferior temporal sulcus, bilateral superior 34!
temporal gyrus, right amygdala, bilateral anterior temporal poles, and right inferior frontal 35!
sulcus (cf. Willems et al., 2015). 36!
37!
Turning to the affective-aesthetic aspects, the results in Figures 7 – 9 and Table 3 allow a 38!
number of predictions concerning a variety of response measures. At the level of direct offline 39!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Scale (Jacobs, 2015b) or similar tools for rating the TRI or similar features (cf. Jacobs et al., 2016a).!
!
!

!
P
!
24!
measures (e.g., questionnaires, scales) sonnets with high values for emotion potential (140, 1!
151, or 144) and/or RID primary process imagery (153, 154, 73) should produce significantly 2!
higher liking ratings, for example, than sonnets scoring low on this composite dimensions. 3!
Activation of the reward networks involved in aesthetic liking of literature (Jacobs et al., 4!
2016b) should correlate with such ratings, as could electrodermal activity (Jacobs et al., 5!
2016a). The data in Figure 9 and Table 4 are of special interest for empirical investigations 6!
because they raise the issue which of the five indices associated with mood perception and/or 7!
induction in poetry reception is the most valid and reliable. While the first three indices 8!
(valence mean, span, and sum) already have been shown to affect mood-related reader 9!
responses to some degree (Aryani et al., 2016; Lüdtke et al., 2014; Jacobs et al., 2016b), the 10!
other two are novel (SEANCE negative, positive) and still await empirical validation. 11!
12!
Figures 10 - 12 introduce three novel variables (TRI, SII, and SAP) that have not yet been used 13!
– as far as we know – in empirical studies on poetry reception. Tentatively, all three can be 14!
expected to correlate positively with liking and (positive) mood ratings, as well as other 15!
response measures (of the indirect type, e.g., electrodermal or neuronal activity) that are 16!
associated with liking. Following Hofmann and Jacobs’s (2014) and Kuchinke et al.’s (2013) 17!
results, a neuroimaging study on sonnet reception should – all other things being equal – also 18!
find increased activity in hippocampus, left inferior frontal gyrus, or the temporal pole for 19!
sonnets high on SAP, reflecting larger sematic competition as a function of more active 20!
representations (cf. also Forgács et al., 2012). 21!
22!
Within-poem contrasts (e.g., Q1-3 vs. C, octave vs. sestet, or linewise) 23!
A straightforward prediction derived from the QNAs discussed in „Question 9“ is that 24!
generally the couplets and the sestets containing them should be easier to process, read and 25!
comprehend than the sonnets’ bodies and octaves, respectively. This effect could be captured 26!
by a variety of measures including ratings, eye tracking or brain-electrical and neuroimaging 27!
measures. Since couplets and sestets also appear to have a higher emotion potential than the 28!
bodies or octaves, response measures sensitive to affective-aesthetic variables also should 29!
produce significant differences for these within-poem contrasts. 30!
31!
The line- and wordwise QNA results shown in Figures 3 and 6 encourage even more fine-32!
grained hypotheses concerning measures related to the comprehensibility and/or affective-33!
aesthetic responses, but will depend on the exact research question at hand. A straightforward 34!
example is to test the prediction of the NCPM that – again, all other things being equal – 35!
higher surprisal values more likely produce higher liking ratings, smaller saccades and longer 36!
fixation durations on a line- or even wordwise basis, e.g., with lines like line 12 from sonnet 1 37!
(surprisal value = 5.85: And, tender churl, makest waste in niggarding), or line two from 38!
sonnet 138 (2.43: I do believe her, though I know she lies). Finally, in line with Steen’s (2004) 39!
proposals, Figure 13 offers a wide field of interesting hypotheses concerning the processing of 40!
foregrounding elements. If everything else was controlled for, the likelihood of recognizing 41!
and/or appreciating stylistic devices, e.g. as assessed by a marking test, should increase quasi-42!
linearly towards the end of sonnets. For metaphorically used words contained in Bob Dylan’s 43!

!
P
!
25!
lyrics of Hurricane, Steen (2004) indeed confirmed this and we can only speculate that the 1!
same should hold for the present sonnets. In addition, the incremental nature of text 2!
comprehension – a reader’s knowledge of the text world becoming progressively larger, more 3!
specific, and more concrete – coupled with the decreasing number of new words could have 4!
measurable effects on a number of mental processes, e.g. attentional, mnestic, or emotional. To 5!
what extent on-line measures of sonnet reception such as eye tracking can capture such effects 6!
is an issue we cannot develop here, but a very general prediction is that overall reading speed 7!
and its multiple correlates should decrease (linearly or non-linearly) with increasing line 8!
number. 9!
10!
Part IV. Computational modeling 11!
In this section we aim to show how QNAs can be usefully combined with computational 12!
modeling for i) identifying those of the many quantifiable sonnet features that play a potential 13!
key role and ii) generating refined hypotheses for empirical investigations. We basically follow 14!
the approach adopted by Jacobs et al. (2016b) for computationally modeling elementary 15!
affective decisions (i.e., dis-liking) to words. 16!
17!
Question 11. Can the two meta-motifs be predicted by a machine learning algorithm 18!
using QNA indices? 19!
More particularly, we adopt an exemplary, formal decision tree modeling approach - a standard 20!
data mining / machine learning technique - to illustrate how QNA data can be used to predict 21!
the topic of texts, in our case the binary decision concerning the above mentioned young man 22!
vs dark lady motifs said to divide the 154 sonnets into two meta categories16. Decision tree 23!
modeling is successfully used for exploring relationships without having a good prior 24!
theoretical model: It can handle even large data problems efficiently allowing to test clear 25!
hypotheses, and the results are usually transparent and easily interpretable. 26!
27!
Stepwise decision tree (recursive partitioning) modeling 28!
Here we were interested in asking the question which of several models best predicted whether 29!
a sonnet belongs to the young man or dark lady category. Following Jacobs et al. (2016b) we 30!
used a stepwise modeling approach going from simple to complex models (i.e., few vs. many 31!
input variables) to see how much complexity in the input space is necessary to obtain an 32!
adequate model performance. 33!
34!
All models were trained on 70% of the sonnets (the randomly chosen training set is the part 35!
that estimates model parameters) and then validated on the remaining 30% (the validation set 36!
is the part that addresses or validates the predictive ability of the model). The stimuli were the 37!
N = 154 sonnets (126 young man and 28 dark lady). We created two sets of models tentatively 38!
termed cognitive/C and affective/A. Model C1 is based on the seven surface variables 39!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
16!We are aware of the fact that the sonnets can be categorized into more than these two groups, e.g. into the
„young man 1“ group (1-17) and the „young man 2“ group (18-126), or the „greek“ group (153-154), but
decided to keep things simple here.
!

!
P
!
26!
summarized in Table 1 above (i.e., number of words, type-token ratio, number of content and 1!
function words, pronoun noun ratio, and familiarity and imageability of content words; Model 2!
C2 comprises the eight CM easability scores shown in Figure 4 (i.e., narrativity, syntactic 3!
simplicity, word concreteness, referential, deep and verb cohesion, connectivity, and 4!
temporality); finally Model C3 accumulates all 76 CM descriptors we computed for the 5!
sonnets (see http://cohmetrix.com/), excluding the eight easability scores of model C2. The 6!
affective set contained Model A1 with the three RID indices (Figure 8), model A2 with the five 7!
mood indices (Figure 9), and model A3 with the 20 SEANCE component scores (Figure 7). 8!
The final „supermodel“ combined models C3 and A3 launching 96 cognitive and affective 9!
variables into the race. 10!
11!
The models were implemented using the PARTITION tool of the JMP Pro 11 software and 12!
model performance was gauged by the number of correct decisions, i.e., whether the model 13!
classified a sonnet correctly as belonging either to category one or two. Descriptively, model 14!
performance is expressed by the number of partitions, i.e., how many decisions are required to 15!
obtain maximum accuracy, generalised R2 and the rate of misclassifcations, i.e. how often the 16!
model classified a sonnet incorrectly. Table 5 summarizes the results. Each model in the table 17!
implements and tests a different hypothesis concerning the factors determining sonnet 18!
classification, e.g., model A1 tests to what extent the three RID scores predict correct 19!
classification. 20!
21!
Table 5. Input variables and performance evaluation for seven decision tree models 22!
23!
24!
25!
26!
27!
28!
29!
30!
31!
32!
33!
34!
35!
36!
37!
38!
39!
Descriptively, the most powerful models C3 and C3+A3 are the winners of this competition 40!
both producing a significant performance with about 98% correct binary decisions regarding 41!
the two major sonnet topics. Adding the 20 „affective“ variables of model A3 to the 76 of the 42!
„cognitive“ model C3 did not really pay: two decisions less, but no gain in accuracy. 43!
Model
Nbr. of
input
variables
Model performance
(nbr of partitions, generalized R2,
misclassification rate %)
C1 surface
structure
(see Table
1)
seven
16,.45,.12
C2 CM
eight
easability
scores
eight
15, .73, .084
C3 CM full
76
13, .90, .019
A1 RID
three
16, .53, .14
A2 mood
indices
five
17, .62, .11
A3
SEANCE
20
20
11, .78, .065
C3+A3
96
11, 90, .019

!
P
!
27!
1!
Figure A1 in the Appendix shows the detailed results for model C3 and helps understand the 2!
key variables that drive the correct classifications in this model. For illustrative purposes, we 3!
comment on it in detail here. The key question concerns the lexical diversity (type-token ratio 4!
content words) variable of CM17: Is a sonnet’s value >.86 and simultaneously features a lexical 5!
diversity (allwords) value18 of >=118.9, then it will be categorized as „young man“. Is its 6!
lexical diversity (allwords) value <118.9 and its hypernymy for verbs value19 <1.56, then it will 7!
also fall into the „young man“ group. Is its hypernymy for verbs value20 >1.56, then another 8!
decision is required before it can be said to be a „young man“ sonnet: Its incidence score of 9!
preposition phrases must be >=113. Is it <113, then its syntactic structure similarity index 10!
(i.e., the proportion of intersection tree nodes between all sentences and across paragraphs) 11!
must be <.06 and its value for the stem overlap adjacent sentences index (index 33 of coh-12!
metrix21) must be also <.077 for it to get into that category. Assuming that the principle of 13!
decision tree modeling is clear now, we renounce on commenting the left branch of the tree in 14!
the same detail. 15!
16!
To summarize, according to the C3 decision model, if a sonnet contains a relatively high 17!
number of new and of less specific words, has a relatively high incidence of preposition 18!
phrases but small syntactic structure similarity and stem overlap – all reducing its global 19!
cohesion – then it very likely deals with the young man topics. In contrast, if a sonnet features 20!
relatively low lexical diversity, but high adverbial phrase density and mean number of words 21!
before the main verb of the main clause in sentences (a good index of working memory load), 22!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
17!Lexical diversity refers to the variety of unique words (types) that occur in a text in relation to the total number of words
(tokens). When the number of word types is equal to the total number of words (tokens), then all of the words are different. In
that case, lexical diversity is at a maximum, and the text is likely to be either very low in cohesion or very short. A high
number of different words in a text indicates that new words need to be integrated into the discourse context. By contrast,
lexical diversity is lower (and cohesion is higher) when more words are used multiple times across the text.
18 Lexical diversity measure for all words.
!
19!Coh-Metrix also uses WordNet to report word hypernymy (i.e., word specificity). In WordNet, each word is located on a
hierarchical scale allowing for the measurement of the number of subordinate words below and superordinate words above
the target word. Thus, entity, as a possible hypernym for the noun chair, would be assigned the number 1. All other possible
hyponyms of entity as it relates to the concept of a chair (e.g., object, furniture, seat, chair, camp chair, folding chair) would
receive higher values. Similar values are assigned for verbs (e.g., hightail, run, travel). As a result, a lower value reflects an
overall use of less specific words, while a higher value reflects an overall use of more specific words.
20!Coh-Metrix also uses WordNet to report word hypernymy (i.e., word specificity). In WordNet, each word is located on a
hierarchical scale allowing for the measurement of the number of subordinate words below and superordinate words above
the target word. Thus, entity, as a possible hypernym for the noun chair, would be assigned the number 1. All other possible
hyponyms of entity as it relates to the concept of a chair (e.g., object, furniture, seat, chair, camp chair, folding chair) would
receive higher values. Similar values are assigned for verbs (e.g., hightail, run, travel). As a result, a lower value reflects an
overall use of less specific words, while a higher value reflects an overall use of more specific words.
21!This global overlap measure of referential cohesion relaxes the noun constraint held by the noun and argument overlap
measures. A noun in one sentence is matched with a content word (i.e., nouns, verbs, adjectives, adverbs) in a previous
sentence that shares a common lemma (e.g., tree/treed; mouse/mousey; price/priced).

!
P
!
28!
more words with a higher age-of-acquisition index, and more cues about temporality22, then it 1!
very likely belongs to the dark lady class. Thus, basically nine of the 76 CM variables of 2!
model C3 suffice for obtaining a very high classification accuracy. 3!
4!
In conclusion, the answer to question 11 above is: Yes, the two meta-motifs of the 154 sonnets 5!
can very well be predicted by a machine learning algorithm using QNA indices. More genrally, 6!
our analyses using decision tree models based on QNA tools like CM and SEANCE suggest 7!
that this methodological combination can serve as a heuristic for classifying texts into meta 8!
categories that could help identify authors, (sub)genres, epochs, or meta-motifs like in the 9!
present application. The combined toolbox (e.g., decision tree model + CM or SEANCE) does 10!
not (yet) produce a perfect result regarding the 154 sonnets, but misclassification rates below 11!
2% surely are encouraging for future applications of this or similar toolboxes. 12!
13!
Limitations and Outlook 14!
Before discussing some obvious limitations of the present work, we would like to borrow 15!
Tsur’s (2008, p. 147) statement paraphrasing Miller (1993, p. 392): „Our task is not to search 16!
for a unique paraphrase of the text, nor to find out how many meanings can be attributed to it, 17!
but to search for grounds that will constrain the basis of interpretations to a plausible set of 18!
alternatives“. We believe that the approach chosen in the present paper is in the spirit of Miller. 19!
If there are at least two basic levels of understanding texts and poetry in particular – evocation 20!
and interpretation (at rereading; Rosenblatt, 1978) – then QNAs plausibly can help capture 21!
aspects of the first level and arguably also of the second. While the number of possible 22!
meanings a reader can (re-)construct from a given poem in multiple re-readings may be quasi-23!
unlimited, empirical findings indicate that students often fail to engage the poems used in a 24!
study in a manner that accounts for the poems’ “poetic significance” with the consequence that 25!
what were essentially “plain sense” prose translations of the poems (cf. Richards, 1929, ten 26!
major pitfalls in poetry reading) rather than “evocations” of their possible meanings resulted 27!
(e.g., Harker, 1994). To capture aspects of deep reading of poetry, we recommend augmenting 28!
such QNA-based statistical analyses by qualitative content analyses done by experts, e.g., 29!
scholars of literature, poetics, or linguistics, as exemplified in Jacobs et al. (2016b). 30!
Of course, QNAs applied to stimulus selection/control and response prediction can only be as 31!
good or useful as the task and the response measures – and the hypotheses meaningfully 32!
relating stimuli and responses – developed by the experimenters allow them to be. That is, the 33!
methods for measuring experience/response should fit well with the hypotheses based on QNA 34!
or other tools. To what extent the direct vs. indirect on- and offline measures of poetry 35!
reception proposed in Dixon and Bortolussi (2015) and intensely debated with Kuiken (2015) 36!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
22!Texts that contain more cues about temporality and that have more consistent temporality (i.e., tense, aspect) are easier to
process and understand. In addition, temporal cohesion contributes to the reader’s situation model level understanding of the
events in the text.!
!
!
!

!
P
!
29!
and Jacobs (2016b) can capture the „plain sense“, evocation and/or interpretation aspects of 1!
any poetry reading act is an open issue that – in our opinion – can benefit from the application 2!
of QNA as much as from the development of more sophisticated models and methods for the 3!
study of literary reading. 4!
5!
This being said, at least two obvious lacunae limit the usefulness of the present QNAs for 6!
studies of the dynamic sound-meaning nexus typical for poetry reception (e.g., Schrott & 7!
Jacobs, 2011; Tsur, 1998). 8!
9!
Implicit sound and iconicity 10!
First, the lack of predictors at the level of implicit or mental sound (i.e. generated via 11!
phonological or prosodic recoding of the printed input), phonological iconicity, rhythm, or 12!
rhyme, which all have been shown to affect reader responses in silent lyrics or poetry 13!
processing to some extent (e.g., Aryani et al., 2016; Menninghaus et al., 2014; Tsur, 2006; 14!
Wallace & Rubin, 1991). As an example, in their ground-breaking case analysis of 15!
Baudelaire’s “Les chats,” Jakobson and Lévi-Strauss (1962) analyzed the phonological texture 16!
of the poem by quantifying the number of nasals in the poem’s first quartet (“two to three per 17!
line”) or the interaction between formal and semantic features (i.e., nasal vowels and the idea 18!
of light) in the last trio. It should be noted, though, that when dealing with written sonnets we 19!
know of no firm evidence that non-expert readers silently read sonnets in any way resembling 20!
theories of scansion. Even reading aloud the sonnets must not strictly follow the iambic 21!
pentameter but take into account subtler intonations observing inner antitheses and parallels 22!
(cf. Vendler, 1997, p. 37). In sum, complementing the present QNAs of sonnets by tools like 23!
SPARSAR (Delmonte, 2016) for quantifying structural, or EMOPHON (Aryani et al., 2013) 24!
for affective sound properties would be a good first step towards allowing predictions about 25!
potential „sound“ (including rhythm) effects. Still, such efforts must be preceded or 26!
accompanied by experiments demonstrating exactly which implicit structural and/or affective 27!
sound properties affect poetic reading acts in addition to – or in interaction with – the present 28!
or other QNA variables. 29!
30!
Metaphoricity and style figures 31!
The second obvious lacuna is the absence of qualitative descriptors of the metaphoricity or, 32!
more generally, the foregrounding/backgrounding quotient (Jacobs, 2015b) of the sonnets (e.g., 33!
McQuarrie & Mick, 1996; McQuire et al., 2016; Pragglejazz group, 2007; Schrott & Jacobs, 34!
2011; Steen, 1999, 2002, 2004; Stockwell 2009). For example, it can be safely assumed that a 35!
line like „Feed’st thy light’s flame with self-substantial fuel,“ from Q2 of sonnet 1 above is not 36!
a summation of the phonological and semantic representations of its individual words but – as 37!
outlined in the introduction – a catachresis which likely evokes aesthetic and reflective reader 38!
responses according to the NCPM (Jacobs, 2015a). Especially in poetry, meaning emerges 39!
dynamically out of the full context determining which semantic fields and senses of a word are 40!
heightened and which are deactivated (cf. Millis & Larson, 2008; Schrott & Jacobs, 2011). 41!
Naturally, the „static“ purely descriptive QNA indices presented are insufficient (and also not 42!
meant) to explain such context-dependent, reader-specific emergent dynamics and attempts at, 43!

!
P
!
30!
say, metaphoric constructions. Moreover, they also neglect potential conceptual or rhetorical 1!
effects produced at a deeper linguistic level, e.g., modifications of tense, subject-position, or 2!
clause-patterns. Finally, they offer no analysis of „what isn’t printed in the text“, e.g. ellipsis or 3!
allusions (cf. Jacobs, 2015b). 4!
5!
Still, the present tools can be augmented by qualitative, typological or taxonomic tools like the 6!
Abstractness Scale for determining foregrounding features such as meter or mimesis (Jacobs, 7!
2015b; Meyer-Sickendieck, 2011), by metaphoricity analyses that, e.g., count and interpret 8!
antitheses or chiasma, so frequently used in the sonnets, or that evaluate the conceptual, 9!
linguistic, communicative, or affective qualities of metaphors (e.g., Schrott & Jacobs, 2011; 10!
Steen, 1999; Stockwell, 2009; Vendler, 1997), as well as by computational linguistic analyses 11!
(e.g., Kintsch, 2000; Kintsch & Magalath, 2011). This should help develop full-fledged 12!
process models of the type discussed in Jacobs (2015b) which may serve at least as 13!
sophisticated null-models for predicting context- and reader-dependent effects of poetic text 14!
features on direct or indirect response measures. 15!
16!
In conclusion, the present QNA approach to sonnets is not meant to replace deep-structure 17!
expert qualitative analyses or critical interpretations of the kind of Jakobson and Jones (1970) 18!
or Vendler (1997), but as a complement or null-model against which any model of 19!
foregrounding effects due to stylistic devices can be tested regarding its account of additional 20!
variance in reader responses. 21!
22!
23!
24!
25!
26!
27!
28!

!
P
!
31!
References 1!
Altmann, U., Bohrn, I. C., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2012). The power of 2!
emotional valence-from cognitive to affective processes in reading. Front. Hum. Neurosci. 6:192. doi: 3!
10.3389/fnhum.2012. 00192 4!
Altmann, U., Bohrn, I. C., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2014). Fact vs fiction-5!
how paratextual information shapes our reading processes. Soc. Cogn. Affect. Neurosci. 9, 22–29. doi: 6!
10.1093/scan/nss098 7!
Aryani, A., Jacobs, A. M., and Conrad, M. (2013). Extracting salient sublexical units from written 8!
texts: “Emophon,” a corpus-based approach to phonological iconicity. Front. Psychol. 4:654. doi: 9!
10.3389/fpsyg.2013.00654 10!
Aryani, A., Kraxenberger, M., Ullrich, S., Jacobs, A. M., & Conrad, M. (2016). Measuring the ba- sic 11!
a ective tone of poems via phonological saliency and iconicity. Psychology of Aesthetics, Creativity, 12!
and the Arts, 10, 191–204. DOI: 10.1037/ aca0000033 13!
Beaty, R. E., Benedek, M., Silvia, P. J., & Schacter, D. L. (2016). Creative cognition and brain 14!
network dynamics. Trends in Cognitive Sciences, 20, 87–95. 15!
http://dx.doi.org/10.1016/j.tics.2015.10.004 16!
Berlyne, D. E. (1971). Aesthetics and Psychobiology. New York: Appleton-Century-Crofts. 17!
Bestgen, Y. (1994). Can emotional valence in stories be determined from words? Cogn. Emot. 8, 21–18!
36. doi: 10.1080/02699939408408926 19!
Bohrn, I. C., Altmann, U., and Jacobs, A. M. (2012a). Looking at the brains behind figurative 20!
language—A quantitative meta-analysis of neuroimaging studies on metaphor, idiom and irony 21!
processing. Neuropsychologia 50, 2669–2683. doi: 10.1016/j.neuropsychologia.2012.07.021 22!
Bohrn, I. C., Altmann, U., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2012b). Old proverbs in 23!
new skins—an FMRI study on defamiliarization. Front. Psychol. 3:204. doi: 24!
10.3389/fpsyg.2012.00204 25!
Bohrn, I. C., Altmann, U., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2013). When we like 26!
what we know—a parametric fMRI analysis of beauty and familiarity. Brain Lang. 124, 1–8. doi: 27!
10.1016/j.bandl.2012.10.003 28!
Bradley, M. M., and Lang, P. J. (1999). Affective Norms for English Words (ANEW): Stimuli 29!
Instruction and Affective Ratings Technical Report C-1. Gainesville, FL: University of Florida. 30!
Briesemeister, B. B., Kuchinke, L., and Jacobs, A. M. (2011). Discrete emotion norms for nouns - 31!
Berlin Affective Word List (DENN-BAWL). Behav. Res. Methods 43, 441–448. doi: 10.3758/s13428-32!
011-0059-y 33!
Briesemeister, B. B., Kuchinke, L., and Jacobs, A. M. (2014). Emotion word recognition: discrete 34!
information effects first, continuous later? Brain Res. 1564, 62–71. doi: 35!
10.1016/j.brainres.2014.03.045 36!

!
P
!
32!
Briesemeister, B. B., Kuchinke, L., Jacobs, A. M., and Braun, M. (2015). Emotions in reading: 1!
dissociation of happiness and positivity. Cogn. Affect. Behav. Neurosci. 15, 287–298. doi: 2!
10.3758/s13415-014-0327-2 3!
Brown, S., Gao, X., Tisdelle, L., Eickhoff, S.B., Liotti, M., (2011). Naturalizing aesthetics: brain areas 4!
for aesthetic appraisal across sensory modalities. NeuroImage 58, 250–258. 5!
Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current 6!
word frequency norms and the introduction of a new and improved word frequency measure for 7!
American English. Behavior Research Methods, 41, 977-990. 8!
Burke, M. (2011). Literary reading, cognition and emotion: An exploration of the oceanic mind. 9!
London: Routledge. 10!
Burke M (2015) The neuroaesthetics of prose fiction: pitfalls, parameters and prospects. Front. Hum. 11!
Neurosci. 9:442. doi: 10.3389/fnhum.2015.00442 12!
Chatterjee, A., and Vartanian, O. (2014). Neuroaesthetics. Trends Cogn. Sci. 18, 370–375. 13!
doi:10.1016/j.tics.2014.03.003 14!
Chen, Q., Zhang, J., Xu, X., Scheepers, C., Yang, Y., & Tanenhaus, M. K. (2016). Prosodic 15!
expectations in silent reading: ERP evidence from rhyme scheme and semantic congruence in classic 16!
Chinese poems. Cognition, 154, 11-21. 17!
Citron, F. M. (2012). Neural correlates of written emotion word processing: a review of recent 18!
electrophysiological and hemodynamic neuroimaging studies. Brain Lang. 122, 211–226. doi: 19!
10.1016/j.bandl.2011 20!
Crossley, S. A., Allen, L. K., Kyle, K., & McNamara, D.S. (2014). Analyzing Discourse Processing 21!
Using the Simple Natural Language Processing Tool (SiNLP). Discourse Processes, 51 (5-6), 511-22!
534. 23!
24!
Crossley, S. A., Kyle, K., and McNamara, D. S. (2015). The tool for the automatic analysis of text 25!
cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research 26!
Methods, 1-11. DOI 10.3758/s13428-015-0651-7 27!
Crossley, S. A., Kyle, K., & McNamara, D. S. (2016). Sentiment Analysis and Social Cognition 28!
Engine (SEANCE): An Automatic Tool for Sentiment, Social Cognition, and Social Order 29!
Analysis. Behavior Research Methods. DOI 10.3758/s13428-016-0743-z 30!
Crystal, D. (2008). ‘Think On My Words’: Exploring Shakespeare’s Language. Cambridge: 31!
Cambridge University Press. 32!
Cupchik, G C (1986) A decade after Berlyne: New directions in experimental aesthetics, Poetics, 15, 33!
345-369. 34!
De Beaugrande, R. (1978). Toward a general theory of creativity. Poetics 8(3): 269-306. 35!
Delmonte, R. (2016). Exploring Shakespeare’s Sonnets with SPARSAR. Linguistics and Literature 36!

!
P
!
33!
Studies 4(1): 61-95, http://www.hrpub.org DOI: 10.13189/lls.2016.040110 1!
Dixon, P., Bortolussi, M. (2015). Measuring Literary Experience: Comment on Jacobs (2015). 2!
Scientific Study of Literature, 5(2), 178–182. 3!
Fellbaum, C. (Ed.). (1998). WordNet: An electronic lexical database. Cambridge: MIT Press. 4!
Forgács, B., Bohrn, I. C., Baudewig, J., Hofmann, M. J., Pléh, C., and Jacobs, A. M. (2012). Neural 5!
correlates of combinatorial semantic processing of literal and figurative noun-noun compound words. 6!
Neuroimage 63, 1432–1442. doi: 10. 1016/j.neuroimage.2012.07.029 7!
Frank, S. L. (2013). Uncertainty reduction as a measure of cognitive processing load in sentence 8!
comprehension. Topics in Cognitive Science, 5, 475–494. 9!
Frank, S. L., Leun, J. O., Giulia, G., and Vigliocco, G. (2015). The ERP response to the amount of 10!
information conveyed by words in sentences. Brain Lang. 140, 1–11. doi: 11!
10.1016/j.bandl.2014.10.006 12!
Goldman, A. (2006) Simulating Minds, Oxford University Press 13!
14!
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text 15!
on cohesion and language. Behavior Research Methods, Instruments, & computers, 36, 193–202. 16!
doi:10. 3758/BF03195564 17!
Graesser, A. C., & McNamara, D. S. (2010). Computational analyses of multilevel discourse 18!
comprehension. Topics in Cognitive Science, DOI: 10.1111/j.1756-8765.2010.01081.x 19!
Graesser, A. C., McNamara, D. S., & Kulikowich, J. (2011). Coh-Metrix: Providing multilevel 20!
analyses of text characteristics. Educational Researcher, 40, 223–234. 21!
Green, M. C., & Brock, T. C. (2000). The role of transportation in the persuasiveness of pub- lic 22!
narratives. Journal of Personality and Social Psychology, 79, 701–721. 23!
Grünbein, D. (1996). ‘‘Katze und Mond,’’ in Galilei Vermisst Dantes Hölle und Bleibt an den Maßen 24!
Hängen, ed D. Grünbein (Frankfurt A. M.: Suhrkamp), 116–128. 25!
Habermas, T., & de Silveira, C. (2008). The development of global coherence in life narratives across 26!
adolescence: Temporal, causal, and thematic aspects. Developmental Psychology, 44, 707–721. 27!
Harker, W.J. (1997) Literary texts as models: Implications for the empirical study of literature. In S. 28!
Tötösy de Zepetnek & I. Sywenky (Eds.), The systemic and empirica lapproach to literature and 29!
culture as theory and application(pp.51- 65). Edmonton, Canada: University of Alberta. 30!
Hofmann, M. J., & Jacobs, A. M. (2014). Interactive activation and competition models and semantic 31!
context: From behavioral to brain data. Neuroscience & Biobehavioral Reviews, 46(1), 85–104. 32!
doi:10.1016/j.neubiorev.2014.06.011 33!
Hsu, C.-T., Conrad, M., and Jacobs, A. M. (2014). Fiction feelings in Harry Potter: Haemodynamic 34!
response in the mid-cingulate cortex correlates with immersive reading experience. Neuroreport, 25, 35!

!
P
!
34!
1356–1361. doi: 10.1097/ WNR.0000000000000272 1!
Hsu, C. T., Jacobs, A. M., Citron, F., & Conrad, M. (2015). The emotion potential of words and 2!
passages in reading Harry Potter - An fMRI study. Brain and Language, 142, 96-114. 3!
Hutcheon, L. (2012) A Theory of Adaptation. 2nd edition with Siobhan O’Flynn (London and New 4!
York: Routledge) 5!
Jacobs, A. M. (2011). Neurokognitive Poetik: Elemente eines Modells des literarischen Lesens 6!
[Neurocognitive poetics: Elements of a model of literary reading]. In R. Schrott and A. M. Jacobs 7!
(Eds.), Gehirn und Gedicht: Wie wir unsere Wirklichkeiten konstruieren [Brain and poetry: How we 8!
construct our realities] (pp. 492–520). Munich: Carl Hanser. 9!
Jacobs, A. M. (2015a). Towards a neurocognitive poetics model of literary reading, in Towards a 10!
Cognitive Neuroscience of Natural Language Use, ed R. Willems (Cambridge: Cambridge University 11!
Press), 135–159. 12!
Jacobs AM (2015b) Neurocognitive poetics: methods and models for investigating the neuronal and 13!
cognitive-affective bases of literature reception. Front. Hum. Neurosci. 9:186. doi: 14!
10.3389/fnhum.2015.00186 15!
Jacobs, A. (2015c). The scientific study of literary experience: Sampling the state of the art. Scientific 16!
Study of Literature, 5(2), 139-170. 17!
Jacobs, A. (2016a). The scientific study of literary experience and neuro-behavioral responses to 18!
Literature: Reply to Commentaries, Scientific Study of Literature, 6(1). 19!
Jacobs, A. M., and Kinder, A. (2015). Worte als Worte erfahren: wie erarbeitet das Gehirn Gedichte 20!
(Experience words as words: how the brain constructs poems), in Kind und Gedicht (Child and Poem), 21!
ed A. Pompe (Berlin: Rombach), 57–76. 22!
Jacobs, A. M., and Schrott, R. (2015). Gefesselt im Kopfkino: Von Kippschaltern, Madeleine Effekten 23!
und Don Quichote Syndromen bei der Immersion in Textwelten (Captivated in the mind’s cinema: of 24!
trigger-switches, Don Quichote syndroms and immersion in text worlds). Available online at: 25!
FIKTION.CC 26!
Jacobs AM, Võ ML-H, Briesemeister BB, Conrad M, Hofmann MJ, Kuchinke L, Lüdtke J and Braun 27!
M (2015) 10 years of BAWLing into affective and aesthetic processes in reading: what are the echoes? 28!
Front. Psychol. 6:714. doi: 10.3389/fpsyg.2015.00714 29!
Jacobs, A. M., Lüdtke, J., Aryani, A., Meyer-Sickendiek, B., and Conrad, M. (2016a). Mood-empathic 30!
and aesthetic responses in poetry reception: A model-guided, multilevel, multimethod approach. 31!
Scientific Study of Literature, 6(1). 32!
Jacobs AM, Hofmann MJ and Kinder A (2016b) On Elementary Affective Decisions: To Like Or Not 33!
to Like, That Is the Question. Front. Psychol. 7:1836. doi: 10.3389/fpsyg.2016.01836 34!
Jacobsen, T. (2006). Bridging the arts and sciences: a framework for the psychology of aesthetics. 35!
Leonardo 39, 155–162. doi: 10.1162/leon.2006.39.2.155 36!

!
P
!
35!
Jakobson, R., Jones, L.G. (1970). Shakespeare’s verbal art in „Th’Expense of Spirit“. De Gruyter. 1!
Jakobson, R., and Lévi-Strauss, C. (1962). Les Chats de Charles Baudelaire. L’Homme 2, 5–21. doi: 2!
10.3406/hom.1962.366446 3!
Kintsch, W. (2000). Metaphor comprehension: a computational theory. Psychon. Bull. Rev. 7, 257–4!
266. doi: 10.3758/bf03212981 5!
Kintsch, W. (2012). Musing about beauty. Cogn. Sci. 36, 635–654. doi: 10.1111/j.1551-6!
6709.2011.01229.x 7!
Kintsch, W., & Mangalath, P. (2011). The construction of meaning. Topics in Cognitive Science, DOI: 8!
10.1111/ j.1756-8765.2010.01107.x 9!
Kivy, P. (1990). Music Alone: Philosophical Reflections on the Purely Musical Experience. Ithaca, 10!
NY: Cornell University Press. 11!
Klare, G. R. (1974–1975). Assessing readability. Reading Research Quarterly, 10, 62–102. 12!
Koelsch, S., Jacobs, A. M., Menninghaus, W., Liebal, K., Klann-Delius, G., von Scheve, C., et al. 13!
(2015). The quartet theory of human emotions: an integrative and neurofunctional model. Phys. Life 14!
Rev. 13, 1–27. doi: 10.1016/j.plrev.2015.03.001 15!
Krumhansl, C. L. (1997). An exploratory study of musical emotions and psychophysiology. Canadian 16!
Journal of Experimental Psychology, 51(4), 336-52. 17!
Kuchinke, L., Fritzemeier, S., Hofmann, M.J., Jacobs, A.M., 2013. Neural correlates of episodic 18!
memory: associative memory and confidence drive hippocampus activations. Behav. Brain Res. 254, 19!
92–101. 20!
Kuhlmann, M., Hofmann, M. J., Briesemeister, B. B., and Jacobs, A. M. (2016). Mixing positive and 21!
negative valence: Affective-semantic integration of bivalent words. Scientific Reports, 6:30718, 1-7. 22!
DOI: 10.1038/srep30718 23!
Kuzmičová, A. (2014). Literary narrative and mental imagery: A view from embodied cognition. 24!
Style, 48(3), 275–293. 25!
Leder H, Belke B, Oeberst A, Augustin MD. (2004). A model of aesthetic appreciation and aesthetic 26!
judgments. Br J Psychol., 95:489–508.  27!
Leder H, Nadal M. (2014). Ten years of a model of aesthetic appreciation and aesthetic judgments: the 28!
aesthetic episode – developments and challenges in empirical aesthetics. Br J Psychol, 105(4):443–29!
64. http://dx.doi.org/10.1111/bjop.12084. 30!
Leder, H., Markey, P. S., and Pelowski, M. (2015). Aesthetic emotions to art – what they are and what 31!
makes them special. Comment on “the quartet theory of human emotions: an integrative and 32!
neurofunctional model” by Koelsch et al. Phys. Life Rev. 13, 67–70. doi: 10.1016/j.plrev.2015.04.037 33!
Lehne, M., Engel, P., Rohrmeier, M., Menninghaus, W., Jacobs, A. M., and Koelsch, S. (2015). 34!

!
P
!
36!
Reading a suspenseful literary text activates brain areas related to social cognition and predictive 1!
inference. PLoS ONE 10:e0124550. doi: 10.1371/journal.pone.0124550 2!
Levinson, J. (1997). Music and negative emotion. In Robinson, J. (Ed.), Music and Meaning (pp. 215–3!
41). Ithaca, NY: Cornell University Press. 4!
Lindquist KA, MacCormack JK and Shablack H (2015) The role of language in emotion: predictions 5!
from psychological constructionism. Front. Psychol. 6:444. doi: 10.3389/fpsyg.2015.00444 6!
Liu, S., Erkkinen, M. G., Healey, M. L., Xu, Y., Swett, K. E., Chow, H. -M., & Braun, A. R. (2015). 7!
Brain activity and connectivity during poetry composition: Toward a multidimensional model of the 8!
creative process. Human Brain Mapping, 36, 3351–3372doi: 10.1002/hbm.22849 9!
Lüdtke, J., and Jacobs, A. M. (2015). The emotion potential of simple sentences: Additive or 10!
interactive effects of nouns and adjectives? Frontiers in Psychology, 6, 1137. doi: 11!
10.3389/fpsyg.2015.01137 12!
Lüdtke, J., Meyer-Sickendiek, B., and Jacobs, A. M. (2014). Immersing in the stillness of an early 13!
morning: Testing the mood empathy hypothesis in poems. Psychology of Aesthetics, Creativity, and 14!
the Arts, 8, 363–377. doi: 10.1037/a0036826 15!
Mann, W., & Thompson, S. (1988). Rhetorical structure theory: toward a functional theory of text 16!
organization. Text 8, 243–281. 17!
Marin MM (2015) Crossing boundaries: toward a general model of neuroaesthetics. Front. Hum. 18!
Neurosci. 9:443. doi: 10.3389/fnhum.2015.00443 19!
Marin MM, Lampatz A, Wandl M and Leder H (2016) Berlyne Revisited: Evidence for the 20!
Multifaceted Nature of Hedonic Tone in the Appreciation of Paintings and Music. Front. Hum. 21!
Neurosci. 10:536. doi: 10.3389/fnhum.2016.00536 22!
Martindale, C (1975) Romantic progression The psychology cf literary history Washington, DC 23!
Hemisphere. 24!
McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C. (2010). Coh-Metrix: 25!
Capturing linguistic features of cohesion. Discourse Processes, 47, 292–330. 26!
McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and 27!
discourse with Coh-Metrix. Cambridge, UK: Cambridge University Press. 28!
Mc Quarrie, E. F., & Mick, D. G. (1996). Figures of Rhetoric in Advertising Language. J CONSUM 29!
RES, 19, 424-437. 30!
McQuire, M., McCollum, L., & Chatterjee, A. (2016). Aptness and beauty in metaphor. Language and 31!
Cognition, Available on CJO 2016 doi:10.1017/langcog.2016.13 32!
Meireles, R.C. (2005). The hermeneutics of symbolical imagery in Shakespeare’s sonnets. 33!
Unpublished Dissertation, Univ. Porto Alegre, Bresil. 34!

!
P
!
37!
Menninghaus, W., Bohrn, I. C., Altmann, U., Lubrich, O., & Jacobs, A. M. (2014). Sounds funny? 1!
Humor effects of phonological and prosodic figures of speech. Psychology of Aesthetics, Creativity, 2!
and the Arts, 8(1), 71. 3!
Menninghaus, W., Bohrn, I. C., Knoop, C. A., Kotz, S. A., Schlotz, W., & Jacobs, A. M. (2015). 4!
Rhetorical features facilitate prosodic processing while handicapping ease of semantic comprehension. 5!
Cognition, 143, 48-60. 6!
Meyer-Sickendiek, B. (2011). Lyrisches Gespür: Vom Geheimen Sensorium Moderner Poesie [The 7!
lyrical sense of feeling. About the secret sensorium of modern poetry]. Paderborn, Germany: Fink. 8!
Miall, D. S.(1989). Beyond the schema given: Affective comprehension of literary narratives, 9!
Cognition & Emotion, 3:1,55 — 78. DOI: 10.1080/02699938908415236 10!
Miall, D. S., and Kuiken, D. (1994). Foregrounding, defamiliarization and affect: response to 11!
literary stories. Poetics 22, 389–407. doi: 10.1016/0304- 422x(94)00011-5 12!
Miall, D.S. & Dissanayake, E. (2003). The poetics of babytalk. Hum Nat (2003) 14: 337. 13!
doi:10.1007/s12110-003-1010-4 14!
Miller, G. 1993. “Images and models, similes and metaphors”. In A. Ortony (ed), Metaphor and 15!
thought, 2nd edition. Cambridge: Cambridge University Press, 357–400. 16!
Millis, K., & Larson, M. (2008). Applying the construction-integration framework to aesthetic 17!
responses to representational artworks. Discourse Processes, 45, 263–287. 18!
Nicklas, P., & Jacobs, A. M. (2016). Rhetorics, neurocognitive poetics and the aesthetics of 19!
adaptation. Poetics Today. In press. 20!
Oatley, K. (1994). A taxonomy of the emotions of literary response and a theory of identification in 21!
fictional narrative. Poetics 23, 53–74. doi: 10.1016/0304- 422x(94)p4296-s 22!
Oatley, K. (2016). Fiction: Simulation of social worlds. Trends in Cognitive Sciences, 20, 618–628. 23!
http://dx.doi.org/10.1016/j.tics.2016.06.002 24!
O’Sullivan, N., Davis, P., Billington, J., Gonzalez-Diaz, V., & Corcoran, R. (2015). “Shall I compare 25!
thee”: The neural basis of literary awareness, and its bene ts to cognition, Cortex, 73, 144–157. doi: 26!
10.1016/j.cortex.2015.08.014 27!
Panksepp, J. (2008). The power of the word may reside in the power of affect. Integr. Psychol. Behav. 28!
Sci. 42, 47–55. doi: 10.1007/s12124-007-9036-5 29!
Pelowski M, Markey PS, Lauring JO and Leder H (2016) Visualizing the Impact of Art: An Update 30!
and Comparison of Current Psychological Models of Art Experience. Front. Hum. Neurosci. 10:160. 31!
doi: 10.3389/fnhum.2016.00160 32!
Pleh, C. (2003). Narrativity in text construction and self construction, Neohelicon 30, 1, 187–205. 33!
Pragglejaz Group (2007). MIP: A method for identifying metaphorically used words in discourse. 34!
Metaphor and Symbol, 22–1: 1–39. 35!

!
P
!
38!
Richards, I.A, (1929). Practical criticism: A study of literary judgment. New York: Harcourt Brace 1!
Jovanovich. 2!
Rosenblatt, L.M. (1978). The reader, the text, thepoem: The transactionaltheoryof the literarywork. 3!
Carbondale, IL: Southern Illinois University Press. 4!
Ryan, Marie-Laure (2001) Narrative asVirtual Reality.Immersion and Interactivity in Literature and 5!
Electronic Media. Baltimore/London: Johns Hopkins Uni- versity Press. 6!
Schmidtke, D. S., Schröder, T., Jacobs, A. M., and Conrad, M. (2014). ANGST: affective norms for 7!
german sentiment terms, derived from the affective norms for english words. Behav. Res. Methods 46, 8!
1108–1118. doi: 10.3758/s13428- 013-0426-y 9!
Schrott, R., and Jacobs, A. M. (2011). Gehirn und Gedicht: Wie wir unsere Wirklichkeiten 10!
konstruieren (Brain and Poetry: How We Construct Our Realities). München: Hanser. 11!
Shaoul, C., & Westbury, C. (2010). Exploring lexical co- occurrence space using HiDEx. Behavior 12!
Research Methods, 42, 393–413. 13!
Shimron, J. (1980). Psychological processes behind the comprehension of a poetic text. Instructional 14!
Science, 9, 43–66. doi: 10.1007/BF00118968 15!
Simonton, DK (1989). Shakespeare's Sonnets: A Case of and for Single-Case Historiometry. Journal 16!
of Personality, 57, 695-721. 17!
Simonton, D. K. (1990). Lexical choices and aesthetic success: A computer content analysis of 154 18!
Shakespeare sonnets. Computers and the Humanities, 24, 254–261. 19!
Smith, N. J., & Levy, R. (2013). The effect of word predictability on reading time is logarithmic. 20!
Cognition, 128, 302–319. 21!
Steen, G. (1999). Analyzing Metaphor in Literature: With Examples from William Wordsworth's "I 22!
Wandered Lonely as a Cloud". Poetics Today, 20, 3, 499-522. 23!
Steen, G. (2002). Metaphor in Bob Dylan's 'Hurricane': Genre, Language, and Style. In: Cognitive 24!
Stylistics: Language and Cognition in Text Analysis. Semino, e., & Culpepper, J. (eds). Amsterdam: 25!
John Benjamins, 183-210. 26!
Steen, G. (2004). Can discourse properties of metaphor affect metaphor recognition? Journal of 27!
Pragmatics, 36, 1295–1313. 28!
Stockwell, P. (2009). The Cognitive Poetics of Literary Resonance, Language and Cognition 1, 25–29!
44. doi: 10.1515/LANGCOG.2009.002 30!
Sylvester T, Braun M, Schmidtke D and Jacobs AM (2016) The Berlin Affective Word List for 31!
Children (kidBAWL): Exploring Processing of Affective Lexical Semantics in the Visual and 32!
Auditory Modalities. Front. Psychol. 7:969. doi: 10.3389/fpsyg.2016.00969 33!
Taruffi, L. (2016). Why We Listen to Sad Music: Effects on Emotion and Cognition. Ph.D. 34!

!
P
!
39!
dissertation, Freie Universität Berlin. 1!
Tsur, R. 1998. Poetic Rhythm: Structure and Performance — An Empirical Study in Cognitive Poetics. 2!
Bern: Peter Lang. 3!
Tsur, R. (2006). Delivery style and listener response in the rhythmical performance of Shakespeare’s 4!
sonnets, College Literature 33:1, 170–196. 5!
Tsur, R. (2008). Deixis in literature: What isn’t cognitive poetics? Pragmatics & Cognition 16, 119–6!
150, doi 10.1075/p&c.16.1.08tsu 7!
Turner, F., & Pöeppel, E. (1983). The neural lyre: poetic meter, the brain and time. Poet. Mag. 12, 8!
277–309. 9!
Ullrich, S., Aryani, A., Kraxenberger, M., Jacobs, A.M., & Conrad, M. (2016). Textual features as 10!
basis of the general affective meaning: Exploring the interplay of lexical affective features, dynamic 11!
inter-lexical shifts, and the basic affective tone in poetry. In revision. 12!
Van Peer, W. (1986). Stylistics and psychology: Investigations of foregrounding. London, UK: Croom 13!
Helm. 14!
Van den Hoven, E., Hartung, F., Burke, M., & Willems, R. (2016). Individual differences in 15!
sensitivity to style during literary reading: Insights from eye-tracking, Collabra, in press. 16!
Vaughan-Evans, A., Trefor, R., Jones, L., Lynch, P., Wyn Jones, M., & Thierry, G. (2016). Implicit 17!
detection of poetic harmony by the naïve brain, Front. Psychol. 18!
Vendler, H. (1997). The art of Shakespeare’s son- nets. Cambridge, MA: Harvard University Press. 19!
Võ, M. L. H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M. J., and Jacobs, A. M. (2009). The 20!
Berlin Affective Word List Reloaded (BAWL-R). Behav. Res. Methods 41, 534–538. doi: 10.3758/B. 21!
R. M.41.2.534 22!
Võ, M. L. H., Jacobs, A. M., and Conrad, M. (2006). Cross-validating the berlin affective word list. 23!
Behav. Res. Methods 38, 606–609. doi: 10.3758/BF03 193892 24!
Wainwright, J. (2011). Poetry: the basics. London: Routledge. 25!
Wallace, W.T., & Rubin, D.C. (1991) Characteristics and constraints in ballads and their effect on 26!
memory. Discourse Processes, 14, 181-202. 27!
Wallentin, M., Nielsen, A. H., Vuust, P., Dohn, A., Roepstor , A., & Lund, T. E. (2011). Amygdala 28!
and heart rate variability responses from listening to emotionally intense parts of a story. Neuroimage, 29!
58, 963–973. doi: 10.1016/j.neuroimage.2011.06.077 30!
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance 31!
for 13,915 English lemmas. Behavior Research Methods, 45, 1191–1207. doi:10.3758/s13428-012-32!
0314- 33!

!
P
!
40!
Westbury, C., Keith, J., Briesemeister, B. B., Hofmann, M. J. & Jacobs, A. M. (2014). Avoid violence, 1!
rioting, and outrage; approach celebration, delight, and strength: Using large text corpora to compute 2!
valence, arousal, and the basic emotions. Quarterly Journal of Experimental Psychology 68, 1599–3!
1622, doi: 10.1080/17470218.2014.970204.  4!
Willems, R. (ed) (2015). Cognitive Neuroscience of Natural Language Use. Cambridge: Cambridge 5!
University Press. 6!
Willems RM, Frank SL, Nijhof AD, Hagoort P, van den Bosch A. 2015. Prediction during natural 7!
language comprehension. Cereb Cortex. 26:2506–2516. 8!
Willems, R., & Jacobs, A. M. (2016). Caring about Dostoyevsky: The untapped potential of studying 9!
literature. Trends in Cognitive Sciences, 20, 243–245. 10!
Yaron, I. (2002). Processing of obscure poetic texts: Mechanism of selection. Journal of Literary 11!
Semantics, 31(2), 133–170. doi: 10.1515/jlse.2002.013 12!
Yaron, I. (2008). What is a “difficult” poem? Towards a definition. Journal of Literary Semantics, 13!
37(2), 129–150. doi: 10.1515/jlse.2008.008 14!
15!
Zeman, A., Milton, F., Smith, A., & Rylance, R. (2013). By heart: An fMRI study of brain activation 16!
by poetry and prose. Journal of Consciousness Studies, 20, 132–158. 17!
18!
19!

!
P
!
41!
Figure Captions 1!
2!
Figure 1. Word clouds for „young man“ (left panel) and „dark lady“ (right panel) sonnets 3!
(https://www.jasondavies.com/wordcloud/) 4!
5!
Figure 2. Distribution of CML2 readability index with shaded area representing the „dark 6!
lady“ sonnets (127 – 154) 7!
8!
Figure 3. Linewise comparison of the CML2 readability index for sonnets 1 and 138 9!
10!
Figure 4. Eight CM easability indices (z scores) for sonnets 1 and 138 11!
12!
Figure 5. Distribution of five CM easability indices (z scores) for all sonnets (shaded area = 13!
„dark lady“ sonnets: 127 – 154) 14!
15!
Figure 6a-c. 16!
a. Distribution of two surprisal indices (means) for all sonnets (shaded area = „dark lady“ 17!
sonnets: 127 – 154) 18!
b. Linewise surprisal for sonnet 1 19!
c. Wordwise surprisal for sonnet 1 20!
21!
Figure 7. Distribution of SEANCE „emotion potential“ (sum of 20 component scores) for all 22!
sonnets (shaded area = „dark lady“ sonnets: 127 – 154; see text for details) 23!
24!
Figure 8. Distribution of three RID indices for all sonnets (shaded area = „dark lady“ sonnets: 25!
127 – 154; see text for details) 26!
27!
Figure 9. Distribution of five „mood“ indices for all sonnets (shaded area = „dark lady“ 28!
sonnets: 127 – 154; see text for details) 29!
30!
Figure 10. Distribution of Thematic Richness Index/TRI (shaded area = „dark lady“ sonnets: 31!
127 – 154; see text for details) 32!
33!
Figure 11. Distribution of Symbolic Imagery Index/SII (shaded area = „dark lady“ sonnets: 34!
127 – 154; see text for details) 35!
36!
Figure 12. Distribution of the Semantic Association Potential SAP (shaded area = „dark lady“ 37!
sonnets: 127 – 154; see text for details) 38!
39!
Figure 13. Percentage of new words (means with confidence intervals) as a function of 40!
position (line) for all sonnets. 41!
42!
43!
44!

!
P
!
42!
Figures 1!
1 2!
3!
2 4!
5!
3 6!
7!
8!
9!
10!
411!
-20
-10
!
"!
#!
$!
%!
"
#
$
%
&
'
(
)
*
"!
""
"#
"$
"%
+,--./0" +,--./0"$)

!
P
!
43!
1!
52!
-6
-4
-2
!
#
%
'
12332/454/6
76-/28/480749:;484/6
<,3=0>,-83./.-.++
?.@.3.-/42;0>,A.+4,-
B..:0>,A.+4,-
C.3D0>,A.+4,-
>,---.8/454/6
E.9:,32;4/6
+,--./0" +,--./0"$)

!
P
!
44!
1!
2!
3!
4!
5!
6!
7!
8!
9!
10!
11!

!
P
!
45!
6a 1!
2!
6b 3!
4!
5!
6c 6!
7!

!
P
!
46!
1!

!
P
!
47!
1!
2!
3!
7 4!
5!
6!
7!
8!
8 9!
10!
11!

!
P
!
48!
1!
9 2!
3!
4!
5!
10 6!
7!
8!
11 9!

!
P
!
49!
1!
2!
12 3!
4!
5!
6!
13 7!
8!
9!

!
P
!
50!
Appendix 1!
2!
Table A1. Various indices for sonnet parts (* = at least p<.05) 3!
Q1
Q2
Q3
C
octave
sestet
CML2
1.4
1.0
2.4
6.7*
1.2
3.8*
Narrativity
-1.6
-1.6
-1.4
-.98*
-1.6
-1.3*
Syntactic
simplicity
0.52
0.77*
0.70
0.83*
–
–
Referential
cohesion
0.65
0.69
0.66
0.77*
–
–
Deep
cohesion
-1.06
-0.73
-0.6
0.84*
-0.9
-0.12*
Emotion
potential
–
–
–
–
5.8
6.1*
Positive
verbs
–
–
–
–
-0.05
-0.01*
Affect for
friends &
family
–
–
–
–
0.4
0.45*
Surprisal
3.7
3.85
3.75
3.56*
3.77
3.69*
4!
Figure A1. Decision tree modeling results for model C3 (see text for details). 5!
!6!
7!