This doctoral thesis is part of a research project on the development of the cognitive compre-hension of film at Würzburg University that was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft) between 2013 and 2019 and awarded to Gerhild Nied-ing. That project examined children’s comprehension of narrative text and its development in illustrated versus non-illustrated formats. For this purpose, van Dijk and Kintsch’s (1983) tri-partite model was used, according to which text recipients form text surface and textbase rep-resentations and construct a situation model. In particular, predictions referring to the influ-ence of illustrations on these three levels of text representation were derived from the inte-grated model of text and picture comprehension (ITPC; Schnotz, 2014), which holds that text-picture units are processed on both text-based (descriptive) and picture-based (depictive) paths. Accordingly, illustrations support the construction of a situation model. Moreover, in line with the embodied cognition account (e.g., Barsalou, 1999), it was assumed that the situa-tion model is grounded in perception and action; text recipients mentally simulate the situation addressed in the text through their neural systems related to perception (perceptual simulation) and action (motor resonance). Therefore, the thesis also examines whether perceptual simula-tion takes place during story reception, whether it improves the comprehension of illustrated stories, and whether motor resonance is related to the comprehension of text accompanied by dynamic illustrations. Finally, predictions concerning the development of comprehending illus-trated text were made in line with Springer’s (2001) hypotheses according to which younger children, compared with older children and adults, focus more on illustrations during text comprehension (perceptual boundedness) and use illustrations for the development of cogni-tive skills (perceptual support). The first research question sought to validate the tripartite model in the context of children’s comprehension of narrative text, so Hypothesis 1 predicted that children yield representations of the text surface, the textbase, and the situation model during text reception. The second research question comprised the assumptions regarding the impact of illustrations on text comprehension. Accordingly, it was expected that illustrations improve the situation model (Hypothesis 2a), especially when they are processed before their corresponding text passages (Hypothesis 2b). Both hypotheses were derived from the ITPC and the assumption that per-ceptual simulation supports the situation model. It was further predicted that dynamic illustra-tions evoke more accurate situation models than static ones (Hypothesis 2c); this followed from the assumption that motor resonance supports the situation model. In line with the ITPC, it was assumed that illustrations impair the textbase (Hypothesis 2d), especially when they are presented after their corresponding text passages (Hypothesis 2e). In accordance with earlier results, it was posited that illustrations have a beneficial effect for the text surface (Hypothesis 2f). The third research question addressed the embodied approach to the situation model. Here, it was assumed that perceptual simulation takes place during text reception (Hypothesis 3a) and that it is more pronounced in illustrated than in non-illustrated text (Hypothesis 3b); the latter hypothesis was related to a necessary premise of the assumption that perceptual sim-ulation improves the comprehension of illustrated text. The fourth research question was relat-ed to perceptual boundedness and perceptual support and predicted age-related differences; younger children were expected to benefit more from illustrations regarding the situation model (Hypothesis 4a) and to simulate vertical object movements in a more pronounced fash-ion (Hypothesis 4b) than older children. In addition, Hypothesis 4c held that perceptual simu-lation is more pronounced in younger children particularly when illustrations are present. Three experiments were conducted to investigate these hypotheses. Experiment 1 (Seger, Wannagat, & Nieding, submitted).compared the tripartite representations of written text without illustrations, with illustrations presented first, and with illustrations presented after their corresponding sentences. Students between 7 and 13 years old (N = 146) took part. Ex-periment 2 (Seger, Wannagat, & Nieding, 2019) investigated the tripartite representations of auditory text, audiovisual text with static illustrations, and audiovisual text with dynamic il-lustrations among children in the same age range (N = 108). In both experiments, a sentence recognition method similar to that introduced by Schmalhofer and Glavanov (1986) was em-ployed. This method enables the simultaneous measurement of all three text representations. Experiment 3 (Seger, Hauf, & Nieding, 2020) determined the perceptual simulation of vertical object movements during the reception of auditory and audiovisual narrative text among chil-dren between 5 and 11 years old and among adults (N = 190). For this experiment, a picture verification task based on Stanfield and Zwaan’s (2001) paradigm and adapted from Hauf (2016) was used. The first two experiments confirmed Hypothesis 1, indicating that the tripartite model is appli-cable to the comprehension of auditory and written narrative text among children. A benefi-cial effect of illustrations to the situation model was observed when they were presented syn-chronously with auditory text (Hypotheses 2a), but not when presented asynchronously with written text (Hypothesis 2b), so the ITPC is partly supported on this point. Hypothesis 2c was rejected, indicating that motor resonance does not make an additional contribution to the comprehension of narrative text with dynamic illustrations. Regarding the textbase, a general negative effect of illustrations was not observed (Hypothesis 2d), but a specific negative effect of illustrations that follow their corresponding text passages was seen (Hypothesis 2e); the latter result is also in line with the ITPC. The text surface (Hypothesis 2f) appears to benefit from illustrations in auditory but not written text. The results obtained in Experiment 3 sug-gest that children and adults perceptually simulate vertical object movements (Hypothesis 3a), but there appears to be no difference between auditory and audiovisual text (Hypothesis 3b), so there is no support for a functional relationship between perceptual simulation and the situ-ation model in illustrated text. Hypotheses 4a–4c were investigated in all three experiments and did not receive support in any of them, which indicates that representations of illustrated and non-illustrated narrative text remain stable within the age range examined here.
