Article

Can narrative films go interactive?

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

New interactive media works based upon audio-visual material often result in distraction rather than sustained engagement. Contrary to postmodern textual and cognitive presumptions, this study uses dual coding theory, cognitive load theory, and constructivist narrative film theory, to claim that distraction results from cognitive and behavioural multi-tasking which lead to split attention problems that cannot be cognitively handled. Focus is upon split attention resulting from the non-critical use of split screens, from decentred, non-cohering audio-visual and multi-narrative formations, and from interaction. The analysis of several new media works, existing tools and models, particularly those pertaining to narrative- oriented `interactive films', instantiates these claims. For narrative interactive audio-visual texts to sustain deep, wide-ranged engagement, multi-tasking split attention problems inhering in computer-based works have to be managed, and - most importantly - made to enhance rather than reduce engagement. This article outlines some major problems and offers viable solutions.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... Common criticisms stem from the interactive film positioned in a liminal space between film and game [23]. As a lean-back medium for entertainment, as opposed to lean-forward user-controlled media like games [26], some see an inherent tension in interactive film between active decision making and the flow of narrative immersion [3,35]. ...
... This question is explicitly posed by critics when they suggest that films like CtrlMovie's Late Shift are more of a big-screen video game than a movie [44]. Some have argued that conscious control breaks narrative immersion in a film when, unlike in a game, ludic immersion is not part of the experience [3]. From an HCI perspective, Pike et al. reported a study of a BCI film called The Disadvantages of Time Travel, which used blinking to switch between an internalised and external view of the narrative, and measures of attention and meditation derived from a consumer-grade EEG sensor to blend video layers [34]. ...
Conference Paper
Introducing interactivity to films has proven a longstanding and difficult challenge due to their narrative-driven, linear and theatre-based nature. Previous research has suggested that Brain-Computer Interfaces (BCI) may be a promising approach but also revealed a tension between being immersed in the film and thinking about control. We report a performance-led and in-the-wild study of a BCI film called The MOMENT covering its design rationale and how it was experienced by the public as controllers, non-controllers and repeat viewers. Our findings suggest that BCI movies should be designed to be credibly controllable, generate personal versions, be watchable as linear films, encourage repeat viewing and fit the medium of cinema. They also reveal how viewers appreciated the sense of editing their own personal cuts, suggesting a new stance on introducing interactivity into lean-back media in which filmmakers release editorial control to users to make their own versions.
... We speculate that the spatial and temporal synchrony of looking (visual), listening (audition), and touching (haptics) may be the sweet spot that garners the young child's attention to e-text in ways that support early literacy experience and learning. Some argue (e.g., Ben-Shaul, 2004), however, that even as touching may re-kindle attention, it can lead to split attention between mind and hand, i.e., between the cognitive interaction with text that looking and listening support and the playful discovery of what the screen page can do—a tendency observed in prior research (Kegel et al., 2009). Yet, as some studies of interactive media suggest (Ben-Shaul, 2004; Mayer & Moreno, 2003), if behavioral actions remain simple to physically engage the user through sound, movement and tactile activities (touching the screen, clapping hands, varying voice pitch, or moving around to interact with the text) and if physical actions offer a good cognitive and sensual " fit " with an unfolding narrative, then they may enhance engagement and deepen experience. ...
... Some argue (e.g., Ben-Shaul, 2004), however, that even as touching may re-kindle attention, it can lead to split attention between mind and hand, i.e., between the cognitive interaction with text that looking and listening support and the playful discovery of what the screen page can do—a tendency observed in prior research (Kegel et al., 2009). Yet, as some studies of interactive media suggest (Ben-Shaul, 2004; Mayer & Moreno, 2003), if behavioral actions remain simple to physically engage the user through sound, movement and tactile activities (touching the screen, clapping hands, varying voice pitch, or moving around to interact with the text) and if physical actions offer a good cognitive and sensual " fit " with an unfolding narrative, then they may enhance engagement and deepen experience. In short, multi-sensory behaviors of engagement do not conduce to multi-tasking split attention and distraction if there is a goodness of fit. ...
Article
Full-text available
Differences between digital devices on children’s engagement with e-books are examined. The sample included 24 typical 4-year olds enrolled in Head Start. Over a 1-month period, video captures of children’s multi-sensory behaviors during shared reading at a tabletop touchscreen computer and teacher-facilitated book browsing with iPads and iPods were obtained. Data were coded on each child at 1-min intervals, examining the simultaneity of behaviors present, then aggregated to determine frequencies of each behavior by device and format. Differences between media devices on median percent of observation time were evaluated. Looking, touching, moving, and gesturing behaviors were significantly different among different devices. Large effect sizes indicated considerable variability attributable to the device. Mobiles support more looking and touching but less moving and gesturing than the tabletop touchscreen; none of the devices favored listening over another. Given the role of haptic perception in digital reading experience, access to mobiles may favor behaviors that support literacy motivation, sense of control, and interaction.
... More problematic, however, is the split attention of the viewer/user between what he/she cognitively constructs from what's going on in front of him/her, and his/her constant awareness to what may potentially lie at stake in options made available by behaviorally changing the course of events. . . . In all of these experiences the behavioral option is restlessly often activated, resulting in the user/viewer being neither here nor there (Ben-Shaul, 2004, p. 157). The experiential position of split attention is by definition irreconcilable with the deep, immersive state of reading that we experience with, for instance, riveting thrillers and suspenseful detective stories. ...
Article
Full-text available
Reading is a multi-sensory activity, entailing perceptual, cognitive and motor interactions with whatever is being read. With digital technology, reading manifests itself as being extensively multi-sensory – both in more explicit and more complex ways than ever before. In different ways from traditional reading technologies such as the codex, digital technology illustrates how the act of reading is intimately connected with and intricately dependent on the fact that we are both body and mind – a fact carrying important implications for even such an apparently intellectual activity as reading, whether recreational, educational or occupational. This article addresses some important and hitherto neglected issues concerning digital reading, with special emphasis on the vital role of our bodies, and in particular our fingers and hands, for the immersive fiction reading experience.
Chapter
Interactive cinema appears as an enticing blend of cinema and games. However, active control has been argued to disrupt the appeal of the cinematic experience, and so-called lean-back interactions have been proposed. Such experiences seek to increase narrative immersion without posing demands to the user, or them necessarily being conscious of the control. This paper seeks to investigate how such mechanism can be designed by leveraging attention-measuring Brain-Computer Interface to cater to individuals’ interests and increase narrative engagement for interactive films. This is done by skipping seamlessly ahead to the next exciting plot point when detecting drops in viewer’s attention, thereby tailoring parts of the film while keeping the overarching story. A short prototype film was produced with virtual production techniques and evaluated in a between group experimental design (n = 24). Participants in the experimental condition watched an interactive version of the film, which contained skippable segments based on viewers’ level of attention, while those in the control condition watched a non-interactive version with an edit matching one from the experimental group. The results showed no significant difference in narrative engagement; however, the experimental group showed a significantly higher overall attention than the control group. This suggests that attention-based skippable segments could have some impact on viewer engagement, and it may be beneficial for creating personalized edits for viewers, although further investigations are needed.
Article
The term “interactive television” initially referred to the idea that television might facilitate two-way communication between viewers. In the 1980s, this term was also used to refer to television’s ability to provide access to interactive menus and electronic services, and in the 1990s, it was used to refer to the idea that viewers could potentially alter the content of television programs. This new definition has now become possible due to the rise of streaming and on-demand services, and the sudden resurgence of interest in interactive television demands a revaluation and reassessment of its complicated history. While some critics celebrate interactivity as empowering and liberating, for example, others argue that it enables ever more virulent forms of control and manipulation. This article will examine the history of this controversial term and its most recent manifestations in order to address the significance of its meaning and ideological function.
Chapter
With the release of the film Black Mirror: Bandersnatch Netflix entered the area of interactive streamed narratives. We performed a qualitative analysis with 169 Netflix subscribers that had watched the episode. The key findings show (1) participants are initially engaged because of curiosity and the novelty value, and desire to explore the narrative regardless of satisfaction, (2) perceived agency is limited due to arbitrary choices and the lack of meaningful consequences, (3) the overall experience is satisfactory but adaptions are desirable in future design to make full use of the potential of the format.
Article
Full-text available
There is a potential inconsistency between the experience of story and the process of interaction. Many interactive narratives ask the interactor to switch between an immersive state of immediacy and a hypermediated awareness of process. The paper examines the interactive CD- Rom Ceremony of Innocence and identifies two design strategies that are used to suture any potential disjuncture. One is the saturation of narrativity throughout the entire work. Analogies are drawn to both classic Expressionism and a more widespread expressivity in cinema. The other is the subversion of the interface itself to reflect narrative concerns. Narrativity is thus situated both globally and at the heart of the interactive experience.
Article
Full-text available
Dual coding theory (DCT) explains human behavior and experience in terms of dynamic associative processes that operate on a rich network of modality-specific verbal and nonverbal (or imagery) representations. We first describe the underlying premises of the theory and then show how the basic DCT mechanisms can be used to model diverse educational phenomena. The research demonstrates that concreteness, imagery, and verbal associative processes play major roles in various educational domains: the representation and comprehension of knowledge, learning and memory of school material, effective instruction, individual differences, achievement motivation and test anxiety, and the learning of motor skills. DCT also has important implications for the science and practice of educational psychology — specifically, for educational research and teacher education. We show not only that DCT provides a unified explanation for diverse topics in education, but also that its mechanistic framework accommodates theories cast in terms of strategies and other high-level psychological processes. Although much additional research needs to be done, the concrete models that DCT offers for the behavior and experience of students, teachers, and educational psychologists further our understanding of educational phenomena and strengthen related pedagogical practices.
Article
Full-text available
We present a novel approach to documentary storytelling that celebrates electronic narrative as a process in which the author(s), a networked presentation system, and the audience actively collaborate in the co-construction of meaning. A spreading-activation network is used to select relevant story elements from a multimedia database and dynamically conjoin them into an appealing, coherent narrative presentation. The flow of positive or negative “energies” through associative keyword links determines which story materials are presented as especially relevant “next steps” and which ones recede into the background, out of sight. The associative nature of this navigation serves to enhance meaning while preserving narrative continuity. This approach is well-suited for the telling of stories that—because of their complexity, breadth, or bulk—are best communicated through variable-presentation systems. Connected to the narrative engine through rich feedback loops and intuitively understandable interfaces, the audience becomes an active partner in the shaping and presentation of story.
Article
Full-text available
Cognitive load theory suggests that effective instructional material facilitates learning by directing cognitive resources toward activities that are relevant to learning rather than toward preliminaries to learning. One example of ineffective instruction occurs if learners unnecessarily are required to mentally integrate disparate sources of mutually referring information such as separate text and diagrams. Such split-source infonnation may generate a heavy cognitive load, because material must be mentally integrated before learning can commence. This article reports findings from six experiments testing the consequences of split-source and integrated information using electrical engineering and biology instructional materials. Experiment 1 was designed to compare conventional instructions with integrated instructions over a period of several months in an industrial training setting. The materials chosen were unintelligible without mental integration. Results favored integrated instructions throughout the 3-month study. Experiment 2 was designed to investigate the possible differences between conventional and integrated instructions in areas in which it was not essential for sources of information to be integrated to be understood. The results suggest that integrated instructions were no better than split-source infonnation in such areas. Experiments 3, 4, and 5 indicate that the introduction of seemingly useful but nonessential explanatory material (e.g., a commentary on a diagram) could have deleterious effects even when presented in integrated format. Experiment 6 found that the need for physical integration was restored if the material was organized in such a manner that individual units could not be understood alone. In light of these results and previous findings, suggestions are made for cognitively guided instructional packages.
Article
Two experiments are reported in which subjects performed two forced-paced serial reaction time tasks separately and together at their maximum sustainable rates of information processing. Experiment 1 investigated the effects on the relationship between single- and dual-task performance of using tasks with the same or different input and output modality characteristics; an additional condition tested the effects on this relationship of using tasks with higher S–R compatibility. Experiment 2 investigated the effects on the relationship between single- and dual-task performance of varying information load (number of S–R alternatives). No significant differences were found in subjects' capacities to process information in single- and dual-task conditions. This finding was unaffected by: (a) the absolute information levels of the tasks, (b) whether inputs and/or outputs involved the same or different modalities, or (c) the level of S–R compatibility. The data from both experiments provide strong support for the single-channel hypothesis.
Article
Computer-based multimedia learning environments — consisting of pictures (such as animation) and words (such as narration) — offer a potentially powerful venue for improving student understanding. How can we use words and pictures to help people understand how scientific systems work, such as how a lightning storm develops, how the human respiratory system operates, or how a bicycle tire pump works? This paper presents a cognitive theory of multimedia learning which draws on dual coding theory, cognitive load theory, and constructivist learning theory. Based on the theory, principles of instructional design for fostering multimedia learning are derived and tested. The multiple representation principle states that it is better to present an explanation in words and pictures than solely in words. The contiguity principle is that it is better to present corresponding words and pictures simultaneously rather than separately when giving a multimedia explanation. The coherence principle is that multimedia explanations are better understood when they include few rather than many extraneous words and sounds. The modality principle is that it is better to present words as auditory narration than as visual on-screen text. The redundancy principle is that it is better to present animation and narration than to present animation, narration, and on-screen text. By beginning with a cognitive theory of how learners process multimedia information, we have been able to conduct focused research that yields some preliminary principles of instructional design for multimedia messages.
Article
Why do people often find that performing two tasks at once is harder than performing one task at a time? Three mechanisms of task interference that might answer that question were investigated: resource competition, confusions, and incompatible task proximity between processing stages. The subjects performed dual-axis compensatory tracking with error displays that were either integrated or separated, with axis controls that either were integrated into one stick or remained separate, and with control dynamics on the two axes that were either the same or different. Tracking error increased and control activity decreased as a function of the combined difficulty of the two control dynamics. Integrated displays and integrated controls both led to increased confusions between tracking axes although error was not reliably affected. Significantly, performance was also affected by whether the integrality of displays matched that of controls. These results suggest that resource competition, confusions, and compatibility of proximity play distinct roles in dual-axis tracking performance.
Article
People often have trouble performing 2 relatively simple tasks concurrently. The causes of this interference and its implications for the nature of attentional limitations have been controversial for 40 years, but recent experimental findings are beginning to provide some answers. Studies of the psychological refractory period effect indicate a stubborn bottleneck encompassing the process of choosing actions and probably memory retrieval generally, together with certain other cognitive operations. Other limitations associated with task preparation, sensory-perceptual processes, and timing can generate additional and distinct forms of interference. These conclusions challenge widely accepted ideas about attentional resources and probe reaction time methodologies. They also suggest new ways of thinking about continuous dual-task performance, effects of extraneous stimulation (e.g., stop signals), and automaticity. Implications for higher mental processes are discussed.
Article
Most existing research on virtual reality concerns issues close to the interface, primarily how to present an underlying simulated world in a convincing fashion. However, for virtual reality to achieve its promise as a rich and popular artistic form, as have the novel, cinema, and television, we believe it will be necessary to explore well beyond the interface, to those issues of content and style that have made traditional media so powerful. We present a case for the importance of this research, then outline several topics we believe are central to the inquiry: developing computational theories for cognitive/emotional agents, presentation style, and drama. 1 The Fantasy Machine A writer of popular science articles visits a laboratory to learn about the new "virtual reality" research. The writer is ushered into the demo room and warned to move slowly until adjusted to the system. The goggles slip on, then the dataglove, and a switch is thrown. As a world appears, the writer's head tur...