Chapter

Experiences of Multimodal Teaching Through a Serious Game: Meanings, Practices and Discourses

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The aim of this chapter is to report on school teachers’ perceptions and approaches to multimodality using a serious game. STEAM is a game designed for helping school teachers to gain awareness of how multimodality may be enacted in the classroom for enhancing the student learning experience. The game embraces the notion of multimodal teaching and learning, as a way to present multiple representations of content such as text, images, video, audio and pervasive media, by augmenting modes with tools, teaching strategies and locations as means to create ideational, interpersonal and textual meanings. A questionnaire was employed to school teachers (n = 54) for understanding how multimodality was experienced through using the serious game as (1) stipulating diversity and increasing knowledge retention, (2) developing senses for attaining deeper understanding of the subject topic, (3) involving students into learning design and (4) supporting student’s autonomy and self-direction. The findings revealed an explicit connection between theory and practice as experienced through the game’s semiotic domain whilst contemplating on attempts to transcend experiences of in-game multimodality to lived classrooms.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Such tools and applications were synchronous and asynchronous messaging, user forums, remote screen sharing, and games. Related concepts of relevance to learning from interactive multimedia are the notions of 'modalities' such as seeing, hearing, feeling, and tasting integrated into multimedia software-like games (e.g., Gee [35]) and 'multimodality' drawing on the process of creating meaning through connecting and combining teaching modes, multimedia, and technology (Lameras and Papageorgiou [36]). Such multimodal resources were coined as 'learning objects' (e.g., Conole [37]) representing simple, interoperable digital learning assets that are predisposed to reuse in multiple learning contexts. ...
Article
This exploratory review attempted to gather evidence from the literature by shedding light on the emerging phenomenon of conceptualising the impact of artificial intelligence in education. The review utilised the PRISMA framework to review the analysis and synthesis process encompassing the search, screening, coding, and data analysis strategy of 141 items included in the corpus. Key findings extracted from the review incorporate a taxonomy of artificial intelligence applications with associated teaching and learning practice and a framework for helping teachers to develop and self-reflect on the skills and capabilities envisioned for employing artificial intelligence in education. Implications for ethical use and a set of propositions for enacting teaching and learning using artificial intelligence are demarcated. The findings of this review contribute to developing a better understanding of how artificial intelligence may enhance teachers’ roles as catalysts in designing, visualising, and orchestrating AI-enabled teaching and learning, and this will, in turn, help to proliferate AI-systems that render computational representations based on meaningful data-driven inferences of the pedagogy, domain, and learner models.
Article
This manuscript introduces I-Three Learning Model (ITLM) intervention to build competency among scholastically backward children by facilitating easy input, processing and output of information. Child receives information through sensory path ways, learning ability is the capacity of the children to collect, process, retain and retrieve information. Children are unique in mental maturity and learning ability. Reasoning is influenced by the auditory, visual, kinaesthetic and tactile inputs. Competency of children with poor social and emotional skills, learning adjustment and academic performance can be improved by enriching their abilities connected to attention, self-learning, logical thinking, reasoning, adjustment, confidence, comprehension and problem solving. This manuscript is both descriptive and exploratory in nature. On the basis of standard Psychological Assessment, a child studying in the eight standard aged 14 years is identified to be poor in social and emotional skills, learning adjustment and academic performance. This case study is carried to derive the findings of these objectives and establishes that ITLM intervention has certainly improve capacity of receiving, processing and retrieving information in the children and recommends for the usage of model for building competency of scholastically backward students.
Conference Paper
Full-text available
STEAM is a serious game developed as a medium for helping teachers to experience multimodality for teaching and learning. A design-based paradigm is adopted to elucidate how in-game design elements coupled with learning may visualize in-game multimodal representations. Multimodality is experienced as a process of creating meaning though connecting and combining different modes, semiotic resources and semiotic ensembles. In this paper, we present the design and usability evaluation of the game. The usability study was conducted with (n=32) school teachers completing an online survey after playtesting the game for identifying, capturing and fine-tuning in-game usability aspects. The findings indicated that the game’s core mechanics, the ingame dialogues and card-game, represent and visualise the content and process of multimodal in-game ensembles whilst the development of in-game feedback and progress indicators was perceived as having the capacity to guide understandings on in-game multimodality and to track in-game progress.
Article
Full-text available
p> In today’s increasingly fast-moving digital world, learners are immersed in multimodal online communication environments in their daily life, through Facebook, Twitter, Instagram, Snapchat and others. This requires educators to reflect the environment in which these learners live, and thus design instructional practices from a multimodal perspective. Multimodality offers new opportunities for digital learners to express themselves, analyze problems and make meaning in multimodal ways as they interpret knowledge differently according to their various educational needs (Kalantzis & Cope, 2015). In this paper we will discuss the significance of integrating multimodality in e-Learning contexts to make meaning and improve learning. The paper will also present a case study of an online course from the College of Education at the University of Illinois Urbana Champaign to show how multimodality works in practice to cater to learner differences by offering a range of activity options and modes of meaning. We will also examine learners’ perceptions of adopting such an approach in the online course. We used survey techniques for data collection and quantitative and qualitative methods for data analysis. Results revealed illuminating insights about the importance of multimodality approach to increase learning potential for digital learners and provided suggestions for future iterations. </p
Article
Full-text available
The integration of educational video games in educational settings in general, and e-learning systems in particular, can be challenging for educators. We propose a framework that aims to facilitate educators' participation in the creation and modification of courses that use educational games. Our approach addresses problems identified by previous experiences with games in educational settings, including assessment of learning outcomes and student tracking. Our framework has been implemented following an application model that takes advantage of pre-existing systems: the educational video game framework and the Learning Activity Management System (LAMS). This approach has been put into practice in a case study carried out in a primary school, covering from the design of the learning experience to the development of the educational games and the deployment and evaluation with students, involving the educators actively and gathering their perceptions. The first impressions expressed by the educators support the potential of the framework in terms of the students' assessment and the personalization of the lesson. Although educators pointed out the difficulty of creating games from scratch, they appreciated the easiness of introducing existing games in their courses and adapting them to their specific educational settings. © International Forum of Educational Technology & Society (IFETS).
Article
Full-text available
The purpose of this article is to propose a blending with purpose multimodal conceptual model for designing and developing blended learning courses and programs. A blended learning model is presented that suggests teachers design instruction to meet the needs of a variety of learners. Specifically, Blending with Purpose: The Multimodal Model recognizes that because learners represent different generations, different personality types, and different learning styles, teachers and instructional designers should seek to use multiple approaches including face-to-face methods and online technologies that meet the needs of a wide spectrum of students. A major benefit of multiple modalities is that they allow students to experience learning in ways in which they are most comfortable while also challenging them to experience and learn in other ways as well. Critical to this model is the concept that academic program and course goals and objectives drive the pedagogical approaches and technologies used. Issues related to definitions of blended learning, how teachers and students use technology, generational characteristics among student populations, and learning styles are examined
Article
Full-text available
Frequently writing is now no longer the central mode of representation in learning materials—textbooks, Web-based resources, teacher-produced materials. Still (as well as moving) images are increasingly prominent as carriers of meaning. Uses and forms of writing have undergone profound changes over the last decades, which calls for a social, pedagogical, and semiotic explanation. Two trends mark that history. The digital media, rather than the (text) book, are more and more the site of appearance and distribution of learning resources, and writing is being displaced by image as the central mode for representation. This poses sharp questions about present and future roles and forms of writing. For text, design and principles of composition move into the foreground. Here we sketch a social semiotic account that aims to elucidate such principles and permits consideration of their epistemological as well as social/pedagogic significance. Linking representation with social factors, we put forward terms to explore two issues: the principles underlying the design of multimodal ensembles and the potential epistemological and pedagogic effects of multimodal designs. Our investigation is set within a research project with a corpus of learning resources for secondary school in Science, Mathematics, and English from the 1930s, the 1980s, and from the first decade of the 21st century, as well as digitally represented and online learning resources from the year 2000 onward.
Article
Full-text available
This paper examines the changing landscape of literacy teaching and learning, revisiting the case for a “pedagogy of multiliteracies” first put by the New London Group in 1996. It describes the dramatically changing social and technological contexts of communication and learning, develops a language with which to talk about representation and communication in educational contexts, and addresses the question of what constitutes appropriate literacy pedagogy for our times.
Article
Full-text available
Enhancing the offer for entrepreneurship education is an important challenge for the nowadays knowledge societies. The eSG project is addressing this issue by analysing the added value that could be contributed by employing serious games (SGs) as a tool for allowing students in particular technology students - to become familiar, mainly through practice, with basic concepts of entrepreneurship and company management. This paper presents the main requirements for the course and SGs obtained by surveying literature, entrepreneurs, students and teachers. We represented the requirements in a table template keeping into account usability, pedagogy, the entrepreneurship skills expressed by state of the art models and three major axes for entrepreneurship education at universities. These table descriptors were then used to assess validity of SGs and choose an appropriate mix for the courses. We have also defined a set of metrics to evaluate the advancement of students during the course. Based on these tools and knowledge, the next steps of the project will involve extensive user testing in the actual courses that are being performed in Genoa, Delft and Barcelona.
Article
Full-text available
The characteristics of contemporary societies are increasingly theorized as global, fluid, and networked. These conditions underpin the emerging knowledge economy as it is shaped by the societal and technological forces of late capitalism. These shifts and developments have significantly affected the communicational landscape of the 21st century. A key aspect of this is the reconfiguration of the representational and communicational resources of image, action, sound, and so on in new multimodal ensembles. The terrain of communication is changing in profound ways and extends to schools and ubiquitous elements of everyday life, even if these changes are occurring to different degrees and at uneven rates. It is against this backdrop that this critical review explores school multimodality and literacy and asks what these changes mean for being literate in this new landscape of the 21st century. The two key arguments in this article are that it is not possible to think about literacy solely as a linguistic accomplishment and that the time for the habitual conjunction of language, print literacy, and learning is over. This review, organized in three parts, does not provide an exhaustive overview of multimodal literacies in and beyond classrooms. Instead, it sets out to highlight key definitions in an expanded approach to new literacies, then to link these to emergent studies of schooling and classroom practice. The first part outlines the new conditions for literacy and the ways in which this is conceptualized in the current research literature. In particular, it introduces three perspectives: New Literacies Studies, multiliteracies, and multimodality. Contemporary conceptualizations of literacy in the school classroom are explored in the second part of the chapter. This discussion is organized around themes that are central to multimodality and multiliteracies. These include multimodal perspectives on pedagogy, design, decisions about connecting with the literacy worldsof students, and the ways in which representations shape curriculum knowledge and learning. Each of these themes is discussed in turn, drawing on a range of examples of multimodal research. The third and final part of the article discusses future directions for multiple literacies, curriculum policy, and schooling.
Article
Full-text available
The rich illustrations of practice encourage both discussion of practical challenges and dilemmas and conceptualization beyond the specific cases. Historically, issues in New Literacy Studies, multimodality, new literacies, and multiliteracies have primarily been addressed theoretically, promoting a shift in educators’ thinking about what constitutes literacy teaching and learning in a world no longer bounded by print text only. Such theory is necessary (and beneficial for re-thinking practices). What Multimodal Composing in Classrooms contributes to this scholarship are the voices of teachers and students talking about changing practices in real classrooms.
Article
Full-text available
Since Windschitl first outlined a research agenda for the World Wide Web and classroom research, significant shifts have occurred in the nature of the Web and the conceptualization of classrooms. Such shifts have affected constructs of learning and instruction, and paths for future research. This article discusses the characteristics of Web 2.0 that differentiate it from the Web of the 1990s, describes the contextual conditions in which students use the Web today, and examines how Web 2.0's unique capabilities and youth's proclivities in using it influence learning and teaching. Two important themes, "learner participation and creativity and online identity formation," emerged from this analysis and support a new wave of research questions. A stronger research focus on students' everyday use of Web 2.0 technologies and their learning with Web 2.0 both in and outside of classrooms is needed. Finally, insights on how educational scholarship might be transformed with Web 2.0 in light of these themes are discussed.
Article
Full-text available
In today's primary classrooms, the definition of “text” has expanded to include multiple modes of representation, with combined elements of print, visual images, and design. Emergent research on literacy highlights the imaginative, interpretive, nonlinear, interactive, dynamic, visual, and mobile features of communication (Coiro, Knobel, Lankshear, & Leu, 2008). Although these interactive features of text are evident in many educational technologies, such as computers or SMART boards, they also are present in recently published children's books. This article highlights the theories and practices of multimodality in K-3 classrooms.
Article
Full-text available
Opinions about the psychological correlates of multimedia computer-supported instructional tools were analyzed by means of a questionnaire concerning the motivational and emotional aspects of multimedia learning, the strategies to be followed during the learning process, the mental abilities and the style of thinking required, the cognitive benefits and outcomes. The questionnaire was distributed to 272 teachers working in kindergarten, primary, and secondary schools. Gender and previous experience with multimedia, as well as the disciplinary fields taught by secondary school teachers, were taken into account. Respondents identified a large number of non-trivial instructional opportunities from multimedia and showed well-defined and internally articulated beliefs. No significant gender effect was found. Differences of school level, disciplinary field, and direct experience with multimedia tools affected a part of the teachers’ representation. Implications for instruction were discussed.
Book
Cambridge Core - Semantics and Pragmatics - Making Sense - by Bill Cope
Conference Paper
The aim of this paper is to report on teachers' experiences of, and approaches to, multimodality in teaching and learning. A small-scale online survey with closed and semi-structured questions has been deployed to school and university teachers (n=68) for eliciting their experiences in multimodal teaching and learning. Thematic analysis has been adopted as the overarching methodology for reporting patterns in the data from the survey. The results from the analysis showed that experiences of multimodality are discerned as: (1) imparting information, (2) enacting collaborative learning and (3) preparing students for exploring concepts. The process of meaning making is exemplified through a developmental progression from more teacher-directed modes through oral, written and visual representations to more student-centered through gestural representations as means of connecting and combining different modes triggered via visual communication, collaboration and exploration.
Article
Supported by ever-evolving digital tools and online spaces, we argue that multiliteracies can be used to close the gap between teacher-directed, individual, and assessment-driven learning, and authentic, shared, and purpose-driven learning. This is particularly evident through multimodal composition and collaboration in primary classrooms. Over two decades ago, the New London Group argued that all meaning-making is multimodal. By representing their knowledge through multiple modes and for local and global audiences, students can express their identity, exercise agency, and foster a sense of authoring through multimodal production. Copyright © The College of Education and Human Ecology, The Ohio State University.
Chapter
This chapter explores recent encouragement to cultivate in students a sensitivity towards the “multimodal” nature of human communication. We consider what this means for educational practice and, in particular, how such an imperative might be addressed with digital tools. In particular we report a field study of secondary school students creating narrated photographs to characterise their local community and to construct sequences in the style of graphic novels. Although students were well engaged by this activity, many were hesitant in using their voice expressively. This variation in voicing confidence reminds us that education creates few opportunities for students to think about their speech in instrumental terms. Yet, we did see in some students a willingness and ability to do this. Adapting speech-for-purpose is a fundamental social skill. Thus, there is a need to take oracy more seriously and to see digital tools as one opening to do so in a practical way. Likewise, this project revealed disparities in students’ confidence with visual expression: differences that implied a lack of experience in seeing the semiotic potential of the image. These observations suggest that educators should help students read (and compose) in these modalities as carefully as they help students to acquire more familiar text literacy.
Article
Mode means the mode of human sensory organs with the external environment, the interaction with only one sensory organ is called single mode and the simultaneous interaction with more sensory organs are called multiple modes. A multimodal online English teaching system is designed, and is applied in the online English teaching of architecture major, and the students are divided into experimental group and control group. Conventional teaching is adopted in the conventional group, while multi-mode online systematic English learning is adopted for the experimental group. According to the employment statistics, it is shown that the experiment group presents some advantages in employment, relieving the employment pressure. The multi-mode learning has a good application effect in the English teaching of science and engineering, and the multi-mode online teaching system designed can be applied for the online English teaching.
Article
Static, animated, and interactive visualizations are frequently used in electronic learning environments. In this article, we provide a brief review of research on visuospatial cognition relevant to designing e-learning tools that use these displays. In the first section, we discuss the possible cognitive benefits of visualizations consider used in e-learning environments. In the second section consider cognitive constraints on the use of visualizations and design guidelines intended to reduce impact of these cognitive constraints. Finally, we consider how individual differences interact with learning from visualizations and how the use of visualizations might be altered for students of different abilities.
Article
Visual intelligence is a key element in the thought processes of the most capable and creative among individuals, and this intelligence is closely related to analogical thinking, a learner's ability to make connections between prior knowledge and newly presented information. This article describes an approach to teaching scientific inquiry at Eastside Elementary in Chattanooga, Tennessee, where each inquiry lesson begins with a drawing or other visual literacy connection. Drawing, by its very nature, encourages visualization, which in turn positions the learner to think metaphorically. What is novel about this approach is that the children are on a perpetual search for patterns. The lessons--drawn from a study carried out by the first author of this discussion (Cowan 2001) and from adaptations of lessons from "The Private Eye" (Ruef 1992)--lead children to recognize, describe, and extend simple repetitive patterns. Children collect data through their senses, paying particular attention to what they can see. As a part of each lesson, children look closely at objects or structures, search for repeating patterns, and guess or hypothesize about the nature of the objects, based on their prior knowledge and the observable patterns. This article also discusses how analogies form a foundation for scientific inquiry.
Article
A considerable amount of research has examined trust since our 1995 publication. We revisit some of the critical issues that we addressed and provide clarifications and extensions of the topics of levels of analysis, time, control systems, reciprocity, and measurement. We also recognize recent research in new areas of trust, such as affect, emotion, violation and repair, distrust, international and cross-cultural issues, and context-specific models, and we identify promising avenues for future research.
Article
Expert video game players often outperform non-players on measures of basic attention and performance. Such differences might result from exposure to video games or they might reflect other group differences between those people who do or do not play video games. Recent research has suggested a causal relationship between playing action video games and improvements in a variety of visual and attentional skills (e.g., [Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423, 534-537]). The current research sought to replicate and extend these results by examining both expert/non-gamer differences and the effects of video game playing on tasks tapping a wider range of cognitive abilities, including attention, memory, and executive control. Non-gamers played 20+ h of an action video game, a puzzle game, or a real-time strategy game. Expert gamers and non-gamers differed on a number of basic cognitive skills: experts could track objects moving at greater speeds, better detected changes to objects stored in visual short-term memory, switched more quickly from one task to another, and mentally rotated objects more efficiently. Strikingly, extensive video game practice did not substantially enhance performance for non-gamers on most cognitive tasks, although they did improve somewhat in mental rotation performance. Our results suggest that at least some differences between video game experts and non-gamers in basic cognitive performance result either from far more extensive video game experience or from pre-existing group differences in abilities that result in a self-selection effect.
Meaning-making and learning in the era of digital text
  • B Cope
  • M Kalantzis
  • S S Abrams
Child-centered versus teacher-directed teaching practices: Associations with the development of academic skills in the first grade at school. Early Childhood Research Quarterly
  • M K Lerkkanen
  • N Kiuru
  • E Pakarinen
  • A M Poikkeus
  • H Rasku-Puttonen
  • M Siekkinen
  • MK Lerkkanen