Content uploaded by Stéphanie Philippe
Author content
All content in this area was uploaded by Stéphanie Philippe on Aug 21, 2020
Content may be subject to copyright.
A Serious Game for Amplifying Awareness on
Multimodal Teaching: Game Design and Usability Study
Petros Lameras
Coventry University, Coventry, UK
ab3430@coventry.ac.uk
Stephanie Philippe
Manzalab, Paris, France
sphilippe@manzalab.com
Lars Oertel
Akademie für berufliche Bildung gGmbH
Dresden, Germany
l.oertel@afbb.de
Abstract— STEAM is a serious game developed as a medium for helping
teachers to experience multimodality for teaching and learning. A design-based
paradigm is adopted to elucidate how in-game design elements coupled with
learning may visualize in-game multimodal representations. Multimodality is
experienced as a process of creating meaning though connecting and combining
different modes, semiotic resources and semiotic ensembles. In this paper, we
present the design and usability evaluation of the game. The usability study was
conducted with (n=32) school teachers completing an online survey after
playtesting the game for identifying, capturing and fine-tuning in-game
usability aspects. The findings indicated that the game’s core mechanics, the in-
game dialogues and card-game, represent and visualise the content and process
of multimodal in-game ensembles whilst the development of in-game feedback
and progress indicators was perceived as having the capacity to guide
understandings on in-game multimodality and to track in-game progress.
Keywords—serious games, multimodality, teaching, usability, game design
1 Introduction
This paper provides a discussion on and analysis of the design and usability
evaluation of a serious game developed to help school teachers to understand the
concept of multimodality as a pedagogically-rich approach to enhancing learning and
teaching. Multimodality is an activity-based and student-centred approach to teaching
in which a series of different tools, technologies, resources and environments are
deployed transcending the basic idea of reading and writing to multiple-form of
mixed-print representations (Jewitt, 2008 [1]). Games for learning and teaching have
been perceived as a medium for instigating playful learning aligned with rich-
mediated content to achieve in-game learning goals. The overarching aim of such
games is to infuse learning content amalgamated with teaching models, frameworks
assessment and feedback in-game representations for discerning a more constructive,
reflective and memorable learning experience (Belloti et al., 2012 [2]; Blanco et al.,
2012 [3]; Boot et al., 2008 [4]).
There is a widespread view from different commentators across disciplines (e.g.
Cope & Kalatzis, 2009 [5], Cowan & Kipriani, 2009 [6]) that school teachers seem to
be overwhelmed by the plethora of teaching representations such as technologies,
resources, pedagogies and tools that may be used for enacting teaching in more
activity-oriented ways that would allow to design activities that encourage students to
be actively involved in situated learning instances in or out of the classroom.
Multimodality is indeed an ill-defined concept encompassing social and cultural
shaped resources with an emphasis on the inherently social negotiated character of
meaning (Lave, 1991 [7]).
The following sections start by elucidating on the design of the STEAM game
encompassing analysis of high-level goal, scenarios and learning objectives along
with an analysis of the core game mechanics. Then we continue with presenting the
methods employed and the findings of the game’s usability evaluation. We conclude
by providing a discussion on usability considerations and implications for multimodal
learning and teaching, limitations and future research.
2 Game design
STEAM is a simple point-and-click game that may be played through a web
browser. The game play sets-up a blend between non-linear dialogues with a Non-
Player Character (NPC) with a set of choices for the player to choose from along with
a mini card game for players to select the correct card combinations for establishing a
multimodal teaching environment.
2.1 In-game goals and outcomes
The game’s narrative sets the player to have the role of a newly appointed teacher
interested to learn more about multimodality and how (e.g. resources, pedagogies,
modes, technology) it can be practiced. The main in-game goal therefore is to create
an awareness of what is multimodality and how it can be practiced. The player
commences the game as having the fictional character of ‘Mary’ a newly appointed
mathematics teacher at Charles Darwin School that attempts to transcend theory and
practice as means to enhance teaching by using multimodal tools, pedagogies and
strategies. Interactions with the students and ways of delivering content is influenced
by the dialogue responses given by the player, which in turn enable the collection of
game cards for designing lesson plans.
2.2 In-game scenarios
To situate multimodality into a learning context, we have designed 3 chapters
encompassing four learning scenarios in which multimodal teaching challenges are
presented to the player. The flow of each scenario determines the in -game sequence
aligning the dialogue and cards game mechanics and the actions of the player (see
Figure 1).
Fig. 1. The flow of a scenario is a logical path for the player to demarcate narrative sequences,
dialogue sequences and card-deck combinations with colleagues and students
Each chapter may be played as a way of introducing the player to three multimodal
in-game objectives following the subject content and flow of each scenario. When the
player loops out of the introduction sequence, then the scenario starts by introducing
the narrative sequence as a means for the in-game dialogue mechanic to engage the
player with the teacher NPC for getting information about what multimodality is and
to highlight that it reflects a range of pedagogies, tools, technology and resources
amalgamated together to form an interacting and multimodal learning experience
within and beyond the classroom context. Then, the player is introduced to the next
core game mechanic, the game cards which are combined together to form a
particular multimodal situation that the teacher would favor. A new dialogue
sequence follows, with students contemplating on and articulating about the choices
the player has made during the dialogues and possible applications in the real-world
teaching domain.
2.3 Core game mechanics
We have designed our core game mechanics to prompt for progressively learning
the objectives of each chapter whilst to rapidly comprehend the multimodal aspects
that the dialogue conveys to the player. The dialogue mechanic drives the
multimodality learning process and twins the pedagogical objectives of each scenario
with player’s chosen response. The dialogue mechanic is part of the narrative
sequence in which the player responds to questions asked either by colleagues or by
students. The player has three options to choose from: one of the dialogue options is
the correct, the other is intermediate and the third is not correct. (see Figure 2).
Fig. 2. Left chart: The dialogue challenge is set for the player to respond. Right chart: The
player has 3 in-game options to choose from
When the player selects a dialogue option it is highlighted with a green frame as a
visual representation to denote the choice is correct, with a yellow frame to show that
the choice is intermediate and with a red frame to signify that the choice is not
correct. The general consensus is to guide players understandings on how
multimodality may be viewed in multiple perspectives that would likely increase
student’s in-game learning and engagement. As the player correctly responds to
questions, up to ten cards are available separated into different categories such as
strategies, activities and locations. The cards need to be correctly combined in order
to form a particular multimodal situation. (see Figure 3).
Fig. 3. Left-chart: The player gains cards to be used for designing a multimodal situation
comprised of strategies, activities and location. Right-chart: the player combines cards
together to design a multimodal situation that influences students’ engagement levels.
When the initial engagement is low, the player should select a highly engaging
combination. In contrast, when the students start the session with likely high
engagement levels, teacher should select a combination that will not further increase
their engagement levels. Hints about the outcome of each card combination are
provided to support the player’s choice. Based on the card selection the player decides
the engagement level bar increases or decreases thereby affecting the average level’s
grade. This means that there are certain card combinations, which are valid, and
others that are not. This does not imply that a teacher may perceive multimodality
differently from what is defined by the game’s script, however the game’s inner logic
presupposes some combinations to be more favorable than others as for the system to
provide the necessary feedback to the player thus structuring the game’s scoring
system.
2.4 In-game feedback and progress indicators
In-game feedback is augmented in different ways for helping the players to
comprehend both their in-game progress but also to understand the concept of
multimodality during the dialogue sequences and card game. For example, the
engagement bar and the learning bar indicating engagement of students and their
learning level. This visual representation type of feedback is displayed during the
dialogue sequence with the students for the player to have instant information on
his/her progress for quickly adjusting performance. We have integrated this type of
visual feedback for the players to be able to comprehend easily the meaning of the
feedback received thus to get extrinsic feedback on their performance (see Figure 4).
Fig. 4. Visual representation of the ‘Engagement level’ and ‘Learning level’ of the students
during the dialogue sequence (top left corner). Textual feedback may be obtained by
clicking on the“?” icon (top right corner).
Suggestions on how to employ alternative pedagogical approaches, technologies
and resources are provided by looking in the library and search for inspiring card
combinations. The design of the multimodal card deck library had a dual feedback
purpose: Firstly, for players to reflect on the combinations already made, and
assimilate the multimodal features chosen and secondly, to experience and understand
new multimodal card combinations, not selected during game-play, that would help to
understand different ways and variations of using multimodality. Players may choose
any of the card decks from strategies, tools and locations categories and then
suggested combinations are provided by the game. For example, if players select the
argumentation debate card from the strategy category then a suggestion pops up
combining a blog tool and a home location (see Figure 5).
Fig. 5. Top-left chart: Setting a multimodal strategy by using the ‘argumentation debate’. Top-
right chart: Selecting the ‘audio’ tool. Bottom-left chart: Selecting a ‘classroom’
location card. Bottom-right chart: Selecting a museum location card.
The number of card combination that a player can make is indicated and also the
game highlights the available cards for activities and location, once a strategy has
been selected as a way for grouping multimodal strategies with associated tools and
locations. Encompassing different ways of assimilating and accommodating feedback
through involving both formative and summative strategies felt it would provide a
balance between players who would wish to constantly and rapidly accumulate their
progress just as when playing entertainment games and for players who would
perceive feedback more of a continuous learning process that would help them
understand distinctive ways of practicing multimodality by attempting to make
diverse combinations of multimodal strategies, tools and locations.
3 Usability evaluation method
The aim of the usability evaluation was to understand how teachers experience the
features of STEAM as means to help refining the usability of the game as part of an
iterative, systematic and user-focused game design process. The method we used for
the evaluation is premised on the concept of inductive reasoning, for inferring the
meanings and outcomes of the data collection and analysis and then investigating
related issues to work-out a usability evaluation synthesis stemming from the
following research questions: (1) How teachers perceive STEAM’s usability features
as means to represent and visualise in-game user interface, multimodal content, core-
game and feedback mechanics? (2) What are the implications of teachers’ STEAM
usability satisfaction on game uptake in the classroom for increasing awareness on
multimodal teaching?
3.1 Data collection
We were quite promiscuous in our ways of selecting the participating teachers
making sure that they were from different cultural backgrounds, subject-topics, years
of teaching experience and dexterity in playing games or familiarity in using games
for learning and teaching. This variation in participants’ characteristics and the
purposive sampling approach that we employed allowed for a more diverse set of
responses adhering to reliability and validation procedures in terms of identifying
usability aspects derived from the data. Thirty-two (n=32) school teachers
participated in the usability study. Participants has to playtest the game for 40 minutes
and then they had to complete a three-point scale usability online questionnaire.
Starting with the playtesting process, we have introduced the game to the participants
and we have elaborated the objectives of the study. We have made explicit all ethical
procedures such as rights to withdraw at any time, data management, handling and
storage, their voluntary participation and the right to withdraw at any time. Then,
participants started to playtest the game experiencing the game’s interface, content
and the dialogues and card game mechanics, contemplating on how multimodality
was inferred and how progress and feedback indicators were related to goals,
objectives, in-game dialogue, and card combinations. Then, participants started to fill
in the online questionnaire with special focus on assessing the extent to which the
game’s usability objectives and user satisfaction were achieved.
3.2 Data analysis
The analysis embarked a quantitative approach to analyse the game usability data
with closed questions underpinning a value in terms of how usability was measured
on an average scale. The analysis was focused on extracting data on specific usability
aspects: (1) general usability aspects (e.g. user-interactions, visual aspects) (2)
aesthetics and interface (layout, amount of information), (3) learning the system
(game difficulty, feedback). Data were then grouped by a researcher to the
overarching usability themes against the data and then they were validated for
consistency between the themes from the research team through a participatory
process of probing, debating and reflecting on the data, the themes and associated
relationships. This helped on mitigating aspects related to validity and reliability and
biased processes of analysis often emanated from a researcher’s presuppositions and
interpretations.
4 Usability results
In what follows, we present the results of the STEAM’s usability study. We have
categorized responses in themes stemmed from the online questionnaire. We provide
a description of each usability response followed by a visual graph representing the
collective responses of the participants.
4.1 Game’s usability
Responding to a generic question in terms of ‘how well were you able to use the
game’, participants felt that there were completely and somewhat being able to use
the STEAM in terms of interacting with the point & click input mechanism to select a
dialogue, continue with the pace of the text and also selecting a particular game card
(see figure 6).
Fig. 6. Overall game usability
In line to this, the game’s visual aspects seemed to contribute to participants’
engagement levels as n=5 reported they were ‘completed satisfied’ and n=21 and 6
said that visual elements did not engaged them into game-play. Visual elements were
key interface components for helping the players to understand the rules, goals and
progress (see figure 7).
Fig. 7. Visual aspects
4.2 Aesthetics and interface
On a scale from 1-9, participants felt that the GUI was aesthetically pleasing n=20
(scales 6-9) and n=7 (scales 1-4) perceived that improvements may be needed mainly
due to observations in terms of missing a back button for going back to the questions
answered previously and also because sometimes there was a mismatch between the
visual feedback icon with the overall score at the end. Also, some text felt a bit small
hence larger text could be implemented in the next game iterations (see figure 8).
Fig. 8. Overall reactions to the GUI
Understanding the amounts of information passing from one screen to the next was
key in providing a balanced approach to how participants perceived information
overload. Participants (n=25, scales 6-9) felt that there were efforts to enhance game
screens in a consistent way and 3 felt that information was overloaded (see figure 9).
Fig. 9. Arrangement of information between game screens
4.3 Learning the system
Participants (n=28 6-9 scale) perceived the game as easy to start playing it. 5
Participants (1-4 scales) felt sthe game was moderately easy due to its straightforward
dialogue selection process and the drag& drop card selection functionality (see figure
10).
Fig. 10. Level of difficulty of playing the game
Feedback provided in terms of responding to a question was found positive with 19
responses (scales 6-9) perceived as helpful in terms of responding to the next question
and reflecting on how multimodality may be experienced There were some concerns
about the association between the dialogue chosen and the feedback provided as
sometimes, a feedback response would not focus on what the player needs to do in
order to correct the false answer or even if the answer is correct, how does this relate
to multimodal teaching? Due to this reason, 8 participants said that feedback was
‘somewhat’ helpful in terms of understanding a question (see figure 11).
Fig. 11. Usefulness of feedback for responding to a question
5 Discussion
The aim of the usability study was to discern teachers’ responses and satisfaction
levels on the STEAM’s usability features as means to refine and fine-tune the game’s
overall usability, interface and player’s adaptability in terms of learning how to play
the game. This will pave the way for introducing the game into schools in general and
for developing teachers’ experiences on multimodal teaching through STEAM.
Contemplating on the usability findings there are certain usability and game design
implications for employing the game into the classroom as a medium to develop
teachers’ awareness on multimodal teaching.
From a usability and game design perspective, the game should focus on a closer-
integration between the core game mechanics as means for players to understand the
essence of different multimodal situations discerned from the dialogues and being
extended through the selection of the strategy-tool-locations cards combinations.
There is also an implication in terms of how feedback is rendered, visualised and
represented after making a choice and also when selecting a card combination.
Feedback therefore should be aligned with the choice made by the player but also
with the game cards selection. Achieving consistency not only between player’s
action and dialogues outcome but also between dialogue’s and game card outcome
would facilitate the process of alignment between interfaces and core mechanics.
6 Conclusion
This paper presented the design and usability evaluation of the STEAM serious
game as a medium for helping teachers to experience multimodal teaching. The
game’s design features have been discussed and analysed along with its game
objectives, scenarios and content coupled with the two core game mechanics, the
dialogue and the game cards.
The findings from the usability study showed that user interface elements are
perceived as logically structured rendering the content, process and mechanics in a
usable way for the users to be able to learn procedural game-play and multimodal
content. While the usability study provides evidence of game features that need to be
refined and fine-tuned to achieve a streamlined game-play experience, there are also
some limitations: Firstly, the sample was relatively small to capture the entire
spectrum of a plethora of usability aspects that may be hindered and secondly
generalisability of usability results may not be possible to be replicated due to
contextual, cultural and technical considerations. To understand how the game
enhanced experiences of multimodality, a study that delimits in-game experiences of
multimodal teaching and how game design aspects influenced such experiences will
shed light to teachers’ ways of understanding multimodality.
7 Acknowledgements
The STEAM project received funding from the European Commission, Erasmus +
with grant agreement n°2016-1-FR01-KA204-024178
8 References
1. Jewitt, Multimodality and literacy in school classrooms. Review of research in education,
2008. 32(1): p. 241-267.
2. Bellotti, F., et al., Designing a Course for Stimulating Entrepreneurship in Higher
Education through Serious Games. Procedia Computer Science, 2012. 15(0): p. 174-186.
3. Blanco, Á.d., et al., A Framework for Simplifying Educator Tasks Related to the
Integration of Games in the Learning Flow. Journal of Educational Technology & Society,
2012. 15(4): p. 305-318.
4. Boot, W.R., et al., The effects of video game playing on attention, memory, and executive
control. Acta Psychologica, 2008. 129(3): p. 387-398.
5. Cope, B., Kalantzis, M, Multiliteracies: New literacies, new learning. Pedagogies: An
international journal, 2009. 4(3): p. 164-195.
6. Cowan, K., Cipriani, S, Of water troughs and the sun: Developing inquiry through
analogy. Young Children, 2009. 64(6).
7. Lave., J., Situating learning in communities of practice. Perspectives on socially shared
cognition, 1991. 2: p. 63-82.
