ArticlePDF Available

The Implications of Multimodality for Media Literacy

Authors:

Abstract

Today's media consumers can consume, produce and disseminate media messages involving multimodal representation. Consequently, in both receptive and expressive modes of communication, multimodal representation demands that media consumers possess a wide range of media-related knowledge and competencies. While multimodal representation in itself poses significant media literacy challenges to the media consumer, this chapter asserts that several concomitant trends in the mediascape further compound the severity of these challenges: the growing ease of manipulability of media content, the rise in media genre-hybridisation and the increasing proliferation of user-generated media content. The chapter then argues that media literacy needs to be reassessed so that greater emphasis be placed on multimodal literacy and visual literacy. Media consumers should also be vested with the skills to read and navigate the multimodal and hypertextual environment, and to recognise the limitations of their foundational knowledge structures. The chapter concludes by making several recommendations for media organisations, policy makers and media consumers to respond to the media literacy challenges posed by our increasingly multimodal and rapidly changing mediascape.
Multimodal Representation And Knowledge
1
This is a preprint version of: Lim, S. S., Nekmat, E. & Nahar, S. N. (2011). The implications of
multimodality for media literacy. In K. O’Halloran & B. A. Smith (Eds.) Multimodal Studies Exploring
Issues and Domains (pp. 169-183). London: Routledge.
The Implications of Multimodality for Media Literacy
Sun Sun LIM, Elmie NEKMAT and Siti Nurharnani NAHAR
Abstract
Today’s media consumers can consume, produce and disseminate media messages
involving multimodal representation. Consequently, in both receptive and expressive modes of
communication, multimodal representation demands that media consumers possess a wide range
of media-related knowledge and competencies. While multimodal representation in itself poses
significant media literacy challenges to the media consumer, this chapter asserts that several
concomitant trends in the mediascape further compound the severity of these challenges: the
growing ease of manipulability of media content, the rise in media genre-hybridisation and the
increasing proliferation of user-generated media content. The chapter then argues that media
literacy needs to be reassessed so that greater emphasis be placed on multimodal literacy and
visual literacy. Media consumers should also be vested with the skills to read and navigate the
multimodal and hypertextual environment, and to recognise the limitations of their foundational
knowledge structures. The chapter concludes by making several recommendations for media
organisations, policy makers and media consumers to respond to the media literacy challenges
posed by our increasingly multimodal and rapidly changing mediascape.
Introduction
With the advent of Web 2.0, the media consumer is endowed with the ability to consume,
produce and disseminate media messages often involving multimodal representations which
incorporate text, images and sound. Consequently, in both receptive and expressive modes of
communication, multimodal representation demands that media consumers have knowledge and
competencies in a wide range of aspects textual understanding, visual and aural literacy, genre
identification, critical analysis, legal know-how, ICT skills, industry insights and more. While
multimodal representation in itself poses significant media literacy challenges to the media
consumer, this chapter argues that several concomitant trends in the mediascape further
compound the severity of these challenges: the growing ease of manipulability of media content,
the rise in media genre-hybridisation and the increasing proliferation of user-generated media
content. The chapter then considers why and how media literacy needs to be reassessed in a
mediascape increasingly marked by multimodality. Finally, it concludes by identifying which
literacies are most critical in our current mediascape and makes several recommendations for
research and policy formulation.
Multimodal Representation And Knowledge
2
Multimodal representation shifts in the mediascape
The mediascape has seen a discernible shift in semiotic modes of representation towards
a growing dominance of visual images. While unimodal text-only documents have certainly not
faded into oblivion, their importance has waned as media continues to evolve with enhanced
capacities for holding and displaying texts in various modes, containing graphics, pictures,
layout techniques, and more (Goodman 1996). Over the past century, there has been a broad
move from the supremacy of writing and the written word to the dominance of the image in
different media technologies (Jewitt and Kress 2003). These technologies, also known as
‘technologies of literacy’ (Warschauer 2003, 115), had notably shifted from the printing press
which privileged the written word, i.e. text, over all other semiotic modes (Kaplan 1995). Kress
argues that in the current media environment, images are assuming increasing importance such
that images lead over text and the screen takes precedence over books (2003). Kress states that
the screen is a “visual entity” (2003, 65) and text that appears on screen is similarly treated as an
image and follows the same principles of visual design. As a result, written text which appears
with images plays a secondary role with regard to conveying meaning. Arguably, this trend arose
from the popularity of film and television, resulting in the format of print media such as
newspapers, magazines and books being altered, such that visual images have become, and
continue to be, increasingly prominent (Kress and van Leeuwen 1996). Multimodal
representation is therefore not entirely novel in this information age but is a reflection of the
intensifying use of multiple communication modes in media, particularly that of visuals. Notably
though, while television, film and printed texts have long been marrying textual and visual
content, the advent of information technology has made multimodal communication much more
prevalent (Warschauer 2003). On top of combining text, photos, videos, audio and graphics in a
single presentation, the decentralized nature of information production further aids the
proliferation of this multimodal communication phenomenon. In sum, multimodal representation
is by no means a new phenomenon, but one that has evolved in its intensity, scale and
complexity with the introduction of new information and communication technologies.
Multimodality and concomitant trends
While multimodal representation in itself poses significant media literacy challenges to
the media consumer, several concomitant trends in the mediascape further compound the
severity of these challenges: the growing ease of manipulability of media content, the rise in
media genre-hybridisation and the increasing proliferation of user-generated media content. Each
point will be discussed and illustrated as follows.
Multimodality and the manipulability of media
With the advent of digitisation, media and information have become extremely easy to
modify and manipulate. As Feldman (1997) argues, compared to analogue media where the
process of reshaping information can often be “difficult, slow and untidy”, digital media on the
other hand allows users to infinitely and easily alter information “at a stroke” (4). This
Multimodal Representation And Knowledge
3
affordance of advanced digital tools has significant implications for all stages of information
representation: from the moment it is created and captured in digital form, to its dissemination,
on to audiences’ engagement with the information and beyond.
While media content of different modes can be easily modified and altered, in light of the
significance of images in today’s mediascape, let us consider the implications of the ease of
manipulating images. Digital graphics are presently used in creating interactive user interfaces,
virtual reality, animation, as well as reconstructing three dimensional objects from their “2D
projectional presentations” (Groß, 1994, 2). The digitisation of graphics enables us to handle
images in unprecedented ways, including the restoration of old and damaged photographs and
the seamless recombination and morphing of snippets of different images. Such affordances have
been adopted with enthusiasm and exploited by artists and media producers in surprising and
creative ways. However, the ability to modify digital images has also opened an avenue for
misinformation and deception. This concern is not unwarranted considering that doctored images
are being disseminated even by established and reputable media organisations such as Reuters
and the Los Angeles Times. A 2006 Reuters photograph of smoke rising from buildings in Beirut
was attacked by American bloggers for having been doctored (BBC News 2006). Upon
investigation, it was found that Adnan Hajj, the Reuters photographer, had distorted the
photograph to include more smoke and damage. In another example, Brian Walski created a new
photograph by manipulating two photographs which he had taken for the Los Angeles Times
(Van Riper 2003). In the first photograph, a US soldier was pictured with his gun pointed
horizontally while in the second photograph, the soldier’s gun was lowered when a man with a
baby stood near him. By combining both photos, the altered image made it appear as though the
soldier was pointing his gun directly at the man with the baby, presumably to heighten the
photograph’s dramatic effect and to enhance its human interest value. Such egregious practices,
even by professionals from renowned news organisations, are a stark reminder that today’s
media consumer needs to be even more critical and sceptical than ever in their consumption of
media content. In these two cases, the visual literacy skills of the consumer are tested as they
need to appreciate the telltale signs of digitally doctored photographs. Yet the sophistication of
today’s graphic applications produce such flawless results that the visual literacy skills of even
shrewd media consumers would be easily defeated. In such a mediascape, media consumers have
the unenviable responsibility of being constantly questioning and discerning about their media
sources, regardless of how established and reputable might be. Quality indicators that used to
serve media consumers well are of diminished value into today’s media environment.
Quite apart from the manipulability of media itself, the access to media content can also
be manipulated, especially online. The prominence of online information can also be directed
such that some content is replicated repeatedly, while others remain obscure and difficult to
access. Given that the World Wide Web is a seemingly infinite mass of information, media
consumers have no option but to locate information online using search engines. The ways in
Multimodal Representation And Knowledge
4
which search engine results are ordered depend on a combination of factors including the design
of search engines algorithms, the search engine’s revenue stream and business model and the
original source of the content (Hargittai 2004). Commercial interests are behind the most popular
Web sites which users frequently visit to get their online content (Hargittai 2004). The order of
search engine results becomes significant because most users are unwilling to explore results
beyond the third page (iProspect 2008). In 2008 the percentage decreased to 9% as compared to
2006 (12%), 2004 (17%) and 2002 (22%). Hence, media consumers are in some sense at the
mercy of search engines. Search engines and directories systematically exclude certain sites in
favour of others, either by design or by accident (Introna and Nissenbaum 2000). For example,
Google has been observed to exclude certain sites from its searches- compared to google.com,
113 sites were excluded, in whole or in part, from the French google.fr and German google.de
(Zittrain and Edelman 2002). In this regard, the discerning media consumer is one who
understands how media industry practices and pressures may result in the omission of particular
content and the amplification of others, and that the information which they derive from online
sources can be fraught with bias. However, it would be fair to say that such esoteric knowledge
about how search engines work would be beyond the average media consumer.
Multimodality and genre-hybridisation
Practices within the media industry are leaning towards the hybridisation of different
media forms. The classical distinctions between documentaries, news, information,
entertainment, dramas, comedies, editorials and advertisements have become blurred. There has
been a growing trend towards eclecticism, where “a cultural text creatively mixes, blends, or
recombines pre-existing and relatively discrete cultural forms, formulas and techniques” (Ott
2007, 58). Such media production practices result in texts which can be particularly challenging
for media consumers as their conventional frames for understanding media content may be
inadequate or even inappropriate. As Campbell and Freed (1993) opined, “television is certainly
a fertile breeding ground for genre confusion. Categories once chiselled in granite melt in a swirl
of crossover jargon: docudrama, infotainment, infomercial, dramedy (77).” Similarly, with print
media such as newspapers and magazines expanding into online platforms with new channels for
interacting with readers, the classical categories of reportage, opinion-editorials and letters from
readers have been partially displaced by hybrid forms such as readers’ blogs and opinion forums,
‘first-person’ citizen journalist reports replete with amateur photos etc.
The combination of genre hybridisation and multimodality has birthed even more
boundary-crossing media types. The rising sophistication of computer animation and production
techniques has facilitated the creation of multimodal content of an extremely high quality, such
that genre hybridisation in television and films has broken new ground. An excellent example is
the British television programme Prehistoric Park which features the well-known British wildlife
documentary host Nigel Marven playing himself. In this programme, Marven (with the help of a
time travel device) is tasked with finding extinct animals from prehistoric eras and bringing them
back to the present day for exhibition in the Prehistoric Park. The prehistoric animals are
Multimodal Representation And Knowledge
5
rendered in computer generated imagery and animatronics, and they interact with the human
actors and natural landscapes in a lifelike fashion. Marven is often shown spying on the
prehistoric creatures in their ‘natural’ habitats and touching them when they are taken into
captivity. While the characters in the show are thus multimodal in nature, the style of the show is
multi-genre and defies classification. Marven appears in his usual guise of the reputable wildlife
expert who addresses viewers directly, as though they were watching a documentary. The
wildlife scenes are set in actual physical locations and bear the patina of a scientifically-based
nature programme, thus appearing highly realistic. However, Prehistoric Park is mostly fictional
and more closely resembles a drama with the typical elements of emotions, humour, suspense
and even tragedy. This blend of multimodality and genre hybridisation sends mixed signals to
viewers as ‘traditional’ production techniques are both obverted and subverted. A sampling of
viewers’ comments on clips of the show excerpted on Youtube.com reveals the confusion that
some viewers experience from “it's my favourite documentary tv show (emphasis mine)”, to
“But it looks so real to me, is this real!?”, “so you mean everything in this is real?” and “im (sic)
watching this on tv right now, IS THIS REAL OR FAKE !?!??!?!?!” (Sidewaysnic, n.d.). As
these comments exemplify, the potent combination of multimodality and genre-hybridisation can
significantly test consumers’ media literacy skills.
Multimodality and the proliferation of user-generated content
Information technology now provides media consumers with the means to create,
replicate and disseminate media content. With the spread of affordable media production
hardware and software, and the emergence of a slew of content sharing sites on the Internet, the
growth of user generated content has been significant. Previously, the high costs of information
production and dissemination served as barriers and restricted the number of content providers to
only those with adequate authority and capital (Metzger 2007). Today, as long as one has the
technical know-how and access to the technology, one can easily become an author. We should
bear in mind of course, that the extent to which individuals can and do avail of such self-
authoring opportunities differs by motivations, skills and interests.
In short, media consumers are now able to produce their own forms of representation,
both uni- and multimodal. This proliferation of user generated content compounds the
subjectivity of the information which they create as each different mode offers the potential for
different “representational and communicational action by their users” (Kress 2003, 5). The
average media consumer is now concurrently in possession of the resources of representation, the
resources of production and the resources of dissemination. These distinct resources require
specific competencies not only in their use but also in the design of information, i.e. in the
consumer’s receptive and expressive modes (Kress 2003). In other words, it is more important
than ever for the average consumer to understand the role of the designer with regards to the
meaning-potentials of the resources which they now enjoy. This is further complicated by the
interplays between the ‘semiotic resources’ (i.e. mode) and the ‘material substance’ (i.e.
Multimodal Representation And Knowledge
6
medium) (Kress and van Leeuwen 2001, 215, 41) which provide various potentialities, but also
present complexities in the representation of information.
Critical reflexivity is also required in view of the synergies and potential for convergence
amongst different forms of digital media such as computers, mobile phones, cameras, palmtops
and many more. The increasing portability of these media tools, coupled with their enhanced
capacity to hold different modes of information greatly increases one’s ability not only to
produce, but also to communicate information through various modes and resources. Inevitably,
media consumers’ exposure to multimodal forms of communication will only increase.
Therefore, possessing the competencies to comprehend how the various modes and ‘material
resources’ affect the representational potential of information not only requires critical
reflectivity of the context in which the information was produced, but also reflexivity in one’s
own production of information.
Reassessing Media Literacy
Media literacy can be defined as the ability to ‘decode, evaluate, analyze and produce
messagesin a variety of forms (Aufderheide 1993, 1). While this pithy definition adequately
captures the multi-faceted nature of media literacy, the way in which media literacy is defined
depends not only on the entity defining it but more significantly, should change to suit the
evolution of the media landscape (Potter, 2004). Indeed, in light of the trends discussed in the
previous section, a reassessment of the concept of media literacy is timely. In this regard, what
are the implications of multimodality for media literacy, especially when it comes to one’s
critical analysis and evaluation of media content? Several scholars have identified several media
literacy imperatives arising from the increasingly complex nature of multimodal representation.
The need for multimodal literacy and multiple literacies
Accompanying the proliferation of multimodal representation in today’s media landscape
have been shifts in how meanings are created and understood (Jewitt and Kress 2003; Lankshear
and Knobel 2003). With the widespread deployment of different modalities, media and materials,
each with its own logic and affordances, media consumers’ meaning-making processes are
getting more complex than ever. Understandably, scholars concerned about the effects of
multimodality have focused on understanding the different ways in which meaning can be
created and communicated in the world today (Kress and van Leeuwen 1996; Baldry 2000; O’
Halloran 2004). They have focused, inter alia, on the ‘semiotic affordances of image, of writing
and of speech and of multimodal texts’ (Jewitt and Kress 2003, 166) and on how ‘intra and inter-
semioses' arising from the interaction within and between two or more semiotic modes empower
or disempower creators and receivers of multimodal texts (O’ Halloran 2004, 224).
The multimodally-literate media consumer is primarily viewed as one who: 1) displays a
systematic understanding of how texts make meanings and how these meanings can be conveyed
by different communicative forms such as language, image, sound, gesture, etc.; 2) possesses the
Multimodal Representation And Knowledge
7
competency to integrate textual analysis with an appreciation of how audiences engage with the
texts under scrutiny; and 3) has the capacity to integrate textual analysis with the political,
economic and social contexts in which the texts are produced and consumed (Burn and Parker
2003, 3-4). As compared to multimodal literacies that emphasise the ability to critically decipher
the meaning-making potential of semiotic resources, media literacy focuses on the skills to
access, consume, assess and produce content (Livingstone 2004), with a shift towards a concern
with critique, reflection and judgement” (Martin 2006, 18). The ability to critically analyze
symbolic texts thus lies at the intersection of multimodal literacy and media literacy and a robust
definition of media literacy that serves today’s mediascape has to take into account
multimodality and incorporate multimodal literacy. Clearly, the range of literacies that comprise
media literacy has widened. There have been calls for ‘new literacies’ and ‘multiple literacies’
for understanding ‘post-typographic’ forms of social practices in the consumption and
production of media content (Lankshear and Knobel 2003, 16-17). It is only with the possession
of multiple literacies that individuals can participate in our highly-mediatised information society
in an efficacious manner (Kellner 2002). A few key literacies are discussed in the subsequent
sections.
Reading and navigating the multimodal and hypertextual environment
While the current multimodal and hypertextual media environment seems to exemplify a
‘brave new world’ in media representations, the basic principles of critical literacy which applied
to traditional print and mass-media contexts are still relevant today. For example, the ability to
critically appraise the heterogeneity of sources, competing authorities, non-linear or visual forms
of representation are not skills which are specific to the multimodal media environment, but have
long been required for consuming text (Livingstone 2003). However, Luke (2000) does refine
the definition of particular literacies which will enhance one’s navigation of hypertextual
environments and evaluation of online information. They are: 1) possessing the adequate meta-
knowledge of how ideas and information ‘bits’ are structured in different media genres and how
they affect people’s reading and uses of information; 2) displaying mastery of the technical and
analytical skills with which to negotiate those representational systems in diverse contexts; and
3) having the capacity to understand and relate these systems and skills as operating within
relations and interests of power within and across social institutions (72).
Jewitt and Kress (2003) also identified salient distinctions between the unimodal, text
environment and the multimodal, hypertextual one. They argue that the former tends to have
horizontal and linear content while the latter ‘immerses one in an intertextual and multimodal
universe of visual, audio, symbolic and linguistic meaning systems’ (73) which are laterally
connected, thus making reading and navigation more challenging. Readers are engaged in a
multimodal reading of texts, and have the added burden of navigating through multi-layered and
multi-coded animation, symbols, linguistic text, photos etc at the same time. In such an
environment, having a contextualised knowledge of the ideas carried by different pieces of
information will no longer suffice. Instead, media consumers are required to understand the
Multimodal Representation And Knowledge
8
relations amongst these ideas and how these ideas are affected by their representation through the
different modes. Furthermore, one also has to be aware of the effects resulting from the process
of following a hyperlink from one webpage to another, with each different page embedded with
different modes of representation. For example, reading about the large number of whales being
driven up to shore elicits a different response as compared to seeing images of the whales on the
shoreline. This process potentially affects not only their cognitive, but also their emotive states,
affecting their ability to make sound judgements about the credibility of the information which
they presented. The ability to read and navigate the multimodal and hypertextual media
environment is thus a key component of the afore-mentioned ‘multiple literacies’ required in
today’s mediascape.
Recognising the limitations of foundational knowledge structures
The critical evaluation of media content rests on an extensive body of knowledge
pertaining to the broader social, cultural, economic, political, and historical contexts in which
media content is produced (Bazalgette 1997). Apart from such contextual knowledge, Potter
(2004) posits a more comprehensive set of foundational knowledge structures that a media
literate person needs to possess for the critical evaluation of information. They are, knowledge
of: 1) media content 2) media effects, 3) media industry, 4) the real-world, and 5) self (Potter
2004). These knowledge structures enable media users to approach problem-solving and
meaning-making with a greater variety of tools. However, the robustness of these knowledge
structures is increasingly challenged by the multimodal nature of online communication, as well
as the three concomitant trends discussed above, i.e. the manipulability of media, media genre-
hybridisation and the emergence of user-generated content. Compared to traditional publishing,
content posted on the Internet may not be filtered through professional gatekeepers, and not all
websites specify traditional authority indicators such as author’s identity or affiliated association.
There are no common standards for posting information online, and digital information may be
easily changed, misused, plagiarized, or created anonymously under false pretences (Fritch and
Cromwell 2001, 2002; Metzger, Flanagin, Eyal, Lemus, and McCann 2003). In addition, since
user generated content can and often is presented in a format similar to that of established media
organisations, Burbules (1998) argues that there is a “levelling effect” where all information
becomes equally easy to access, thus contributing to the perception that all authors offer the same
level of credibility to Internet consumers (109).
In such circumstances, how relevant is one’s knowledge of media effects or media
industries when evaluating information sources? For example, is it sufficient to claim that
information presented by a trustworthy online news agency is more reliable than an independent
source which posts photographs of a similar event on a blog and even includes embedded video
footage of the actual event? In this instance, the modes of representation might actually be more
revealing indicators of information reliability than knowledge about the media producer or media
industry as a whole. Furthermore, the increasingly hazy divisions between media producers and
consumers makes it difficult to establish valid and reliable criteria for ascertaining the quality,
Multimodal Representation And Knowledge
9
ideology, market influences or professional production values of online content. One emerging
grey area is the incorporation of user-generated content by mainstream media organizations
seeking to ride the wave of citizen journalism. Viewers and readers are encouraged to contribute
content which then becomes embedded within the company’s proprietary material. For example,
readers’ and viewers’ contributions are regularly showcased in Korea’s OhMyNews, Singapore’s
Straits Times’s STOMP and CNN’s iReporter, whereas BBC News online currently picks up
more Internet traffic from micro-blogging site Twitter than it sends there (Hitwise, 2008). Yet
these organisations also issue disclaimers to absolve themselves from errors and inaccuracies in
these contributions from their audiences, clearly signalling that incorporation of user-generated
content is not tantamount to editorial endorsement.
In such a media milieu, the keystone of critical media literacy, i.e. the knowledge of the
operations and consequences of media producers as prime quality indicators, is no longer as
valid as it used to be (Livingstone 2004). Hence, while media consumers need to maintain and
grow their media-related knowledge structures, another literacy which they must possess is the
ability to recognise the limitations of these structures, and to deploy them in a reflexive manner
that suits the ever-changing contextual demands.
Enhancing visual literacy
The growing dominance of visuals as a form of representation also necessitates an
increased focus on what it means to ‘read’ images from the media. Kress (2003) stresses the
importance of understanding the logic of new reading paths, where the image ‘dominates the
semiotic organization of the screen’ (65). Other scholars and experts on visual literacy have also
emphasized the importance of understanding the effects of ‘representational conventions’ of
visual images used in media for the creation and sharing of meanings (Messaris 1998, 70).
Arguably, visual representations have the potential to imbue information with an aura of
reliability and a veneer of truthfulness. Hence, visually literate media consumers are also those
who can interpret the content of visual images, examine their social impact, and evaluate the
purpose, audience and ownership of visuals (Bamford 2003). Besides the new logics of reading
required in the highly visual media environment and the critical reflection skills needed for the
contextual appraisal of visual images, the competency to ascertain the authenticity and reliability
of visual images in media is especially vital in today’s multimodal environment. Therefore,
visual literacy is yet another literacy in the repertoire of literacies which today’s media
consumers should possess.
Conclusion and recommendations
The nature, intensity and scale of multimodality in today’s media content pose interesting
and potentially daunting challenges for consumers’ media literacy skills. The multimodal
environment necessitates that consumers be even more critical in evaluating media content
information now, more than ever, since the onus of information credibility no longer rests mainly
on traditional gatekeepers. Associated trends such as the growing ease of manipulability of
Multimodal Representation And Knowledge
10
media content, the rise in media genre-hybridisation and the increasing proliferation of user-
generated media content also serve to heighten the magnitude of these media literacy challenges.
In this regard, what can media organisations, public agencies and individuals themselves do to
confront these challenges or, at the very least, to ameliorate their effects?
Media organisations should take the initiative to provide evidence of reflexivity on their
part. In particular, they should be more transparent about their media production processes and
policies. Media consumers may then have a set of ‘environmental standards of media practice’
(Silverstone 2007, 176) on which to base their own judgements. Media organisations can also
seek to introduce their own quality indicators which serve as guidelines on how media content
can be read, understood and criticized. A good example is online encyclopaedia Wikipedia
which practises collaborative writing and editing of articles by volunteers who source, format,
rewrite and link articles. All Wikipedia content must strive to adhere to the policies of Neutral
Point of View, Verifiability and No Original Research (Ayers, Matthews, and Yates 2008, 200).
Hence, entries must be objective, have their claims supported with reliable sources and be based
on content previously published by third parties. Entries which do not meet these benchmarks are
flagged with standard message templates to warn readers of the inadequacy of the content. More
information providers would do well to emulate Wikipedia’s establishment of transparent
standards and its user-friendly style of communicating them. Similarly, search engines should
explain more clearly how their search results are derived and clearly differentiate sponsored hits,
so that consumers can be more well-informed in their use of search engine results (Machill
2004). Google has been a trailblazer in this regard, where sponsored links are clearly
demarcated. Measures such as those taken by Wikipedia and Google will help to raise public
confidence in media organisations and the content which they provide. As media organisations
seek to pursue market share and industry recognition, introducing such measures need not be
onerous but can be acts of enlightened self-interest. However, it should also be cautioned that
these measures and reliability indicators are double-edged as media consumers may go into
‘auto-pilot’ mode and make snap judgments about information credibility on the basis of these
assurances.
Policy interventions are also urgently required to help people keep pace with ever-
changing media trends so that they can continue to function well in a highly mediatised society.
On a public policy level, there is a need to prioritise which components of media literacy need to
be most urgently inculcated so that citizens can avail of new media opportunities to maximise
benefits and minimise harms. Training in multimodality needs to be built into the formal school
curriculum from an early age. To assume that young media consumers, popularly referred to as
the digital natives, are well-versed in multimodal content would be a mistake. They may be
highly attuned to the functional aspects of the multimodal, multi-media environment. However,
their ability to consume media in a critical and discerning fashion may be wanting, as several
studies suggest (see for example Hobbs and Frost, 2003; Livingstone and Bober, 2004; Shenton,
Multimodal Representation And Knowledge
11
2004). Hence, it is critical that training in media literacy, and multimodal literacy be
incorporated into the formal curriculum as early as possible, particularly since these children
would already have grown up in an environment where multimodal representation is a given.
Furthermore, multimodal media literacy training should focus on technical competencies and
critical discernment concurrently, rather than in isolation, for better results (Potter 2008). To this
end, Kalantzis and Cope (2000) proposed a ‘pedagogy of multiliteracies’ to instil an appreciation
for the multimodal media environment, comprising: 1) Situated practice working from a base
of the student’s personal interests and life experience; 2) Overt instruction comparing and
contrasting different patterns and conventions of meaning in varied cultural settings; 3) Critical
framing critiquing and contextualizing information and messages; and 4) Transformed practice
transferral of learning from one context to another, and putting theory to practice (239-242).
Such a holistic, life-centred approach better reflects the realities of the current media
environment where multimodality is ubiquitous.
As for the media literacy skills which need to be imparted to prepare individuals for the
multimodal environment, special attention must be paid to critical literacy in both receptive and
expressive modes so that people can be discerning media consumers as well as producers. In
Singapore for example, the trend is towards the imparting of functional literacy skills e.g.
teaching people how to blog, with less attention being paid to critical literacy dimensions, e.g.
helping people to understand the impact of blogging (Lim and Nekmat, 2009). In this regard,
programmes tailored to inculcate critical ‘prosumption’ of media content need to emphasise that
the semiotic democracy and multimedia affordances which come with the production of media
content have personal and societal impact, as well as legal implications.
In light of the fact that the mediascape is in a constant state of flux, media literacy
education must not end with the formal school years. Continuing adult education is also essential
so that working adults can keep abreast of the latest media trends which may well impact on their
self-efficacy, workplace productivity and sense of well-being. To this end, media literacy
programmes targeted at adults should capitalise on trends in informal learning (see for example
Drotner 2008). For example, fostering a core group of technology evangelists or ambassadors
who engage in peer-to-peer teaching would be one approach. Public resources should also be
enhanced for self-instruction and independent learning in the acquisition of media literacy skills.
Online portals, public information booths and community libraries can be key nodes for
disseminating such resources. Ultimately, policy makers need to realise that the repertoire of
media literacy skills which individuals require will constantly increase in number and change in
composition. Media literacy programmes must therefore be tailored accordingly and refreshed
constantly in today’s rapidly progressing multimodal and multimedia landscape.
What then of media consumers themselves? Above all, they need to come to terms with
the mercurial nature of the mediascape, and recognise that heuristics for understanding and
Multimodal Representation And Knowledge
12
assessing media content are being rapidly superseded. While the instinct to surrender helplessly
to these apparently inexorable trends is great, media consumers should still seek to equip
themselves with the competencies to critically and profitably access, consume and produce
media content. Indeed, it can never be sufficiently stressed that the responsibility ultimately lies
with media consumers to be conscious of the limitless possibilities and potential pitfalls in our
multimodal media environment.
References
Aufderheide, P. 1993. Media Literacy: A report of the national leadership conference on media
literacy. Aspen, CO: Aspen Institute.
Ayers, P., C. Matthews, and B. Yates. 2008. How Wikipedia Works and how you can be part of
it. San Francisco, CA: No Starch Press.
Baldry, A. P., ed. 2000. Multimodality and multimediality in the distance learning Age. Ed. A.
P. Baldry. Campobasso, Italy: Palladino Editore.
Bamford, A. 2003. The visual literacy white paper. UK: Adobe Systems Incorporated.
http://www.adobe.com/education/digkids/resources/visual_literacy.html (accessed
November 22, 2008).
Bazalgette, C. 1997. An agenda for the second phase of media literacy development. In Media
literacy in the information age: Current perspectives, information and behaviour, ed. R. W.
Kubey, 69-78. New Brunswick, NJ: Transaction.
BBC News. 2006. Reuters drops Beirut photographer. BBC News, August 08.
< http://news.bbc.co.uk/2/hi/middle_east/5254838.stm> (accessed August 29, 2008).
Burbules, N. C. 1998. Rhetorics of the Web: Hyperreading and critical literacy. In Page to
screen: Taking literacy into the electronic era, ed. I. Snyder, 102–122. London: Routledge.
Burn, A. and D. Parker. 2003. Analysing Media Texts. London: Continuum International
Publishing.
Campbell, R. and R. Freed. 1993. 'We Know It When We See It’: Postmodernism and
Television. Television Quarterly 26:75–87.
Drotner, K., H.S. Jensen and K.C. Schrøder. 2008. Informal learning and digital media.
Newcastle, UK: Cambridge Scholars.
Feldman, T. 1997. An Introduction to Digital Media. New York, NY: Routledge.
Multimodal Representation And Knowledge
13
Fritch, J. W., and R. L. Cromwell. 2001. Evaluating Internet resources: Identity, affiliation, and
cognitive authority in a networked world. Journal of the American Society for Information
Science and Technology 52(6): 499–507.
Fritch, J. W., and R. L. Cromwell. 2002. Delving deeper into evaluation: Exploring cognitive
authority on the Internet. Reference Services Review 30 (3): 242–254.
Goodman, S. 1996. Visual English. In Redesigning English: New Texts, New Identities, ed. S.
Goodman and D. Graddol, 38–72. London: Routledge.
Groß, M. 1994. Visual Computing: The Integration of Computer Graphics, Visual Perception
and Imaging. Berlin: Springer-Verlag. Hargittai, E. 2004. The Changing Online Landscape:
From Free-for-All To Commercial Gatekeeping. In Community Practice in the Network
Society: Local Actions/Global Interaction, ed. P. Day and D. Schuler, 66-76. New York:
Routledge.
Hitwise. July 2008. “Hitwise UK Social Networking Update”. http://sg.hitwise.com/press-center/
hitwiseHS2004/uk-facebook-22072208.php (accessed December 3, 2008).
Hobbs, R., and R. Frost. 2003. Measuring the acquisition of media-literacy skills. Reading
Research Quarterly 38(3): 330-355.
Introna, L. D., and H. Nissenbaum. 2000. Shaping The Web: Why The Politics of SearchEngines
Matters. The Information Society 16: 169-185.
iProspect. April 2008. Blended Search Results Study. http://www.iprospect.com/premiumPDFs/
researchstudy_apr2008_blendedsearchresults.pdf (accessed September 28, 2008).
Jewitt, C., and G. Kress. 2003. Multimodal Literacy. London: Peter Lang.
Kalantzis, M., and B. Cope. 2000. A Multiliteracies Pedagogy: A pedagogical supplement. In
Multiliteracies: Literacy learning and the design of social futures, ed. M. Kalantzis and B.
Cope, 239-248. NY: Routledge.
Kaplan, N. 1995. E-literacies. Computer-mediated Communication Magazine 2 (3): 3-35.
Kellner, D. 2002. Technological revolution, multiple literacies, and the restructuring of
education. In Silicon Literacies, ed. I. Snyder, 154-169. London: Routledge.
Kress, G. 2003. Literacy in the New Media Age. New York, NY: Routledge.
Kress, G., and T. van Leeuwen. 1996. Reading images: the grammar of visual design. London:
Routledge.
Multimodal Representation And Knowledge
14
Kress, G., and T. van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of
Contemporary Communication. London: Arnold Publishers.
Lankshear, C., and M. Knobel. 2003. New Literacies: Changing knowledge and classroom
learning. Philadelphia, PA: Open University Press.
Lim, S. S., and E. Nekmat. 2009. Media Education in Singapore –New media, new literacies? In
Media Education in Asia, ed. Cheung, C. K., Netherlands: Springer. (In press).
Livingstone, S. 2003. The Changing Nature and Uses of Media Literacy. European
Medi@Culture-Online. http://www.lse.ac.uk/collections/media@lse/pdf/Media@lseEWP4_
july03.pdf (accessed September 27, 2008).
Livingstone, S. 2004. Media Literacy and the Challenge of New Information and
Communication Technologies. The Communication Review 7(1): 3-14.
Livingstone, S., and M. Bober. 2004. UK children go online: surveying the experiences of young
people and their parents. Monograph. July. http://eprints.lse.ac.uk/395/. (accessed July 6,
2008).
Luke, C. 2000. Cyber-schooling and technological change. In Multiliteracies: Literacy Learning
and the Design of Social Futures, ed. B. Cope and M. Kalantzis, 69-91. New York:
Routledge.
Machill, M., C. Neuberger, W. Schweiger, and W.Wirth. 2002. Navigating the Internet: A Study
of German-Language Search Engines. European Journal of Communication 19: 321-347.
Martin, A. 2006. Literacies for the digital age: A preview of part 1. In Digital Literacies for
Learning, ed. A. Martin and D. Madigan. 3-26. London, Ridgmount: Facet Publishing.
Messaris, P. 1998. Visual aspects of Media Literacy. Journal of Communication 48, (1): 70-80.
Metzger, M. J. 2007. Making Sense of Credibility on the Web: Models for Evaluating Online
Information and Recommendations for Future Research. Journal of the American Society for
Information Science and Technology 58 (13): 2078-2091.
Metzger, M. J., A. J. Flanagin, K. Eyal, D. R. Lemus, and R. M. McCann. 2003. Credibility for
the 21st century: Integrating perspectives on source, message, and media credibility in the
contemporary media environment. Communication Yearbook 27 (1):293-335.
O’ Halloran, K. L., ed. 2004. Multimodal Discourse Analysis. Ed. K. L. O’ Halloran. NewYork:
Continuum.
Ott, B. L. 2007. The Small Screen: How Television Equips Us to Live in the Information Age.
London: Blackwell.
Multimodal Representation And Knowledge
15
Potter, W. J. 2004. Theory of Media Literacy: A Cognitive Approach. Thousand Oaks: CA: Sage
Publications Inc.
Potter, W. J. 2008. Media Literacy 4th Edition. LA: Sage Publications.
SidewaysNic, "Prehistoric Park," YouTube video, 2:26, from a performance televised by Animal
Planet, added May 22, 2007, http://www.youtube.com/watch?v=hp-XPlj67As.
Shenton, A. K. 2004. Research into young people's information-seeking: perspectives and
methods. ASLIB Proceedings 56, no. 4: 243-254.
Silverstone, R. 2007. Media and Morality: On the Rise of the Mediapolis. London: Polity.
Van Riper, F. 2003 .“Manipulating Truth, Losing Credibility”. The Washington Post, April 09.
<http://www.washingtonpost.com/wp-srv/photo/essays/vanRiper/030409.htm> (accessed
September 19, 2008).
Warschauer, M. 2003. Technology and Social Inclusion: Rethinking the Digital Divide. London,
England: The MIT Press.
Zittrain, J., and E. Benjamin. 2002. Localized Google search result exclusions: Statement of
issues and call for data. Berkman Center for Internet & Society, Harvard Law School,
October 26. <http://cyber.law.harvard.edu/filtering/google/> (accessed September 29, 2008).
Author(s) mailing addresses
Sun Sun, LIM, (Corresponding author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-11, Singapore 117589
Phone: 6516-1175 Fax: 6779-4911
Email: cnmlss@nus.edu.sg
Elmie, NEKMAT (co-author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-15, Singapore 117589
Phone: 6516-8224 Fax: 6779-4911
Email: cnmmen@nus.edu.sg
Siti Nurharnarni, NAHAR (co-author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-15, Singapore 117589
Phone: 6516-3019 / 9876-1217 Fax: 6779-4911
Email: cnmsnn@nus.edu.sg
... A host of scholarship has combined the notion of multimodality with digital and media literacies. Lim et al. (2011) stress the more active consumer as a perquisite for multimodal proficiency in digital culture "This proliferation of user-generated content compounds the subjectivity of the information, which they create as each different mode offers the potential for different representational and communicational action by their users" (Kress, 2003, p. 5) (p. 174). ...
... Developing multimodal competencies (Kellner, 2002;Kress, 2003;Jewitt & Kress, 2003) can help students navigate the complex landscape for meaning creation, shared narratives, and peer-sourced content. Again, Lim et al. (2011) write: ...
... 175) As young people are utilizing digital skills and practices to broaden their repertoire of social and civic communication, media literacy education cannot stop at the critical analysis of messages. It must include the ability to navigate a complex and multifaceted digital media terrain in both "receptive and expressive modes" (Lim et al., 2011). This entails the ability to discern types of content, modes of information delivery and reception, savvy sharing, expression and reception skills, and the need to see value in vibrant peer-to-peer networks. ...
Book
Full-text available
Media Literacy and the Emerging Citizen is about enhancing engagement in a digital media culture and the models that educators, parents and policy makers can utilize to place media-savvy youth into positions of purpose, responsibility and power. Two specific challenges are at the core of this book’s argument that media literacy is the path toward more active and robust civic engagement in the 21st century: How can media literacy enable core competencies for value-driven, diverse and robust digital media use? How can media literacy enable a more civic-minded participatory culture? These challenges are great, but they need to be examined in their entirety if media literacy is to begin to address the opportunities they present for democracy, participation and discourse in a digital media age. By presenting information that places media literacy at the center of what it means to be an engaged citizen, educators and policy makers will understand why media literacy must be integrated into formal and informal education systems before it’s too late
... The combination of media and multimodal literacy represents a new era in communication, where individuals are not only consumers but also active contributors to the evolving digital narrative (Lim et al., 2011). In this realm, creativity has limitless possibilities, expanding expression beyond the confines of vocabulary into the vast world of multimedia. ...
Article
This article critically examines the intricate relationship between literacy and multimedia, focusing on multimodal literacies within the framework of educational equity. It navigates intersections among literacy, social-emotional learning, and educational equity, expanding from the analysis of fanfiction as mentor texts to the transformative empowerment of communities through technology integration. Connections between literacy engagement and cultural practices are explored through the lens of enhancing opportunities for English language learning. By tracing the historical roots of fanfiction and scrutinizing its contemporary manifestations in popular culture, this article explores the potential of multimodal literacies. These concepts cultivate literacy skills and foster creativity, empathy, and inclusivity—especially crucial for diverse learners, including those embarking on the journey of English language acquisition. Serving as a comprehensive resource, the article provides nuanced insights into the multifaceted ways in which multimodal literacies can effectively bridge the engagement gap. It offers practical applications, outlining a path towards a more equitable and enriching educational landscape in the digital age, with a specific focus on promoting linguistic proficiency among English language learners. The article stands as a valuable guide for educators, researchers, and practitioners, offering concrete strategies for leveraging multimodal literacies to create inclusive and empowering learning environments.
... Indeed, the emergence of "hybridized discourses" where previously distinct media forms such as information, advertisements, entertainment, editorials, or news become creatively mixed and blended into new forms such as 'docudrama', 'infotainment', 'infomercial' (Fairclough 1996, Rahm 2006, Feng and Wignell 2011, Lim, Nekmat et al. 2011) is another concern for informed choice in contemporary contexts. This type of "interdiscursivity" (Fairclough 1992) means that texts now draw on many different styles of discourse (for example scientific or educational discourse), and employ voices from all walks of life (for example, voices of experts, celebrities, children) within a single text to reinforce the credibility and desirability of the information from different angles. ...
... Indeed, the emergence of 'hybridized' discourses where previously distinct media forms such as information, advertisements, entertainment, editorials, or news become creatively mixed and blended into new forms such as 'docudrama', 'infotainment', 'infomercial' (Fairclough, 1995;Feng & Wignell, 2011;Lim, Nekmat, & Nahar, 2011;Rahm, 2006) is another concern for informedchoice in contemporary contexts. This type of 'interdiscursivity' (Fairclough, 1992) that is, the blending of various genres and purposes, means that texts often draw on many different styles of discourse within a single text (for example, scientific or educational discourse), and employ 'voices' from all walks of life (for example, the voices of experts, celebrities, children) to reinforce the credibility and desirability of a particular activity or organization from different angles (Feng & Wignell, 2011). ...
Article
The provision of information is generally not a technical activity, but rather a contextualized social action. Previous research about informed-choice and decision-making for parents of children who are deaf or hard-of-hearing (POCDHH) has demonstrated this, highlighting the close relationship between contextual factors and the nature of information parents are provided with to support decision-making on behalf of their children. Such contextual factors involve human elements such as attitudes, values and beliefs of individuals involved in the transfer of information, as well as broader contextual factors such as changing information technology and changing markets. This paper reviews literature from a range of fields relating directly and indirectly to issues of informed decision-making for POCDHH. These studies provide an overview of issues such as current understandings of what type of information does (or does not) support decision-making, as well as highlighting the importance of considering how information is presented. Approaches used in other fields to address issues of reliability of information are also discussed. To complement this literature review, the article includes an auto-ethnographic component documenting my own attempts as a POCDHH to garner reliable information on behalf of my daughter within a discursive environment where the material often appeared inconsistent with best practice informed-choice and decision-making principles.
Chapter
In this chapter I consider contemporary global conditions pointing to what some scholars term “a global risk society” where digital media and Cosmopolitan Critical Literacy offer a counterpoint to human rights, health, climate, and terrorist threats. By examining current research in global youth communication across nation-state boundaries via the Internet, existing research suggests that tapping into digital media literacy and critical media literacy will be crucial for developing an informed and critical citizenry. At present, studies of transnational youth navigating old and new affiliations across national borders are in their infancy. Nevertheless, the existing research holds promise for developing global world citizens who can realize an ethos of cosmopolitan, critical citizenship through the affordances of digital media.
Article
Full-text available
This article provides a critical review of the English-language literature on media education. It presents its constitutive dimensions and particularities; it further characterizes media education as a recent and distinct field growing rapidly. This field brings together theoretical knowledge, pedagogical practices, sought-after competencies and policies. These elements are introduced and described. This article also proposes three criticisms of scholarly writings. Firstly, they show a high level conceptual confusion concerning the sets of knowledge and competencies that media education tries to foster. They also struggle to articulate the constitutive dimensions of the field with relation to one another. Finally, the analysis of public policies, particularly of school curricula, is peripheral and incomplete. Knowledge of curriculum content and school-based practices remains underdeveloped.
Article
Full-text available
Despite the increased role of digital curation tools and platforms in the daily life of social network users, little research has focused on the competencies and dispositions that young people develop to effectively curate content online. This paper details the results of a mixed method study exploring the curation competencies of young people in digital culture. Forty-seven college students from two institutions in the north-eastern United States used the social curation platform Storify to curate essays on the topic of income inequality. Their curated stories were coded to explore for narrative development, consistency, sourcing, analysis, and content type. Regression models were used to assess clarity and balance of the curated stories, and a detailed questionnaire explored dispositions towards curation as a relevant and effective mode for engagement in digital culture. The paper argues that curation can enhance core media analysis and storytelling skills, and an understanding about the role of peer-to-peer platforms and collaborative spaces in digital culture. The results advocate the utilization of student- and creation-driven pedagogies that embrace curation as core digital and media literacy competencies for young people in daily life.
Article
Full-text available
In today's hypermedia landscape, youth and young adults are increasingly using social media platforms, online aggregators and mobile applications for daily information use. Communication educators, armed with a host of free, easy-to-use online tools, have the ability to create dynamic approaches to teaching and learning about information and communication flow online. In this paper we explore the concept of curation as a student- and creation-driven pedagogical tool to enhance digital and media literacy education. We present a theoretical justification for curation and present six key ways that curation can be used to teach about critical thinking, analysis and expression online. We utilize a case study of the digital curation platform Storify to explore how curation works in the classroom, and present a framework that integrates curation pedagogy into core media literacy education learning outcomes.
Article
Technological capabilities and features of the Internet and World Wide Web have prompted concerns about the verity of online information, the credibility of new media, and the new responsibilities placed on media consumers. Reflecting these concerns, scholars have shown a renewed interest in the credibility of sources, their messages, and the media that carry them. Nonetheless, researchers who are currently reengaging the issue of information credibility have yet to take full advantage of the rich heritage left by credibility research conducted over the last half century. The primary aim of this chapter is to show how past research can inform present attempts to understand credibility in the new media environment, focusing particularly on Web-based information. Toward that end, this chapter reviews, synthesizes, and integrates the substantial literature on source, message, and media credibility; addresses issues of credibility conceptualization, operationalization, and measurement; suggests strategies to ...
Article
Within both academic and policy discourses, the concept of media literacy is being extended from its traditional focus on print and audiovisual media to encompass the internet and other new media. The present article addresses three central questions currently facing the public, policy-makers and academy: What is media literacy? How is it changing? And what are the uses of literacy? The article begins with a definition: media literacy is the ability to access, analyse, evaluate and create messages across a variety of contexts. This four-component model is then examined for its applicability to the internet. Having advocated this skills-based approach to media literacy in relation to the internet, the article identifies some outstanding issues for new media literacy crucial to any policy of promoting media literacy among the population. The outcome is to extend our understanding of media literacy so as to encompass the historically and culturally conditioned relationship among three processes: (i) the symbolic and material representation of knowledge, culture and values; (ii) the diffusion of interpretative skills and abilities across a (stratified) population; and (iii) the institutional, especially, the state management of the power that access to and skilled use of knowledge brings to those who are ‘literate’
Article
In this 'new media age' the screen has replaced the book as the dominant medium of communication. This dramatic change has made image, rather than writing, the centre of communication. In this groundbreaking book, Gunther Kress considers the effects of a revolution that has radically altered the relationship between writing and the book. Taking into account social, economic, communication and technological factors, Kress explores how these changes will affect the future of literacy. Kress considers the likely larger-level social and cultural effects of that future, arguing that the effects of the move to the screen as the dominant medium of communication will produce far-reaching shifts in terms of power - and not just in the sphere of communication. The democratic potentials and effects of the new information and communication technologies will, Kress contends, have the widest imaginable consequences. Literacy in the New Media Age is suitable for anyone fascinated by literacy and its wider political and cultural implications. It will be of particular interest to those studying education, communication studies, media studies or linguistics.
Book
Television is one of the most important socializing forces in contemporary culture. This book is a cultural history of prime-time television in America during the 1990s. Examines changes that took place in programming, such as the rapid adoption of cable, the proliferation of content providers, the development of niche marketing, the introduction of high-definition television, the blurring of traditional genres, and the creation of new formats like reality-based programming Argues that television programmes of the 1990s afforded viewers a symbolic resource for negotiating the psychological challenges associated with the shift from the Industrial Age to the Information Age Explores the ways in which television provided viewers with tools for coming to terms with their fears about living in the fast-paced , increasingly diverse, information-laden society of the 90s.