Content uploaded by Sun Sun Lim
Author content
All content in this area was uploaded by Sun Sun Lim on Jul 04, 2016
Content may be subject to copyright.
Content uploaded by Sun Sun Lim
Author content
All content in this area was uploaded by Sun Sun Lim on Apr 11, 2015
Content may be subject to copyright.
Multimodal Representation And Knowledge
1
This is a preprint version of: Lim, S. S., Nekmat, E. & Nahar, S. N. (2011). The implications of
multimodality for media literacy. In K. O’Halloran & B. A. Smith (Eds.) Multimodal Studies – Exploring
Issues and Domains (pp. 169-183). London: Routledge.
The Implications of Multimodality for Media Literacy
Sun Sun LIM, Elmie NEKMAT and Siti Nurharnani NAHAR
Abstract
Today’s media consumers can consume, produce and disseminate media messages
involving multimodal representation. Consequently, in both receptive and expressive modes of
communication, multimodal representation demands that media consumers possess a wide range
of media-related knowledge and competencies. While multimodal representation in itself poses
significant media literacy challenges to the media consumer, this chapter asserts that several
concomitant trends in the mediascape further compound the severity of these challenges: the
growing ease of manipulability of media content, the rise in media genre-hybridisation and the
increasing proliferation of user-generated media content. The chapter then argues that media
literacy needs to be reassessed so that greater emphasis be placed on multimodal literacy and
visual literacy. Media consumers should also be vested with the skills to read and navigate the
multimodal and hypertextual environment, and to recognise the limitations of their foundational
knowledge structures. The chapter concludes by making several recommendations for media
organisations, policy makers and media consumers to respond to the media literacy challenges
posed by our increasingly multimodal and rapidly changing mediascape.
Introduction
With the advent of Web 2.0, the media consumer is endowed with the ability to consume,
produce and disseminate media messages often involving multimodal representations which
incorporate text, images and sound. Consequently, in both receptive and expressive modes of
communication, multimodal representation demands that media consumers have knowledge and
competencies in a wide range of aspects – textual understanding, visual and aural literacy, genre
identification, critical analysis, legal know-how, ICT skills, industry insights and more. While
multimodal representation in itself poses significant media literacy challenges to the media
consumer, this chapter argues that several concomitant trends in the mediascape further
compound the severity of these challenges: the growing ease of manipulability of media content,
the rise in media genre-hybridisation and the increasing proliferation of user-generated media
content. The chapter then considers why and how media literacy needs to be reassessed in a
mediascape increasingly marked by multimodality. Finally, it concludes by identifying which
literacies are most critical in our current mediascape and makes several recommendations for
research and policy formulation.
Multimodal Representation And Knowledge
2
Multimodal representation – shifts in the mediascape
The mediascape has seen a discernible shift in semiotic modes of representation towards
a growing dominance of visual images. While unimodal text-only documents have certainly not
faded into oblivion, their importance has waned as media continues to evolve with enhanced
capacities for holding and displaying texts in various modes, containing graphics, pictures,
layout techniques, and more (Goodman 1996). Over the past century, there has been a broad
move from the supremacy of writing and the written word to the dominance of the image in
different media technologies (Jewitt and Kress 2003). These technologies, also known as
‘technologies of literacy’ (Warschauer 2003, 115), had notably shifted from the printing press
which privileged the written word, i.e. text, over all other semiotic modes (Kaplan 1995). Kress
argues that in the current media environment, images are assuming increasing importance such
that images lead over text and the screen takes precedence over books (2003). Kress states that
the screen is a “visual entity” (2003, 65) and text that appears on screen is similarly treated as an
image and follows the same principles of visual design. As a result, written text which appears
with images plays a secondary role with regard to conveying meaning. Arguably, this trend arose
from the popularity of film and television, resulting in the format of print media such as
newspapers, magazines and books being altered, such that visual images have become, and
continue to be, increasingly prominent (Kress and van Leeuwen 1996). Multimodal
representation is therefore not entirely novel in this information age but is a reflection of the
intensifying use of multiple communication modes in media, particularly that of visuals. Notably
though, while television, film and printed texts have long been marrying textual and visual
content, the advent of information technology has made multimodal communication much more
prevalent (Warschauer 2003). On top of combining text, photos, videos, audio and graphics in a
single presentation, the decentralized nature of information production further aids the
proliferation of this multimodal communication phenomenon. In sum, multimodal representation
is by no means a new phenomenon, but one that has evolved in its intensity, scale and
complexity with the introduction of new information and communication technologies.
Multimodality and concomitant trends
While multimodal representation in itself poses significant media literacy challenges to
the media consumer, several concomitant trends in the mediascape further compound the
severity of these challenges: the growing ease of manipulability of media content, the rise in
media genre-hybridisation and the increasing proliferation of user-generated media content. Each
point will be discussed and illustrated as follows.
Multimodality and the manipulability of media
With the advent of digitisation, media and information have become extremely easy to
modify and manipulate. As Feldman (1997) argues, compared to analogue media where the
process of reshaping information can often be “difficult, slow and untidy”, digital media on the
other hand allows users to infinitely and easily alter information “at a stroke” (4). This
Multimodal Representation And Knowledge
3
affordance of advanced digital tools has significant implications for all stages of information
representation: from the moment it is created and captured in digital form, to its dissemination,
on to audiences’ engagement with the information and beyond.
While media content of different modes can be easily modified and altered, in light of the
significance of images in today’s mediascape, let us consider the implications of the ease of
manipulating images. Digital graphics are presently used in creating interactive user interfaces,
virtual reality, animation, as well as reconstructing three dimensional objects from their “2D
projectional presentations” (Groß, 1994, 2). The digitisation of graphics enables us to handle
images in unprecedented ways, including the restoration of old and damaged photographs and
the seamless recombination and morphing of snippets of different images. Such affordances have
been adopted with enthusiasm and exploited by artists and media producers in surprising and
creative ways. However, the ability to modify digital images has also opened an avenue for
misinformation and deception. This concern is not unwarranted considering that doctored images
are being disseminated even by established and reputable media organisations such as Reuters
and the Los Angeles Times. A 2006 Reuters photograph of smoke rising from buildings in Beirut
was attacked by American bloggers for having been doctored (BBC News 2006). Upon
investigation, it was found that Adnan Hajj, the Reuters photographer, had distorted the
photograph to include more smoke and damage. In another example, Brian Walski created a new
photograph by manipulating two photographs which he had taken for the Los Angeles Times
(Van Riper 2003). In the first photograph, a US soldier was pictured with his gun pointed
horizontally while in the second photograph, the soldier’s gun was lowered when a man with a
baby stood near him. By combining both photos, the altered image made it appear as though the
soldier was pointing his gun directly at the man with the baby, presumably to heighten the
photograph’s dramatic effect and to enhance its human interest value. Such egregious practices,
even by professionals from renowned news organisations, are a stark reminder that today’s
media consumer needs to be even more critical and sceptical than ever in their consumption of
media content. In these two cases, the visual literacy skills of the consumer are tested as they
need to appreciate the telltale signs of digitally doctored photographs. Yet the sophistication of
today’s graphic applications produce such flawless results that the visual literacy skills of even
shrewd media consumers would be easily defeated. In such a mediascape, media consumers have
the unenviable responsibility of being constantly questioning and discerning about their media
sources, regardless of how established and reputable might be. Quality indicators that used to
serve media consumers well are of diminished value into today’s media environment.
Quite apart from the manipulability of media itself, the access to media content can also
be manipulated, especially online. The prominence of online information can also be directed
such that some content is replicated repeatedly, while others remain obscure and difficult to
access. Given that the World Wide Web is a seemingly infinite mass of information, media
consumers have no option but to locate information online using search engines. The ways in
Multimodal Representation And Knowledge
4
which search engine results are ordered depend on a combination of factors including the design
of search engines algorithms, the search engine’s revenue stream and business model and the
original source of the content (Hargittai 2004). Commercial interests are behind the most popular
Web sites which users frequently visit to get their online content (Hargittai 2004). The order of
search engine results becomes significant because most users are unwilling to explore results
beyond the third page (iProspect 2008). In 2008 the percentage decreased to 9% as compared to
2006 (12%), 2004 (17%) and 2002 (22%). Hence, media consumers are in some sense at the
mercy of search engines. Search engines and directories systematically exclude certain sites in
favour of others, either by design or by accident (Introna and Nissenbaum 2000). For example,
Google has been observed to exclude certain sites from its searches- compared to google.com,
113 sites were excluded, in whole or in part, from the French google.fr and German google.de
(Zittrain and Edelman 2002). In this regard, the discerning media consumer is one who
understands how media industry practices and pressures may result in the omission of particular
content and the amplification of others, and that the information which they derive from online
sources can be fraught with bias. However, it would be fair to say that such esoteric knowledge
about how search engines work would be beyond the average media consumer.
Multimodality and genre-hybridisation
Practices within the media industry are leaning towards the hybridisation of different
media forms. The classical distinctions between documentaries, news, information,
entertainment, dramas, comedies, editorials and advertisements have become blurred. There has
been a growing trend towards eclecticism, where “a cultural text creatively mixes, blends, or
recombines pre-existing and relatively discrete cultural forms, formulas and techniques” (Ott
2007, 58). Such media production practices result in texts which can be particularly challenging
for media consumers as their conventional frames for understanding media content may be
inadequate or even inappropriate. As Campbell and Freed (1993) opined, “television is certainly
a fertile breeding ground for genre confusion. Categories once chiselled in granite melt in a swirl
of crossover jargon: docudrama, infotainment, infomercial, dramedy (77).” Similarly, with print
media such as newspapers and magazines expanding into online platforms with new channels for
interacting with readers, the classical categories of reportage, opinion-editorials and letters from
readers have been partially displaced by hybrid forms such as readers’ blogs and opinion forums,
‘first-person’ citizen journalist reports replete with amateur photos etc.
The combination of genre hybridisation and multimodality has birthed even more
boundary-crossing media types. The rising sophistication of computer animation and production
techniques has facilitated the creation of multimodal content of an extremely high quality, such
that genre hybridisation in television and films has broken new ground. An excellent example is
the British television programme Prehistoric Park which features the well-known British wildlife
documentary host Nigel Marven playing himself. In this programme, Marven (with the help of a
time travel device) is tasked with finding extinct animals from prehistoric eras and bringing them
back to the present day for exhibition in the Prehistoric Park. The prehistoric animals are
Multimodal Representation And Knowledge
5
rendered in computer generated imagery and animatronics, and they interact with the human
actors and natural landscapes in a lifelike fashion. Marven is often shown spying on the
prehistoric creatures in their ‘natural’ habitats and touching them when they are taken into
captivity. While the characters in the show are thus multimodal in nature, the style of the show is
multi-genre and defies classification. Marven appears in his usual guise of the reputable wildlife
expert who addresses viewers directly, as though they were watching a documentary. The
wildlife scenes are set in actual physical locations and bear the patina of a scientifically-based
nature programme, thus appearing highly realistic. However, Prehistoric Park is mostly fictional
and more closely resembles a drama with the typical elements of emotions, humour, suspense
and even tragedy. This blend of multimodality and genre hybridisation sends mixed signals to
viewers as ‘traditional’ production techniques are both obverted and subverted. A sampling of
viewers’ comments on clips of the show excerpted on Youtube.com reveals the confusion that
some viewers experience – from “it's my favourite documentary tv show (emphasis mine)”, to
“But it looks so real to me, is this real!?”, “so you mean everything in this is real?” and “im (sic)
watching this on tv right now, IS THIS REAL OR FAKE !?!??!?!?!” (Sidewaysnic, n.d.). As
these comments exemplify, the potent combination of multimodality and genre-hybridisation can
significantly test consumers’ media literacy skills.
Multimodality and the proliferation of user-generated content
Information technology now provides media consumers with the means to create,
replicate and disseminate media content. With the spread of affordable media production
hardware and software, and the emergence of a slew of content sharing sites on the Internet, the
growth of user generated content has been significant. Previously, the high costs of information
production and dissemination served as barriers and restricted the number of content providers to
only those with adequate authority and capital (Metzger 2007). Today, as long as one has the
technical know-how and access to the technology, one can easily become an author. We should
bear in mind of course, that the extent to which individuals can and do avail of such self-
authoring opportunities differs by motivations, skills and interests.
In short, media consumers are now able to produce their own forms of representation,
both uni- and multimodal. This proliferation of user generated content compounds the
subjectivity of the information which they create as each different mode offers the potential for
different “representational and communicational action by their users” (Kress 2003, 5). The
average media consumer is now concurrently in possession of the resources of representation, the
resources of production and the resources of dissemination. These distinct resources require
specific competencies not only in their use but also in the design of information, i.e. in the
consumer’s receptive and expressive modes (Kress 2003). In other words, it is more important
than ever for the average consumer to understand the role of the designer with regards to the
meaning-potentials of the resources which they now enjoy. This is further complicated by the
interplays between the ‘semiotic resources’ (i.e. mode) and the ‘material substance’ (i.e.
Multimodal Representation And Knowledge
6
medium) (Kress and van Leeuwen 2001, 215, 41) which provide various potentialities, but also
present complexities in the representation of information.
Critical reflexivity is also required in view of the synergies and potential for convergence
amongst different forms of digital media such as computers, mobile phones, cameras, palmtops
and many more. The increasing portability of these media tools, coupled with their enhanced
capacity to hold different modes of information greatly increases one’s ability not only to
produce, but also to communicate information through various modes and resources. Inevitably,
media consumers’ exposure to multimodal forms of communication will only increase.
Therefore, possessing the competencies to comprehend how the various modes and ‘material
resources’ affect the representational potential of information not only requires critical
reflectivity of the context in which the information was produced, but also reflexivity in one’s
own production of information.
Reassessing Media Literacy
Media literacy can be defined as the ability to ‘decode, evaluate, analyze and produce
messages’ in a variety of forms (Aufderheide 1993, 1). While this pithy definition adequately
captures the multi-faceted nature of media literacy, the way in which media literacy is defined
depends not only on the entity defining it but more significantly, should change to suit the
evolution of the media landscape (Potter, 2004). Indeed, in light of the trends discussed in the
previous section, a reassessment of the concept of media literacy is timely. In this regard, what
are the implications of multimodality for media literacy, especially when it comes to one’s
critical analysis and evaluation of media content? Several scholars have identified several media
literacy imperatives arising from the increasingly complex nature of multimodal representation.
The need for multimodal literacy and multiple literacies
Accompanying the proliferation of multimodal representation in today’s media landscape
have been shifts in how meanings are created and understood (Jewitt and Kress 2003; Lankshear
and Knobel 2003). With the widespread deployment of different modalities, media and materials,
each with its own logic and affordances, media consumers’ meaning-making processes are
getting more complex than ever. Understandably, scholars concerned about the effects of
multimodality have focused on understanding the different ways in which meaning can be
created and communicated in the world today (Kress and van Leeuwen 1996; Baldry 2000; O’
Halloran 2004). They have focused, inter alia, on the ‘semiotic affordances of image, of writing
and of speech and of multimodal texts’ (Jewitt and Kress 2003, 166) and on how ‘intra and inter-
semioses' arising from the interaction within and between two or more semiotic modes empower
or disempower creators and receivers of multimodal texts (O’ Halloran 2004, 224).
The multimodally-literate media consumer is primarily viewed as one who: 1) displays a
systematic understanding of how texts make meanings and how these meanings can be conveyed
by different communicative forms such as language, image, sound, gesture, etc.; 2) possesses the
Multimodal Representation And Knowledge
7
competency to integrate textual analysis with an appreciation of how audiences engage with the
texts under scrutiny; and 3) has the capacity to integrate textual analysis with the political,
economic and social contexts in which the texts are produced and consumed (Burn and Parker
2003, 3-4). As compared to multimodal literacies that emphasise the ability to critically decipher
the meaning-making potential of semiotic resources, media literacy focuses on the skills to
access, consume, assess and produce content (Livingstone 2004), with a shift “towards a concern
with critique, reflection and judgement” (Martin 2006, 18). The ability to critically analyze
symbolic texts thus lies at the intersection of multimodal literacy and media literacy and a robust
definition of media literacy that serves today’s mediascape has to take into account
multimodality and incorporate multimodal literacy. Clearly, the range of literacies that comprise
media literacy has widened. There have been calls for ‘new literacies’ and ‘multiple literacies’
for understanding ‘post-typographic’ forms of social practices in the consumption and
production of media content (Lankshear and Knobel 2003, 16-17). It is only with the possession
of multiple literacies that individuals can participate in our highly-mediatised information society
in an efficacious manner (Kellner 2002). A few key literacies are discussed in the subsequent
sections.
Reading and navigating the multimodal and hypertextual environment
While the current multimodal and hypertextual media environment seems to exemplify a
‘brave new world’ in media representations, the basic principles of critical literacy which applied
to traditional print and mass-media contexts are still relevant today. For example, the ability to
critically appraise the heterogeneity of sources, competing authorities, non-linear or visual forms
of representation are not skills which are specific to the multimodal media environment, but have
long been required for consuming text (Livingstone 2003). However, Luke (2000) does refine
the definition of particular literacies which will enhance one’s navigation of hypertextual
environments and evaluation of online information. They are: 1) possessing the adequate meta-
knowledge of how ideas and information ‘bits’ are structured in different media genres and how
they affect people’s reading and uses of information; 2) displaying mastery of the technical and
analytical skills with which to negotiate those representational systems in diverse contexts; and
3) having the capacity to understand and relate these systems and skills as operating within
relations and interests of power within and across social institutions (72).
Jewitt and Kress (2003) also identified salient distinctions between the unimodal, text
environment and the multimodal, hypertextual one. They argue that the former tends to have
horizontal and linear content while the latter ‘immerses one in an intertextual and multimodal
universe of visual, audio, symbolic and linguistic meaning systems’ (73) which are laterally
connected, thus making reading and navigation more challenging. Readers are engaged in a
multimodal reading of texts, and have the added burden of navigating through multi-layered and
multi-coded animation, symbols, linguistic text, photos etc at the same time. In such an
environment, having a contextualised knowledge of the ideas carried by different pieces of
information will no longer suffice. Instead, media consumers are required to understand the
Multimodal Representation And Knowledge
8
relations amongst these ideas and how these ideas are affected by their representation through the
different modes. Furthermore, one also has to be aware of the effects resulting from the process
of following a hyperlink from one webpage to another, with each different page embedded with
different modes of representation. For example, reading about the large number of whales being
driven up to shore elicits a different response as compared to seeing images of the whales on the
shoreline. This process potentially affects not only their cognitive, but also their emotive states,
affecting their ability to make sound judgements about the credibility of the information which
they presented. The ability to read and navigate the multimodal and hypertextual media
environment is thus a key component of the afore-mentioned ‘multiple literacies’ required in
today’s mediascape.
Recognising the limitations of foundational knowledge structures
The critical evaluation of media content rests on an extensive body of knowledge
pertaining to the broader social, cultural, economic, political, and historical contexts in which
media content is produced (Bazalgette 1997). Apart from such contextual knowledge, Potter
(2004) posits a more comprehensive set of foundational knowledge structures that a media
literate person needs to possess for the critical evaluation of information. They are, knowledge
of: 1) media content 2) media effects, 3) media industry, 4) the real-world, and 5) self (Potter
2004). These knowledge structures enable media users to approach problem-solving and
meaning-making with a greater variety of tools. However, the robustness of these knowledge
structures is increasingly challenged by the multimodal nature of online communication, as well
as the three concomitant trends discussed above, i.e. the manipulability of media, media genre-
hybridisation and the emergence of user-generated content. Compared to traditional publishing,
content posted on the Internet may not be filtered through professional gatekeepers, and not all
websites specify traditional authority indicators such as author’s identity or affiliated association.
There are no common standards for posting information online, and digital information may be
easily changed, misused, plagiarized, or created anonymously under false pretences (Fritch and
Cromwell 2001, 2002; Metzger, Flanagin, Eyal, Lemus, and McCann 2003). In addition, since
user generated content can and often is presented in a format similar to that of established media
organisations, Burbules (1998) argues that there is a “levelling effect” where all information
becomes equally easy to access, thus contributing to the perception that all authors offer the same
level of credibility to Internet consumers (109).
In such circumstances, how relevant is one’s knowledge of media effects or media
industries when evaluating information sources? For example, is it sufficient to claim that
information presented by a trustworthy online news agency is more reliable than an independent
source which posts photographs of a similar event on a blog and even includes embedded video
footage of the actual event? In this instance, the modes of representation might actually be more
revealing indicators of information reliability than knowledge about the media producer or media
industry as a whole. Furthermore, the increasingly hazy divisions between media producers and
consumers makes it difficult to establish valid and reliable criteria for ascertaining the quality,
Multimodal Representation And Knowledge
9
ideology, market influences or professional production values of online content. One emerging
grey area is the incorporation of user-generated content by mainstream media organizations
seeking to ride the wave of citizen journalism. Viewers and readers are encouraged to contribute
content which then becomes embedded within the company’s proprietary material. For example,
readers’ and viewers’ contributions are regularly showcased in Korea’s OhMyNews, Singapore’s
Straits Times’s STOMP and CNN’s iReporter, whereas BBC News online currently picks up
more Internet traffic from micro-blogging site Twitter than it sends there (Hitwise, 2008). Yet
these organisations also issue disclaimers to absolve themselves from errors and inaccuracies in
these contributions from their audiences, clearly signalling that incorporation of user-generated
content is not tantamount to editorial endorsement.
In such a media milieu, the keystone of critical media literacy, i.e. the knowledge of the
operations and consequences of media producers as prime quality indicators, is no longer as
valid as it used to be (Livingstone 2004). Hence, while media consumers need to maintain and
grow their media-related knowledge structures, another literacy which they must possess is the
ability to recognise the limitations of these structures, and to deploy them in a reflexive manner
that suits the ever-changing contextual demands.
Enhancing visual literacy
The growing dominance of visuals as a form of representation also necessitates an
increased focus on what it means to ‘read’ images from the media. Kress (2003) stresses the
importance of understanding the logic of new reading paths, where the image ‘dominates the
semiotic organization of the screen’ (65). Other scholars and experts on visual literacy have also
emphasized the importance of understanding the effects of ‘representational conventions’ of
visual images used in media for the creation and sharing of meanings (Messaris 1998, 70).
Arguably, visual representations have the potential to imbue information with an aura of
reliability and a veneer of truthfulness. Hence, visually literate media consumers are also those
who can interpret the content of visual images, examine their social impact, and evaluate the
purpose, audience and ownership of visuals (Bamford 2003). Besides the new logics of reading
required in the highly visual media environment and the critical reflection skills needed for the
contextual appraisal of visual images, the competency to ascertain the authenticity and reliability
of visual images in media is especially vital in today’s multimodal environment. Therefore,
visual literacy is yet another literacy in the repertoire of literacies which today’s media
consumers should possess.
Conclusion and recommendations
The nature, intensity and scale of multimodality in today’s media content pose interesting
and potentially daunting challenges for consumers’ media literacy skills. The multimodal
environment necessitates that consumers be even more critical in evaluating media content
information now, more than ever, since the onus of information credibility no longer rests mainly
on traditional gatekeepers. Associated trends such as the growing ease of manipulability of
Multimodal Representation And Knowledge
10
media content, the rise in media genre-hybridisation and the increasing proliferation of user-
generated media content also serve to heighten the magnitude of these media literacy challenges.
In this regard, what can media organisations, public agencies and individuals themselves do to
confront these challenges or, at the very least, to ameliorate their effects?
Media organisations should take the initiative to provide evidence of reflexivity on their
part. In particular, they should be more transparent about their media production processes and
policies. Media consumers may then have a set of ‘environmental standards of media practice’
(Silverstone 2007, 176) on which to base their own judgements. Media organisations can also
seek to introduce their own quality indicators which serve as guidelines on how media content
can be read, understood and criticized. A good example is online encyclopaedia Wikipedia
which practises collaborative writing and editing of articles by volunteers who source, format,
rewrite and link articles. All Wikipedia content must strive to adhere to the policies of Neutral
Point of View, Verifiability and No Original Research (Ayers, Matthews, and Yates 2008, 200).
Hence, entries must be objective, have their claims supported with reliable sources and be based
on content previously published by third parties. Entries which do not meet these benchmarks are
flagged with standard message templates to warn readers of the inadequacy of the content. More
information providers would do well to emulate Wikipedia’s establishment of transparent
standards and its user-friendly style of communicating them. Similarly, search engines should
explain more clearly how their search results are derived and clearly differentiate sponsored hits,
so that consumers can be more well-informed in their use of search engine results (Machill
2004). Google has been a trailblazer in this regard, where sponsored links are clearly
demarcated. Measures such as those taken by Wikipedia and Google will help to raise public
confidence in media organisations and the content which they provide. As media organisations
seek to pursue market share and industry recognition, introducing such measures need not be
onerous but can be acts of enlightened self-interest. However, it should also be cautioned that
these measures and reliability indicators are double-edged as media consumers may go into
‘auto-pilot’ mode and make snap judgments about information credibility on the basis of these
assurances.
Policy interventions are also urgently required to help people keep pace with ever-
changing media trends so that they can continue to function well in a highly mediatised society.
On a public policy level, there is a need to prioritise which components of media literacy need to
be most urgently inculcated so that citizens can avail of new media opportunities to maximise
benefits and minimise harms. Training in multimodality needs to be built into the formal school
curriculum from an early age. To assume that young media consumers, popularly referred to as
the digital natives, are well-versed in multimodal content would be a mistake. They may be
highly attuned to the functional aspects of the multimodal, multi-media environment. However,
their ability to consume media in a critical and discerning fashion may be wanting, as several
studies suggest (see for example Hobbs and Frost, 2003; Livingstone and Bober, 2004; Shenton,
Multimodal Representation And Knowledge
11
2004). Hence, it is critical that training in media literacy, and multimodal literacy be
incorporated into the formal curriculum as early as possible, particularly since these children
would already have grown up in an environment where multimodal representation is a given.
Furthermore, multimodal media literacy training should focus on technical competencies and
critical discernment concurrently, rather than in isolation, for better results (Potter 2008). To this
end, Kalantzis and Cope (2000) proposed a ‘pedagogy of multiliteracies’ to instil an appreciation
for the multimodal media environment, comprising: 1) Situated practice – working from a base
of the student’s personal interests and life experience; 2) Overt instruction – comparing and
contrasting different patterns and conventions of meaning in varied cultural settings; 3) Critical
framing – critiquing and contextualizing information and messages; and 4) Transformed practice
– transferral of learning from one context to another, and putting theory to practice (239-242).
Such a holistic, life-centred approach better reflects the realities of the current media
environment where multimodality is ubiquitous.
As for the media literacy skills which need to be imparted to prepare individuals for the
multimodal environment, special attention must be paid to critical literacy in both receptive and
expressive modes so that people can be discerning media consumers as well as producers. In
Singapore for example, the trend is towards the imparting of functional literacy skills e.g.
teaching people how to blog, with less attention being paid to critical literacy dimensions, e.g.
helping people to understand the impact of blogging (Lim and Nekmat, 2009). In this regard,
programmes tailored to inculcate critical ‘prosumption’ of media content need to emphasise that
the semiotic democracy and multimedia affordances which come with the production of media
content have personal and societal impact, as well as legal implications.
In light of the fact that the mediascape is in a constant state of flux, media literacy
education must not end with the formal school years. Continuing adult education is also essential
so that working adults can keep abreast of the latest media trends which may well impact on their
self-efficacy, workplace productivity and sense of well-being. To this end, media literacy
programmes targeted at adults should capitalise on trends in informal learning (see for example
Drotner 2008). For example, fostering a core group of technology evangelists or ambassadors
who engage in peer-to-peer teaching would be one approach. Public resources should also be
enhanced for self-instruction and independent learning in the acquisition of media literacy skills.
Online portals, public information booths and community libraries can be key nodes for
disseminating such resources. Ultimately, policy makers need to realise that the repertoire of
media literacy skills which individuals require will constantly increase in number and change in
composition. Media literacy programmes must therefore be tailored accordingly and refreshed
constantly in today’s rapidly progressing multimodal and multimedia landscape.
What then of media consumers themselves? Above all, they need to come to terms with
the mercurial nature of the mediascape, and recognise that heuristics for understanding and
Multimodal Representation And Knowledge
12
assessing media content are being rapidly superseded. While the instinct to surrender helplessly
to these apparently inexorable trends is great, media consumers should still seek to equip
themselves with the competencies to critically and profitably access, consume and produce
media content. Indeed, it can never be sufficiently stressed that the responsibility ultimately lies
with media consumers to be conscious of the limitless possibilities and potential pitfalls in our
multimodal media environment.
References
Aufderheide, P. 1993. Media Literacy: A report of the national leadership conference on media
literacy. Aspen, CO: Aspen Institute.
Ayers, P., C. Matthews, and B. Yates. 2008. How Wikipedia Works and how you can be part of
it. San Francisco, CA: No Starch Press.
Baldry, A. P., ed. 2000. Multimodality and multimediality in the distance learning Age. Ed. A.
P. Baldry. Campobasso, Italy: Palladino Editore.
Bamford, A. 2003. The visual literacy white paper. UK: Adobe Systems Incorporated.
http://www.adobe.com/education/digkids/resources/visual_literacy.html (accessed
November 22, 2008).
Bazalgette, C. 1997. An agenda for the second phase of media literacy development. In Media
literacy in the information age: Current perspectives, information and behaviour, ed. R. W.
Kubey, 69-78. New Brunswick, NJ: Transaction.
BBC News. 2006. Reuters drops Beirut photographer. BBC News, August 08.
< http://news.bbc.co.uk/2/hi/middle_east/5254838.stm> (accessed August 29, 2008).
Burbules, N. C. 1998. Rhetorics of the Web: Hyperreading and critical literacy. In Page to
screen: Taking literacy into the electronic era, ed. I. Snyder, 102–122. London: Routledge.
Burn, A. and D. Parker. 2003. Analysing Media Texts. London: Continuum International
Publishing.
Campbell, R. and R. Freed. 1993. 'We Know It When We See It’: Postmodernism and
Television. Television Quarterly 26:75–87.
Drotner, K., H.S. Jensen and K.C. Schrøder. 2008. Informal learning and digital media.
Newcastle, UK: Cambridge Scholars.
Feldman, T. 1997. An Introduction to Digital Media. New York, NY: Routledge.
Multimodal Representation And Knowledge
13
Fritch, J. W., and R. L. Cromwell. 2001. Evaluating Internet resources: Identity, affiliation, and
cognitive authority in a networked world. Journal of the American Society for Information
Science and Technology 52(6): 499–507.
Fritch, J. W., and R. L. Cromwell. 2002. Delving deeper into evaluation: Exploring cognitive
authority on the Internet. Reference Services Review 30 (3): 242–254.
Goodman, S. 1996. Visual English. In Redesigning English: New Texts, New Identities, ed. S.
Goodman and D. Graddol, 38–72. London: Routledge.
Groß, M. 1994. Visual Computing: The Integration of Computer Graphics, Visual Perception
and Imaging. Berlin: Springer-Verlag. Hargittai, E. 2004. The Changing Online Landscape:
From Free-for-All To Commercial Gatekeeping. In Community Practice in the Network
Society: Local Actions/Global Interaction, ed. P. Day and D. Schuler, 66-76. New York:
Routledge.
Hitwise. July 2008. “Hitwise UK Social Networking Update”. http://sg.hitwise.com/press-center/
hitwiseHS2004/uk-facebook-22072208.php (accessed December 3, 2008).
Hobbs, R., and R. Frost. 2003. Measuring the acquisition of media-literacy skills. Reading
Research Quarterly 38(3): 330-355.
Introna, L. D., and H. Nissenbaum. 2000. Shaping The Web: Why The Politics of SearchEngines
Matters. The Information Society 16: 169-185.
iProspect. April 2008. Blended Search Results Study. http://www.iprospect.com/premiumPDFs/
researchstudy_apr2008_blendedsearchresults.pdf (accessed September 28, 2008).
Jewitt, C., and G. Kress. 2003. Multimodal Literacy. London: Peter Lang.
Kalantzis, M., and B. Cope. 2000. A Multiliteracies Pedagogy: A pedagogical supplement. In
Multiliteracies: Literacy learning and the design of social futures, ed. M. Kalantzis and B.
Cope, 239-248. NY: Routledge.
Kaplan, N. 1995. E-literacies. Computer-mediated Communication Magazine 2 (3): 3-35.
Kellner, D. 2002. Technological revolution, multiple literacies, and the restructuring of
education. In Silicon Literacies, ed. I. Snyder, 154-169. London: Routledge.
Kress, G. 2003. Literacy in the New Media Age. New York, NY: Routledge.
Kress, G., and T. van Leeuwen. 1996. Reading images: the grammar of visual design. London:
Routledge.
Multimodal Representation And Knowledge
14
Kress, G., and T. van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of
Contemporary Communication. London: Arnold Publishers.
Lankshear, C., and M. Knobel. 2003. New Literacies: Changing knowledge and classroom
learning. Philadelphia, PA: Open University Press.
Lim, S. S., and E. Nekmat. 2009. Media Education in Singapore –New media, new literacies? In
Media Education in Asia, ed. Cheung, C. K., Netherlands: Springer. (In press).
Livingstone, S. 2003. The Changing Nature and Uses of Media Literacy. European
Medi@Culture-Online. http://www.lse.ac.uk/collections/media@lse/pdf/Media@lseEWP4_
july03.pdf (accessed September 27, 2008).
Livingstone, S. 2004. Media Literacy and the Challenge of New Information and
Communication Technologies. The Communication Review 7(1): 3-14.
Livingstone, S., and M. Bober. 2004. UK children go online: surveying the experiences of young
people and their parents. Monograph. July. http://eprints.lse.ac.uk/395/. (accessed July 6,
2008).
Luke, C. 2000. Cyber-schooling and technological change. In Multiliteracies: Literacy Learning
and the Design of Social Futures, ed. B. Cope and M. Kalantzis, 69-91. New York:
Routledge.
Machill, M., C. Neuberger, W. Schweiger, and W.Wirth. 2002. Navigating the Internet: A Study
of German-Language Search Engines. European Journal of Communication 19: 321-347.
Martin, A. 2006. Literacies for the digital age: A preview of part 1. In Digital Literacies for
Learning, ed. A. Martin and D. Madigan. 3-26. London, Ridgmount: Facet Publishing.
Messaris, P. 1998. Visual aspects of Media Literacy. Journal of Communication 48, (1): 70-80.
Metzger, M. J. 2007. Making Sense of Credibility on the Web: Models for Evaluating Online
Information and Recommendations for Future Research. Journal of the American Society for
Information Science and Technology 58 (13): 2078-2091.
Metzger, M. J., A. J. Flanagin, K. Eyal, D. R. Lemus, and R. M. McCann. 2003. Credibility for
the 21st century: Integrating perspectives on source, message, and media credibility in the
contemporary media environment. Communication Yearbook 27 (1):293-335.
O’ Halloran, K. L., ed. 2004. Multimodal Discourse Analysis. Ed. K. L. O’ Halloran. NewYork:
Continuum.
Ott, B. L. 2007. The Small Screen: How Television Equips Us to Live in the Information Age.
London: Blackwell.
Multimodal Representation And Knowledge
15
Potter, W. J. 2004. Theory of Media Literacy: A Cognitive Approach. Thousand Oaks: CA: Sage
Publications Inc.
Potter, W. J. 2008. Media Literacy 4th Edition. LA: Sage Publications.
SidewaysNic, "Prehistoric Park," YouTube video, 2:26, from a performance televised by Animal
Planet, added May 22, 2007, http://www.youtube.com/watch?v=hp-XPlj67As.
Shenton, A. K. 2004. Research into young people's information-seeking: perspectives and
methods. ASLIB Proceedings 56, no. 4: 243-254.
Silverstone, R. 2007. Media and Morality: On the Rise of the Mediapolis. London: Polity.
Van Riper, F. 2003 .“Manipulating Truth, Losing Credibility”. The Washington Post, April 09.
<http://www.washingtonpost.com/wp-srv/photo/essays/vanRiper/030409.htm> (accessed
September 19, 2008).
Warschauer, M. 2003. Technology and Social Inclusion: Rethinking the Digital Divide. London,
England: The MIT Press.
Zittrain, J., and E. Benjamin. 2002. Localized Google search result exclusions: Statement of
issues and call for data. Berkman Center for Internet & Society, Harvard Law School,
October 26. <http://cyber.law.harvard.edu/filtering/google/> (accessed September 29, 2008).
Author(s) mailing addresses
Sun Sun, LIM, (Corresponding author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-11, Singapore 117589
Phone: 6516-1175 Fax: 6779-4911
Email: cnmlss@nus.edu.sg
Elmie, NEKMAT (co-author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-15, Singapore 117589
Phone: 6516-8224 Fax: 6779-4911
Email: cnmmen@nus.edu.sg
Siti Nurharnarni, NAHAR (co-author)
Communications and New Media, Arts and Social Sciences, National University of Singapore. Blk AS6,
11 Law Link, #03-15, Singapore 117589
Phone: 6516-3019 / 9876-1217 Fax: 6779-4911
Email: cnmsnn@nus.edu.sg