Content uploaded by Michele Zappavigna
Author content
All content in this area was uploaded by Michele Zappavigna on Oct 27, 2021
Content may be subject to copyright.
A social semiotic perspective on emoji 1
PRE-PROOF VERSION ONLY. PLEASE EMAIL IF YOU NEED THE PUBLISHED
VERSION WITH PAGE NUMBERS. SOME EMOJI MAY NOT APPEAR
CORRECTLY IN THIS VERSION.
A social semiotic perspective on emoji:
How emoji and language interact to make meaning in digital messages
1. Introduction
There is currently extensive interest in linguistics and communication studies in the role
that non-linguistic features such as emoji play in digitally mediated communication. Emoji are
widespread in social media discourse, involved in the expression of emotion, conveying
stances, and negotiating interpersonal alignments. This study draws on the theoretical
orientation of social semiotics to explore the kinds of meanings that can be made as emoji
interact with language (the linguistic ‘co-text’) in a social media post or digital message.
Emoji are a subcategory of what Herring and Dainas (2017) classify as ‘graphicons’ (graphical
icons) – a classification of graphical features found on social media platforms that also includes
emoticons, stickers, GIFs, images, and videos. Emoji are small images represented using code
points (a numerical code assigned to each character) in the Unicode Standard. While the first
recorded case of emoticon use occurred in 1982 on the Carnegie Mellon bulletin board, emoji
were created in Japan in 1997. Since then, emoji have been widely adopted by computer-
mediated-communication users: in 2016 almost 50% of messages sent via messaging apps
contained emoji (Emogi Research Team, 2016). As emoji have become more popular, users
have sought to expand their meaning potential and the Unicode Consortium has accepted many
proposals to expand the emoji catalogue with more varied experiences, such as representation
of a greater variety of skin tones, genders, family types, regional foods etc, although the skin
A social semiotic perspective on emoji 2
tone modifiers have been critiqued as enacting “an institutionalized form of colour-blind racism
which insists that concerns regarding racial representation and identity are irrelevant to
“neutral” technical systems and workplaces” (Miltner, 2020, p.517). In 2021, the Unicode
Standard contained over 3,300 emoji. Emoji thus constitute a vital area of linguistic
investigation, both due to their increasing usage and to their evolving meaning-making
potential.
In seeking to advance our understanding of emoji, this paper elaborates and applies an
analytical framework for exploring how emoji interact with language. Applying this framework
to the dataset suggests both typical meanings realised by specific emoji and the high degree to
which emoji meaning is context-sensitive. In regard to the latter point, this study applies Zhao’s
(2011) minimum mapping hypothesis to interpret how emoji interact with language, proposing
that the meaning jointly construed is the area of shared meaning potential across
communicative modes.
The analysis presented in this paper is underpinned by the theoretical framework of
Systemic Functional Linguistics (SFL). Relevant principles of SFL will be outlined in Section
3, however a brief glossary of terms used in the paper is provided here. These include the three
metafunctions conceptualised in SFL theory: the ideational metafunction, concerned with
experiential meaning, the interpersonal function, concerned with enacting relationships, and
the textual function, describing resources for organising meaning into a coherent text (Halliday,
1978); and the systems that describe attitudinal meaning (ATTITUDE): AFFECT (emotional states)
APPRECIATION (aesthetic evaluations) and JUDGEMENT (social/moral evaluation). Lastly, ‘co-
text’ refers to semiotic resources occurring within the same text.
2. Linguistic Research on emoji
Semiotic research on emoji to date has evolved from focusing on their attitudinal
A social semiotic perspective on emoji 3
meaning to recognising their semantic, pragmatic and phatic function within texts (cf. Danesi,
2016), often drawing parallels with how gesture contributes to semiosis (cf. Gawne &
McCulloch, 2019). Outcomes of this work relevant to this study will be reviewed below. An
alternate avenue of enquiry has investigated the sociolinguistic features of emoji, exploring
how emoji use is distributed across text genres, and by age, gender, race and cultural context
(see Miltner, 2020; Albawardi, 2018; Herring & Dainas, 2020; Nishimura, 2015). This is
undoubtedly a rich and valid field of enquiry; however its results are not immediately relevant
to this study, thus it will not be reviewed in any detail.
2.1. Pragmatic approaches
A primary issue to address in reviewing literature on pragmatic approaches to emoji
semiosis is the relationship between emoji and emoticons. Emoticons and emoji are distinct
types of graphicons; however, the terms have been used interchangeably, possibly as a
reflection of a common software feature that automatically converts some emoticons into emoji
(e.g. in Microsoft Word). While the richer pictorial affordances of Unicode afford emoji a
greater meaning potential than is available to ASCII emoticons, research which discusses
emoticons is nevertheless relevant to this study. This reflects the functional, intermodal
approach taken here, which follows Albert (2020) in grouping different categories of
graphicons as semiotic resources interacting with language.
As surveyed in Dresner and Herring (2014), early work on emoticons (Rezabek &
Cochenour, 1998; Walther & D’addario, 2001) described their semiotic function as constrained
to affective meaning, a view summed up in Crystal’s description of emoticons as
“combinations of keyboard characters designed to show an emotional facial expression” (2001,
p. 36). More recent work disputes this definition, arguing that, further to attitudinal meaning,
emoji construe phatic and pragmatic functions. Primary exponents of this view are Dresner and
Herring, who point to the multiple, context-dependent functions emoticons can realise, and
A social semiotic perspective on emoji 4
challenge the assumption that emoticons can make meaning independent of co-text. In their
(2017) work, Herring and Dainas identify six main functions of graphicon use in Facebook
comments: “reaction, tone modification, riffing, action, and narrative sequence” (2017, p.
2185).
Dresner and Herring (2017) is among the numerous works that apply a pragmatic lens
to emoji analysis. Sampietro (2016, p. 109) argues that emoji are used by writers to “align with
the interlocutor, to express informality or to enhance phatic communion and expressive speech
acts, especially greetings.” Focusing on how sarcasm is realised in written communication,
Thompson and Filik (2016, p. 116) observe that “tongue face, wink face, and ellipsis all
occurred significantly more frequently with sarcastic than literal comments”. Similarly,
Skovholt, Grønning, and Kankaanranta (2014) observe that emoticons are used as markers of
humorous intent and irony, and that they can be used to hedge accompanying speech acts, such
as face-threatening and directive acts. Another avenue of pragmatic enquiry has investigated
how emoji and emoticons can be used as discourse markers akin to punctuation, but that can
simultaneously realise other kinds of meaning (cf. Na’aman et al., 2017; Sampietro, 2016;
Dresner & Herring, 2010; Provine et al., 2007). These findings are confirmed by work on emoji
within the field of grapholinguistics (Dürscheid & Siever, 2017).
Danesi’s (2016) work marshals various strands of phatic, pragmatic and semantic
research to address the question of whether emoji constitute an independent language,
ultimately concluding that they do not (yet). Danesi distinguishes adjunctive emoji, that make
meaning alongside language, and substitutive emoji, that replace language, and observes that
emoji intelligibility declines in correlation to its independence from language. Danesi also
emphasises that unlike language, emoji interpretation does not require any formal instruction.
In describing emoji semiosis, Danesi foregrounds their ‘adding tone’ function, which is
congruent with the ‘tone modification’ function described by Herring & Dainas (2017).
A social semiotic perspective on emoji 5
2.2. Context-sensitive approaches
While work exploring how emoji make meaning independently of language has yielded
valuable insights, with sustained, widespread use, emojis’ specific pragmatic meanings may
have become diluted (Konrad, Herring & Choi, 2020). In response, some scholars have adopted
analytical frameworks that aim to describe how emoji make meaning in combination with
linguistic resources. In this vein, grapholinguistic research on emoji advocates context-specific
interpretation of their meaning (Dürscheid & Meletis, 2019). Accordingly, Dürscheid and
Meletis state, “this renders the emoji graphematically ambiguous, as the specific linguistic unit
it refers to is not fixed but variable and determined by the context or the reader’s interpretation
of a given text in which it is used” (2019, p.174). Similarly accommodating co-occurring
language into their analytical framework, Ge and Herring (2018) adapt rhetorical structure
theory to analyse how emoji sequences relate to their accompanying co-text rhetorically and
logically, concluding that in some respects, emoji appear to be “developing into an independent
language” (p. 15).
A related vein of argument that runs through analysis of emoji and emoticons is the
likening of these resources to gesture. At a surface level, this is motivated by the popularity of
emoji which iconically represent facial expressions and body gestures (i.e.: ‘Thumbs Up’
i
!"#$,
‘OK Hand’ !%&'). Describing the relationship between emoji and gesture, Dresner and Herring
(2014, p. 66) comment that their analysis “does not rule out an iconic mapping between the
function of emoticons and some bodily and facial movements”. However, a direct equivalence
of emoji and gesture risks proscription of emoji’s meaning potential – as Albert observes, “the
formal analogy between emoji faces in general and the corresponding facial expressions
provokes the misleading inference that there must also be a functional analogy” (2020, p. 68).
A more nuanced parallel drawn between these modes is that like co-speech gesture, the
A social semiotic perspective on emoji 6
meaning made by emoji is to varying degrees dependent on their linguistic co-text. This
dependent relationship to language prompts Abercrombie’s (1968) designation of modes such
as co-speech gesture as ‘paralinguistic’, a term we adopt here. Employing McNeill’s (1992)
diagnostic criteria for determining the degree to which semiotic modes can function
independently of language, Gawne and McCulloch observe that “gestures and co-speech emoji
are closely integrated into meaning with the accompanying speech/text” (2019, p. 13). They
argue that unlike language, emoji are global and synthetic, non-combinatoric, context-sensitive
and do not have standards of form. In so doing, they integrate pragmatic/phatic/attitudinal
findings of earlier work into a single model for emoji semiosis.
In treating emoji as a paralinguistic modality and allowing interpretation of their
meaning to be guided by co-text, Gawne and McCulloch’s work aligns with a social semiotic
approach such as that adopted by [Author B] and Parkwell (2019). This approach maps how
emoji realise meaning across the three metafunctions described in the social semiotic theory of
SFL. Accordingly, [Author B]’s analysis of emoticons used in Twitter posts described their
function of construing (generally positive) interpersonal meaning among interactants, but also
noted their textual function as discourse markers. In line with social semiotic work on
intermodal semiosis (cf. Martin, 2008), [Author B]’s analysis interpreted emoticons in context,
observing that “Viewed alone, emoticons display a high degree of ‘fuzziness’… interpersonal
meaning is more readily studied when we look at how emoticons work in tandem with
evaluative meanings made in the verbiage” (2012, p. 80). A further consequence of this
‘fuzziness’ is that meaning construed by emoticons cannot be described by more delicately
differentiated features of semiotic systems. Whereas in SFL theory linguistic resources
realising ATTITUDE can be subdivided into AFFECT, APPRECIATION, and JUDGEMENT, [Author
B] found that viewed in isolation, emoticons can only construe generalised ATTITUDE.
Moreover, with regards to distinguishing how emoticons construe solidarity and attitude,
A social semiotic perspective on emoji 7
[Author B] notes that “it is not possible to neatly compartmentalize emotion and solidarity.
Emotion is involved in the way solidarity is expressed since alignment with others is generally
a positive emotional experience” (2012, p. 68).
Also approaching emoji semiosis from a social semiotic perspective, Parkwell’s (2019)
analysis elaborated a framework for describing how meaning is made across modes. Parkwell
drew on social semiotic work on multimodality (Kress & van Leeuwen, 2001; O'Halloran,
2004), and on [Author C]’s analysis of the discourse of social media, in particular their
metafunctional analysis of hashtags. Parkwell’s analysis explored how a single emoji (‘toilet’
!()*+) can construe meaning in each of the metafunctions, concluding that “emoji are a highly
contextual, flexible modality, likely to continue to shift and morph with the changing needs
and contexts of social media users” (Parkwell, 2019, p. 9).
3. Systemic Functional Linguistics (SFL)
3.1. A Social Semiotic Approach to Exploring Emoji
As attested by the work of [Author B] and Parkwell (2019), a social semiotic approach
to exploring emoji offers a robust analytical framework for describing intermodal semiosis.
This framework is underpinned by the theoretical architecture of SFL as elaborated by Halliday
(1978) and colleagues. A key theoretical principle of SFL relevant to this study is the
complementarity of paradigmatic and syntagmatic relations as perspectives on semiosis.
Language users make meaning by instantiating choices from systems of language, and these
choices interact as a text unfolds. Paradigmatic relations describe the relationship between all
the potential options available to a language user at a particular point in a text, or as Halliday
and Matthiessen (2014, p. 22) indicate, “what could go instead of what”. Syntagmatic relations,
on the other hand, are concerned with “what goes together with what” (Halliday & Matthiessen,
2014, p. 22), offering insight into the patterns and regularities that occur as language unfolds.
A social semiotic perspective on emoji 8
In this research into the semiotic affordances of emoji, paradigmatic relations provide a view
on how emoji are organised into systems of options, while syntagmatic relations account for
their contribution to a text’s meaning as it unfolds as discourse.
Another key dimension of SFL useful for understanding emoji is the concept of
semiotic metafunctions, whereby semiotic resources simultaneously enact an ideational
function of construing experience, an interpersonal function of enacting relationships and a
textual function of organising discourse into coherent text. The choices in meaning most
relevant to this study are at the level of discourse semantics, concerning the ideational system,
IDEATION, the interpersonal systems, APPRAISAL and INVOLVEMENT, and the textual system,
PERIODICITY. These systems are explained in more detail in the method section where they form
the basis of the coding rubric applied to annotate the dataset.
3.2. Intermodal semiosis
Within the social semiotic tradition, multimodal discourse analysis (MDA) has
emerged as a robust framework for analysing how non-linguistic modes realise meaning, both
independently and in conjunction with language (Kress & van Leeuwen, 2001). MDA is
underpinned by the SFL theoretical architecture described above. Accordingly, multimodal
discourse analysts have elaborated systems that describe how paradigmatic and syntagmatic
choices unfold in modalities other than language, and how resources in these modes realise
meaning across metafunctions. A further, crucial region of enquiry of MDA is how meaning
enacted across modalities interacts, given that many genres employ multiple modalities within
a single text. This approach is informed by what Lemke terms the ‘combinatorial principle’ for
interpreting intermodal semiosis, which “shows us how we can mean more, mean new kinds
of meanings never before meant and not otherwise mean-able, when this process occurs both
within and across different semiotic modalities” (1998, p. 92). Lemke’s approach to describing
A social semiotic perspective on emoji 9
intermodality thus proposes that the most fruitful analysis of intermodal texts accounts for both
meaning made within modes and across them. The term ‘intermodality’ is favoured over
‘multimodality’ here as it foregrounds the relation among modes as the site of enquiry, rather
than the description of their semiosis in parallel (see Bateman, 2014). Reflecting on work
exploring the interaction among linguistic, paralinguistic and pictorial resources in pedagogic
contexts, Lemke notes that despite the analytical utility of separating these “into different
‘channels,’”, this approach “neglects the inherent unity of communicative meaning-making
which makes the co-ordination among channels not only possible, but normal” (1998, p. 94).
Accordingly, while the method adopted here begins by analysing linguistic and emoji semiosis
monomodally, description of meanings construed across texts reflects a unitary interpretation.
In this regard, the MDA approach parallels the context-sensitive approaches described in the
previous section, such as the rhetorical structure theory-informed approach employed by Ge
and Herring (2018) and the grapholinguistic approach taken by Dürscheid and Meletis (2019).
As the combinatorial principle has been applied to analyses of intermodality in other
contexts and genres, a number of more specific intermodal relationships have been observed.
Those most relevant to this study are intermodal coupling and minimum mapping. A social
semiotic multimodal discourse analysis approach is also relevant to the polemic regarding
whether emoji can be considered a stand-alone language (cf. Danesi, 2016). This approach
sidesteps the question by considering emoji a semiotic resource (like images, sounds, etc.) with
particular semiotic affordances. As such, while we refer to SFL’s ‘supramodal’ grammar to
describe these affordances, we recognise that in some regions of meaningmaking emoji afford
fewer and thus less delicate options for making meaning than language, but in others they afford
more.
3.2.1. Intermodal coupling
A social semiotic perspective on emoji 10
Intermodal coupling provides the core unit of analysis for describing intermodal
semiosis. An intermodal coupling occurs when resources across communicative modes
interact, yielding a new semiotic unit. This definition follows the approach outlined by Painter,
Martin, and Unsworth (2013). While Painter et al.’s work was limited to interactions within
metafunctions (constrained by the principle of convergence/divergence), they acknowledge
that intermodal couplings can also occur across metafunctions. Work on how resources interact
across metafunctions (but within a single modality) has been especially productive in
describing the interaction of attitudinal and ideational resources (e.g.: Knight, 2013; Martin,
[Author A]; [Author D]). These resources frequently interact to construe language users’
evaluations about ideational targets; as Martin states, “feelings are always about something –
they are always interpersonal attitudes to ideational experience” (2004, p. 337). For example,
in the linguistic component of text #77 in the dataset for this study,
Best day of my life so far ❤
the positive ATTITUDE (APPRECIATION) Best is targeting the ideational ENTITY day of my life so
far. Consequently, we would describe the resulting meaning as an evaluative coupling, notated
(following Martin et al., 2013) as [day of my life so far / +ve APPRECIATION] (‘+ve’, ‘-ve’ are
used as shorthand for ‘positive’, ‘negative’; ‘APPRECIATION’ is the subcategory of ATTITUDE
describing resources for construing aesthetic evaluations). Accounting for the role of emoji
resources in this text introduces an additional layer of complexity, as the positive ATTITUDE
realized by the emoji ❤ interacts with the positive APPRECIATION of Best, thus also
participating in the evaluative coupling. A principled description of this interaction requires us
to account for how meaning is distributed across these modes, as well as how their individual
meanings combine within the coupling. In response, we incorporate the principle of minimum
mapping into our analytical framework.
A social semiotic perspective on emoji 11
3.2.2. Minimum mapping
Complementing Painter et al.’s work, Zhao (2011) proposes the minimum mapping
hypothesis as part of a more principled framework for the analysis of meaning across modes.
This hypothesis states that “if verbiage and image can co-construe one aspect of a social action,
e.g. process (what is going on), participant (the participants that engage in the process), etc.,
they form a verbiage-image coupling” (Zhao, 2011, p. 171). Thus, while individual resources
might potentially construe generalized or multiplied meanings, when they co-occur in a text
their meaning potential is constrained to the region of meaning shared among them. See, for
instance, text #44:
Incheon Airport ✈
Here, the ✈ emoji in isolation does not necessarily denote the ‘airport’ ENTITY (or rather, it is
a less salient option of its meaning potential); however, when combined with the lexical item
‘airport’ as a coupling, the meaning across these modes is specified as construing that ENTITY
(cf. Dainas & Herring’s, 2017 ‘mention’ function of graphicons).
4. Dataset and Method
The dataset considered in this study is a corpus of 1000 digital messages (SMS
messages and social media posts) collected in 2019 and containing at least one emoji, collected
from 50 undergraduate media students at an Australian University, with each student
contributing 20 messages. Each of these students was interviewed for one hour about the emoji
they used in these messages. Due to the scope of the study and the necessity for close textual
analysis of both the emoji and the co-text across multiple dimensions of meaning, a random
selection of 200 texts was analysed. Of this selection, approximately 40 (see Appendix) are
discussed in this study. The decision of which to include was guided by the methodological
principle of theoretical sampling as proposed by Glaser & Strauss (2006), whereby an “analyst
A social semiotic perspective on emoji 12
jointly collects, codes, and analyzes his data and decides what data to collect next and where
to find them, in order to develop his theory as it emerges” (p.45). Accordingly, the examples
presented in this paper were selected for their value in illustrating particular theoretical
phenomena noted by the analysts.
The method for analysing the data was qualitative, with individual messages manually
annotated according to a coding rubric. This rubric was designed to code for the kinds of
meanings that emoji can make intramodally (independent of language) and intermodally via
various kinds of relations with the linguistic component of the message. Categories of meaning
were drawn from Martin (1992) and Martin and White’s (2005) discourse semantics systems
of IDEATION (construing experience), APPRAISAL (expressing emotion and evaluation),
INVOLVEMENT (enacting solidarity) and PERIODICITY (organising text). Linguistic realisations
of features within these systems are shown in Table 1. Relationships between individual
features were annotated as couplings, drawing on the relational principles of intermodal
semiosis explained earlier in this paper. Accordingly, four kinds of intermodal couplings were
identified: convergent IDEATION, convergent ATTITUDE, convergent GRADUATION and
intermodal evaluative couplings. By coding the meaning realised by emoji first in isolation and
then in conjunction with linguistic co-text, this method allowed the researchers to identify
instances where the interaction between modalities impacted emoji meaning. In turn, these
instances form the basis of the results and discussion.
System
Subsystem
Description
Example
IDEATION
ENTITY
Things, activities and semiotic
entities
coffee (#10 from dataset),
helping (#147),
publication (#52).
OCCURRENCE
Activities or processes
involving one or more entity
they’ve taken it off Netflix
(#64), I had salad (#63)
STATE
States or changes affecting
one or more entities
Hyundai exec for Sale
(#31).
APPRAISAL
ATTITUDE
Emotion or evaluation
AFFECT (emotion): happy
(#183), sad (#7);
A social semiotic perspective on emoji 13
JUDGEMENT (social
esteem/sanction):
wholesome (#181), freaks
(#200)
GRADUATION
Up or downscales attitudinal
resources
cutest (#90).
INVOLVEMENT
Non-gradable resources for
construing solidarity among
interactants, including
resources that mark
interactants as part of a shared
community, such as
swearing/taboo lexis, slang,
and naming
twinning (#45) (slang
term for resembling
another person’s
appearance)
PERIODICITY
Organisation of text in terms of
information structure
sometimes you gotta let
your skin breathe a little
(#200) (temporal
circumstance in Theme
position)
Table 1: Coding Rubric for discourse semantic systems for annotation of layers in WebAnno.
The annotation software, WebAnno, was employed to systematically code the data.
WebAnno is a Java-based, web-hosted annotation program that accommodates Unicode
characters such as emoji and has a flexible, customizable interface for designing coding rubrics.
Data was first converted to .csv file format and uploaded into a new WebAnno project,
formatted for one entry per line. The software enabled two kinds of annotation to occur:
annotation of features in discrete layers (coding meaning for isolated semiotic resources, such
as a single emoji or lexical item) and annotation of types of relations between features. The
discourse semantic systems shown in Table 1 formed the annotation ‘Layers’ within WebAnno,
forming ‘tags’ with ‘tagsets’.
An example of how these layer features and couplings are shown when coded in the
WebAnno interface is found in Figure 1 – an excerpt of the coding for text #13. We can see
that each individual feature and coupling is labelled in the interface and, whilst it cannot be
shown in a static image such as this one, each label is highlighted in bold font when the mouse
pointer is held over it (like ‘cumulative’ in this image). The program abbreviates the features
A social semiotic perspective on emoji 14
in the visualisation: Att = Attitude; Ide = Ideation; Pos = Position (relating to the location in
the text where an emoji occurs); Pol = Polarity (specifying whether instances of ATTITUDE are
positive or negative). Coding of emoji resources for these features was conducted by
interpreting emoji intramodally (within a single mode) and then intermodally in batches of 40-
50 texts. After each batch the researchers would confer to discuss any issues in the coding
rubric or points of interest. In light of revisions to the coding rubric or how it should be applied
to particular cases, coding was repeated/updated where necessary.
Figure 1: example of WebAnno coding
Considering the importance of the context of communication for interpreting the
meaning of emoji, the authors of texts were interviewed and asked about the tenor of the text
(the relationship between the author and recipient or audience), the field of the text (the topic
or domain), and the mode (the platform via which the text was produced and disseminated).
Annotation results were compared to these descriptions. The degree of convergence
(agreement) between the annotation and the authors’ descriptions were coded as either ‘fully
agree’, ‘partially agree’ or ‘disagree’. The annotation fully agreed with the author descriptions
in 81% of cases, and if including partial agreement, accorded with 93% of instances.
5. Results and Discussion
Given the goal of mapping emoji semiosis according to SFL theory, a primary
outcome of the analysis was the identification of typical emoji realisations of semiotic choices
in SFL discourse semantic systems. Once this had been concluded, these realisations informed
A social semiotic perspective on emoji 15
analysis of more complex instances of intermodal semiosis. Accordingly, the results of this
study can be divided between the elaboration of a social semiotic analytical framework for
analysing emoji semiosis and the description of particular regions of intermodal meaning
realised by the interaction between emoji and language.
5.1. Typical emoji realisation of discourse semantic features
The first relevant result of the analysis is the description of typical meanings realised
by emoji. Identification of typical emoji realisations of discourse semantic features was guided
by an understanding of how intermodal semiosis acts to foreground particular regions of emoji
meaning potential. We are aware that capturing the dynamic, context-dependent semiosis of
emoji in a static representation such as a table inevitably simplifies and omits aspects of
individual emojis’ meaning potential; however given the repeated use of some emoji to
construe particular semiotic features we feel it is justified to note these typical realisations.
These realisations are summarised below and form the basis for the descriptions of semiosis
realised through the interaction between emoji and language discussed in the following
subsections.
ATTITUDE is the discourse semantic system within the interpersonal metafunction
concerned with describing resources for construing evaluative meaning. Typical emoji
realisations of ATTITUDE include iconic representations of facial expressions, emblematic
gestures and ideational entities/occurrences/states with attitudinal connotations (Table 2).
Layer
Description
Examples
Positive Polarity
Negative Polarity
ATTITUDE
facial expressions
!,-. !/0123 ☺ 🥰 !45678 !9:;<=
🥳 !>?@ABC !DEFGHIJ !KLMN !OPQR
!STUVWXY !Z[\]^ !_`abc !defghi !jklm
!nop
!qrst
!uvwx
!yz{|
!}~•€
!•‚ƒ„… !†‡ˆ‰ !Š‹Œ•Ž• !•‘’“”• !–—˜™
!š›œ• !žŸ ¡ !¢£¤¥¦§¨© !ª«¬ !-®¯°
!±²³´
emblematic gestures
!µ¶·¸
!"#$
!%&'
!¹º»¼
!½¾¿À
!ÁÂÃ
!ÄÅÆÇÈ
! ÉÊËÌÍÎÏ
♂
! ÉÐÑÒÓÔÕÖ×
♂
A social semiotic perspective on emoji 16
ideational
ENTITIES/OCCURENCES/STATES
!ØÙÚ !ÛÜÝ !Þßà 🖤 !áâã
!äåæ !çèé !êëì !Þßà !íîïð
!ñòó !ôõö÷ !øùúû !üýþÿ !!"#$%&'()*+
!,-./01234 !5678 !9:; !<= !>?@ABC
!DEFGHI
!JKL
!MNOPQ
Table 2: ATTITUDE realised by emoji.
GRADUATION is the discourse semantic system for upscaling or downscaling ATTITUDE.
In the data analysed here, emoji appear to only realise upscaling of ATTITUDE. This is achieved
either through repetition of emoji construing ATTITUDE, through emoji realising stylised icons
of intensified facial expressions (as compared to a baseline realisation such as the Smiling Face
!,-.), or through iconic realisation of ideational entities that connote paralinguistic features that
realise upscaled emphasis (Table 3). An example of the latter option is found in text #40, where
the raised volume in spoken language connoted by the ‘loudspeaker’ emoji !RSTUVW serves to
upscale the written language in the text.
Layer
Description
Examples
GRADUATION:
Upscale
repetition
! ! !ŠŠŠ‹‹‹ŒŒŒ•••ŽŽŽ•••
intensification of facial
expression
‘grinning face with big eyes’
!Z[\]^
and
‘weary face’
!•‘’“”•
.
iconic representation of
paralinguistic emphasis
!RSTUVW
5 PLACES LEFT!
!RSTUVW
There are
only FIVE tickets remaining (#40)
Table 3: GRADUATION realised by emoji.
INVOLVEMENT is the interpersonal system for expressing solidarity by employing
semiotic resources to encode meaning that can only be interpreted by members of a particular
community. In the context of emoji semiosis, these include emoji that are used as emblems
within particular communities and emoji that connote taboo meaning which is acceptable when
used among solidary interactants (Table 4).
Layer
Description
Examples
INVOLVEMENT
community
specific
emblems
‘rainbow’
!XYZ[\]^
encoding solidarity among members of
the LGBTQ+ community in text #12:
Love is love
!XYZ[\]^
Proud of love.
taboo semiosis
‘Pile of Poo’ emoji encoding positive ATTITUDE
among interactants in text #17:
A social semiotic perspective on emoji 17
Happy birthday to my fave twins!!!!
! !DDEEFFGGHHII
#thosewhogetfattogetherstaytogether
Table 4: INVOLVEMENT realised by emoji.
IDEATION is the discourse semantic system describing resources for construing
experience. Emoji typically encode ideational meaning through iconic representations of
physical entities, processes and qualities (Table 5). While emoji were observed to realise
entities and occurrences even independently of language, all instances where emoji realise state
figures occurred as intermodal couplings.
Layer
Description
Examples
ENTITY
OCCURRENCE
IDEATION
Iconic representation of entities
and processes
‘desert island’
!_`abcdef (#11)
‘person running’
! Éghijkl
♀
(#40)
Table 5: IDEATION realised by emoji.
The PERIODICITY system describes choices in how information is ordered within a text.
As emoji are less formally bound by syntagmatic relations to surrounding co-text than linguistic
resources, the coding rubric for PERIODICITY choices has been simplified to three features
corresponding to emoji location within a text: INITIATING (occurring before linguistic
resources), INTEGRATED (occurring after some and before other linguistic resources) or
CUMULATIVE (occurring after linguistic resources) (Table 6).
Layer
Description
Examples
INITIAL
INTEGRATED
CUMULATIVE
PERIODICITY
Where the
emoji is
located in the
text.
!5678 We are finalists
in the category of
Fitness Service in
2019 Melbourne
North Local
Business Awards!
(#89)
chillssssss ! !ÄÄÅÅÆÆÇÇÈÈ
@shawnmendes
(#73)
do we really
have to do this
!uvwx (#22)
Table 6: PERIODICITY realised by emoji
A social semiotic perspective on emoji 18
5.2. Annotation of relations between features as couplings
The second relevant result of the analysis concerns the categorisation of coupling
relations found in the data. Couplings among semiotic resources can occur either within or
across metafunctions and modalities, thus coupling options can be represented as a matrix
organised around two variables: intramodal vs intermodal and intrametafunctional vs
intermetafunctional, as shown in Table 7 referring to text #90:
the cutest baby ! ! !ŠÞm‹nŒo•Ž•ßàpqrsÉ
Intrametafunctional
Intermetafunctional
Intramodal
Coupling of interpersonal
metafunction resources for
ATTITUDE (cutest) and
GRADUATION (cutest) within the
linguistic mode of text #90.
Coupling of interpersonal metafunction
resources for ATTITUDE (cutest) and
IDEATIONAL (baby) within the linguistic
mode of text #90.
Intermodal
Coupling of interpersonal
metafunction resources for
ATTITUDE in the linguistic mode
(cutest) and in emoji (!Þßà) in text
#90.
Coupling of interpersonal metafunction
resource of ATTITUDE in the linguistic
mode (cutest) with the ideational
metafunction resource of IDEATION in
the emoji mode (‘Cat Face’
!mnopqrs
)
Table 7: Applying a semiotic couplings matrix to text #90
Describing intermodal semiosis within a metafunction is substantially different to
describing intermodal semiosis across metafunctions, thus we have divided intermodal
couplings into two kinds: convergent intermodal couplings and evaluative intermodal
couplings. In turn, convergent intermodal couplings can be further specified by metafunction
or system, for instance IDEATIONAL convergent intermodal coupling or ATTITUDINAL
convergent intermodal coupling (illustrated in Table 8Error! Reference source not found.).
Feature
Subtype
Realisation Examples
Description
Convergent
IDEATION
ENTITY
Leighton Beach 🏖 (#43)
Leighton Beach converges
with ‘beach’ 🏖
OCCURRENCE
Do you want to
!9:;
today?
(#188)
‘flexing biceps’
!9:;
converges with the mental
process clause do you
want to to construe the
occurrence ‘lift weights’
STATE
new nose piercing !üýþÿ
‘sparkles’ !üýþÿ converges
A social semiotic perspective on emoji 19
(#192)
with nose piercing to
construe the state
‘sparkling nose piercing’
Convergent
ATTITUDE
AFFECT
its sad bc this is the standard
for literally 90% of the
media/pr/marketing
internships i see online !qrst”
(#7)
sad converges with
‘persevering face’ !qrst
APPRECIATION
The good stuff !%&' ((#56)
Interaction between good
and ‘Ok hand’
!%&'
:
JUDGEMENT
Went to harry styles concert
with 0 expectations but
came back in awe???? What
a guy !defghi (#72)
what a guy converges with
‘Star-Struck’ !defghi
Convergent
GRADUATION
GRADUATION
Youre such a little
! ! !ttuvvwwxxyz{ (#9)
such converges with
repetition of ‘mouse’ !tvwx
Evaluative
coupling
Is anyone selling Taylor
Swift tickets? !š›œ•. (#176)
Is anyone selling Taylor
Swift tickets? realises an
ideational OCCURRENCE,
while !š›œ• realises negative
ATTITUDE targeting this
occurrence.
Table 8: Discourse semantic features realised by intermodal couplings
However, it should be noted here that in instances where emoji and language converge
to construe ATTITUDE, that ATTITUDE will always have an IDEATIONAL target, thus an evaluative
coupling will also be construed. This is the case in text #56:
The good stuff !%&'
Here, emoji and language converge to construe positive APPRECIATION for the
IDEATIONAL target stuff, thus forming the evaluative coupling [stuff / +ve APPRECIATION]. As
posited by the minimum mapping hypothesis, the intermodal relationship exists between the
ATTITUDINAL meaning construed by each mode as well as between ATTITUDINAL and
IDEATIONAL meaning divided by mode, thus this text contains both a convergent attitudinal
coupling (good + !%&'| and an evaluative coupling (stuff + !%&'|.
5.3. Applying principles of minimum mapping to the dataset
A social semiotic perspective on emoji 20
The third relevant result of the analysis concerns the application of the minimum
mapping hypothesis to the interpretation of meaning made by the interaction between emoji
and language. Application of the principle of minimum mapping introduced in Section 3.2.2 to
emoji-language semiosis revealed that the semiotic relations between interacting resources are
temporary. Thus individual resources could act to specify differing aspects of their meaning
potential depending on what they were coupled with, the coupling’s co-text, and the text’s
context. For example, the emoji !%&' was used in texts #56 and #88:
#56: The good stuff !%&'
#88: Take 3 for the sea !%&' (more is welcome too :) ;) ).
In text #56 the emoji’s interaction with the verbiage good foregrounds its ATTITUDINAL
meaning (as the emoji iconically represents the hand gesture emblematic for ‘OK’), with the
resulting intermodal coupling construing positive APPRECIATION. Contrastingly in text #88
interaction with the verbiage three foregrounds the ideational aspect of the emoji realising the
ENTITY ‘three’ (as the emoji iconically represents a hand holding up three fingers). In this
dataset no instances were found of individual resources being repeated within a text with
divergent meanings foregrounded, most likely due to the short length of texts. However, the
underlying principle of temporary interaction allows for such divergence, especially in longer
texts.
Minimum mapping was useful in guiding analysis of interaction across modes within a
single metafunction, but it did not help us describe intermodal couplings where each mode is
realising meaning in a different metafunction. Take, for example, text #176:
Is anyone selling Taylor Swift tickets? !š›œ•.
In this text, IDEATIONAL meaning is realised by the linguistic component of the text, while
ATTITUDINAL resources are realised by the emoji component. We can interpret the emoji as
construing generalised negative ATTITUDE, and the field of the IDEATIONAL meaning is
A social semiotic perspective on emoji 21
established as ‘obtaining tickets to a Taylor Swift concert’. But with no interacting resources
within the interpersonal metafunction to minimally map onto, the ATTITUDE of the intermodal
evaluative coupling remains unspecified (cf. [Author B]’s description of ‘fuzzy’ meaning).
Plausible interpretations of the emoji include negative AFFECT, reflecting the author’s concern
about finding a ticket, or negative APPRECIATION for the absence of tickets. Consequently, in
cases such as this we refrain from specifying more delicate ATTITUDE features and notate the
coupling as [obtaining tickets to a Taylor Swift concert / -ve ATTITUDE].
5.4. Taxonomic relations within convergent IDEATION ENTITY couplings
The fourth relevant result of the analysis concerns the distribution of meaning across
modes in intermodal, convergent ideational couplings. A particular region of intermodal
semiosis that was observed in the data is that convergent ideational meaning realising ENTITIES
across modes often follows a pattern of hypo/hypernymy, whereby meaning construed by one
mode will be a superordinate or subordinate to meaning construed by the other. For example,
text #55:
Thoroughly disappointed with the lack of milk !}~•€•‚
Here the verbiage milk realises one primary ENTITY – ‘milk’ (other potential interpretations are
possible, as [etc.] in Figure 2 allows for). Conversely, the ‘baby bottle’ entity iconically
realised by the !}~•€•‚ emoji can be divided into the entities that constitute the graphicon: ‘bottle’,
‘milk’, ‘teat’, and so forth. Applying the minimum mapping hypothesis, we propose that the
intermodal meaning jointly construed by the emoji and language is the ‘milk’ entity construed
by both modes, while other entities potentially construed by only one mode are not realised.
The intermodal semiosis of this text is summarised in Figure 2, with linguistic semiotic
resources highlighted in red, emoji in blue, and intermodal semiosis in green. In turn, the
intermodally construed entity ‘milk’ is nested within a linguistically realised STATE, ‘lack of
A social semiotic perspective on emoji 22
milk’.
Figure 2: intermodal ideational semiosis in text #55
A more complex example of shared IDEATIONAL meaning mapped across modes can be
seen in text #198:
If you want a break from studying and pet some animals !_`abc there is a petting zoo at
the Physics Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!! It’s
entirely free and there are trained handlers with lots of animals !!!ƒ„…†‡ˆ‰Š‹Œ•Ž••‘’“
Here we can see how the linguistically construed IDEATIONAL ENTITIES some animals, petting
zoo, trained handlers and lots of animals can be organised into a taxonomy that intersects with
the meaning of the ‘cow’ !ƒ†‡Š‹ŒŽ, ‘rabbit’ !„ˆ•• and ‘goat’ !…‰•‘’“ emojis, as shown in Figure 3.
Collectively, language and emoji realise a taxonomic category coded as ‘petting zoo animals’.
This relationship echoes Zhao’s (2011) description of ‘metonomysing’ relations in language-
image texts.
A social semiotic perspective on emoji 23
Figure 3: intermodal ideational semiosis in text #198
5.5. Intermodal specification
A fifth noteworthy pattern observed in the data is that intermodal interaction can serve
to specify particular dimensions of the meaning potential of each mode. Consider, for example,
text #169:
Smaller than expected but cosy none the less (: Saw ! ! !”•…–—˜™š›‰œ•žŸ•‘’“+ ! ! ¡¢£¤¥¦§¨©. I reckon it
would be a whole 'nother scene at night. Wouldn't mind coming back in spring
sometime as well with all the flowers in bloom~
Here we can see how Saw ! ! !”•…–—˜™š›‰œ•žŸ•‘’“ construes an intermodal OCCURRENCE which can be
rendered linguistically as ‘I saw pigs, chickens and goats’. However, following from the
syntactical patterning of the preceding co-text, rather than construing its typical, monomodal
meaning of ‘knife and fork’ or ‘cutlery’, the ! ¢¤¥ emoji construes the process ‘eating’ as part of
the OCCURRENCE which can be rendered as ‘ate rabbit’: ! ! ¡¢£¤¥¦§¨©. This interpretation also sustains
continuity in the IDEATIONAL field of the text, with farm animals remaining the targets of the
author’s activities (as opposed to incongruously suggesting that the author saw pigs, chickens,
A social semiotic perspective on emoji 24
goats, cutlery and rabbits). From a minimum mapping perspective, the verbal group dimension
of the ! ¢¤¥ emoji has been foregrounded because it intersects with the meaning realised by the
co-text, while the ENTITY dimension is backgrounded.
Another instance of linguistic co-text shifting the meaning of emoji from an
IDEATIONAL ENTITY to an OCCURRENCE occurs in text #67:
I can see how "interested" everyone is ! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯
Here the quotation marks around ‘interested’ suggest this word could be interpreted
ironically/sarcastically, implying that ‘everyone’ is in fact not interested. This opens a potential
region of meaning with which the !ª«¬-®¯ emoji which follows can be cross-referenced, inviting a
reader to consider what relationship might exist between the STATE ‘everyone is (not)
interested’ and the iconically realised ENTITY ‘mobile phone’. Accordingly, a plausible
interpretation of the ! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯ emoji sequence is that it construes an OCCURRENCE rendered
as ‘looking at mobile phones’ which in many contexts is associated with lack of interest. In this
case, it is noteworthy that the interaction among modes functions to specify meaning bi-
directionally, as both the quoted verbiage, “interested” and the !ª«¬-®¯ emoji are examples of
semiotic resources that are to some degree indeterminate and context dependent. By applying
a minimum mapping approach to interpreting intermodal semiosis, we can see how the
dimensions of individual semiotic resources that are shared across modes are foregrounded,
while those that are not shared are backgrounded. The intermodal semiosis occurring in this
text is summarised in Figure 4.
A social semiotic perspective on emoji 25
Figure 4: intermodal ideational semiosis in text #67
5.6. Proximal attitudinal prosody
A final observed pattern in attitudinal meaning mapped across modes is that ATTITUDE
realised by emoji typically ‘washes’ over adjacent co-text both forwards and backwards. As
such we have termed this proximal attitudinal prosody. As Halliday (1979) and Martin (1995,
1996) have described, different kinds of meaning unfold according to different structural
configurations. Halliday describes interpersonal meaning as “strung throughout the clause as a
continuous motif or colouring” whose “effect is cumulative” (1979, p. 67). For example, in text
#27 we can see how the positive ATTITUDE construed by the words Merry, safe, and happy
accumulates and intensifies over the course of the text:
A belated Merry Christmas and a safe and happy holidays to y'all !°±²³´µ¶!üýþÿ
It is thus unsurprising to see ATTITUDE construed by emoji contributing to prosodic meaning,
for instance in text #52:
Extremely chuffed that my first publication, an article for Discourse & Society, is now
online - a nice wrap on the first fortnight of my PhD !Z[\]^
Here the general positive ATTITUDE construed by ‘Grinning Face with Big Eyes’ !Z[\]^ builds on
the positive ATTITUDE realised by chuffed and nice, combining to realise convergent ATTITUDE
couplings with these lexical items of AFFECT and APPRECIATION, respectively.
A social semiotic perspective on emoji 26
A more unexpected observation in the interaction between intermodal ATTITUDE
resources is that while the prosodic unfolding of attitudinal meaning in the linguistic mode is
largely unidirectional, attitudinal emoji appear to demonstrate a freer relation to their co-text.
An example of this can be found in text #99:
Me & My worst frenemy. ❤
In this text, the first ATTITUDE resource is the lexical item worst, which construes upscaled
negative APPRECIATION. This is followed by frenemy – a portmanteau of the lexical items
‘friend’ and ‘enemy’ – which denotes both positive and negative AFFECT. Looking at only the
linguistic resources, the attitudinal prosody of this text would appear to be either negative or
ambivalent. However, interaction with the positive ATTITUDE construed by the ‘Red Heart’ ❤
emoji foregrounds the positive ATTITUDE of the verbiage while backgrounding the negative,
resulting in an overall ATTITUDE construed in the text being positive. This suggests that the
ATTITUDE construed by the emoji is ‘washing’ backwards over the preceding co-text in a kind
of reverse prosody. This aspect of convergent attitudinal couplings parallels the ‘tone
modification’ function observed in pragmatics-informed descriptions of emoji semiosis such
as Herring & Dainas (2017).
While the relation of emoji to attitudinal prosody does appear to be freer than that of
linguistic resources, attitudinal emoji can nonetheless be governed by syntagmatic relations
within a text. For instance, in text #111 the ‘Crying Face’ !·¸¹º» and ‘Red Heart’ ❤ emoji only
interact with the clauses that precede them, acting as culminations for the linguistic ATTITUDE
construed therein:
Gonna miss this school !·¸¹º» Love every one of you boys, thanks for the memories ❤
This suggests attitudinal meaning realised by emoji behaves somewhat differently from
linguistic ATTITUDE, both in how it interacts with proximal attitudinal resources and how it can
serve to conclude stretches of prosody. This observation parallels work both within social
A social semiotic perspective on emoji 27
semiotics (such as Knox, 2009) and in other theoretical traditions that notes how emoji and
emoticons can function like clause-final punctuation (cf. Na’aman et al., 2017; Sampietro,
2016; Dresner & Herring, 2010; Provine et al., 2007).
6. Conclusion
This paper has presented a social semiotic perspective on emoji semiosis. Informed by
the theoretical architecture of SFL and social semiotic work on MDA, we have proposed an
analytical framework that describes interactions in meaning realised by emoji and linguistic
resources. This framework was applied to the data to map typical emoji realisations of choices
from discourse semantic systems across metafunctions, thus revealing patterns in how meaning
is construed across modalities. It was observed that intermodal convergent IDEATIONAL
meaning can be described as a shared IDEATIONAL taxonomy wherein meaning realised in one
modality is sub- or superordinate to the other, thus construing an intermodal field of discourse.
A second noteworthy interaction between emoji and language is that a minimum mapping
approach to intermodality reveals that the region of meaning potential that overlaps across
modes corresponds with the region of meaning realised by each mode. This supports the
productivity of an MDA-informed approach to interpreting emoji semiosis and provides a
valuable guiding principle for emoji-language intermodality. A third pattern observed in the
interaction between emoji and language is that the ATTITUDE realised by emoji behaves
somewhat differently from linguistically construed ATTITUDE insofar as it appears to interact
prosodically with proximal linguistic resources occurring both before and after in the co-text.
These results constitute a modest advancement towards mapping the relations
governing how emoji and language jointly construe meaning. They take us beyond the correct
but vague observations regarding the fuzziness and context-dependency of emoji prevalent in
the area of study to date, and offer the potential for a more principled study of these relations
A social semiotic perspective on emoji 28
in future. It might be especially fruitful to combine the social semiotic approach of this study
with quantitative approaches in Artificial Intelligence (cf. Miller et al., 2016) and theories of
communication (cf. Veszelszki, 2017) to explore how these relations unfold across larger
datasets.
While the theoretical framework of SFL and the MDA methodology differ from more
prevalent pragmatics-informed explorations of emoji semiosis, a number of points of
intersection between the two approaches were found. Methodologically, an MDA approach to
analysing how emoji and language interact to make meaning is congruent with work informed
by similarly intermodal frameworks such as the adapted rhetorical structure theory employed
by Ge and Herring (2017) and the grapholinguistic approach of Dürscheid and Meletis (2019).
In terms of results, convergent ideational couplings appear to realise the ‘mention’ function
described by Herring and Dainas (2017), while the dynamics of proximal attitudinal prosody,
where attitude realised by emoji ‘washes’ across co-occurring language, largely corresponds
to Danesi’s (2016) ‘adding tone’ function. These intersections both validate the findings of this
work and suggest the two traditions might be fruitfully cross-referenced in future research.
A salient limitation of this study is its limited dataset. Consequently, numerous semiotic
functions observed by other researchers working with more varied data were not observed in
our data. For instance, it would be valuable to apply the analytical framework proposed here to
texts composed solely of emoji so as to explore how emoji construe meaning in the absence of
language. In light of recent work on emoji sequences such as (Ge & Herring, 2018), this
dimension of emoji use merits closer attention. A further limitation of this study is that the data
and method employed are unable to account for the role the widely used emoji prediction
features in digital keyboards play in suggesting particular combinations of emoji and written
language. Integrating results of work in this area such as Barbieri, Ballesteros, Ronzano &
Saggion (2018) into the design of future research would expand the descriptive power of
A social semiotic perspective on emoji 29
resulting models.
7. Acknowledgements
The research presented in this study was funded by the Australian government.
i
Emoji glosses are sourced from https://emojipedia.org/ (accessed 11.11.2020)
A social semiotic perspective on emoji 30
8. References
ABERCROMBIE, D. (1968). Paralanguage. British journal of disorders of communication,
3(1), 55-59.
ALBAWARDI, A. H. (2018). Digital literacy practices of Saudi female university students.
University of Reading,
ALBERT, G. (2020). Beyond the binary: Emoji as a challenge to the image-word distinction.
In C. Thurlow, C. Dürscheid, & F. Diémoz (Eds.), Visualizing Digital Discourse (pp.
65-80): De Gruyter Mouton.
BARBIERI, F., BALLESTEROS, M., RONZANO, F., & SAGGION, H. (2018). Multimodal
emoji prediction. arXiv preprint arXiv:1803.02392.
BATEMAN, J. A. (2014). Text and image: A critical introduction to the visual/verbal divide:
Routledge.
.
CRYSTAL, D. (2001). Language and the Internet. Cambridge University Press.
DAINAS, A., & HERRING, S. C. (2021). Interpreting emoji pragmatics. In C. Xie, F. Yus, &
H. H. (Eds.), Internet pragmatics: Theory and practice. Amsterdam: John Benjamins.
DANESI, M. (2016). The semiotics of emoji: The rise of visual language in the age of the
internet: Bloomsbury Publishing.
DRESNER, E., & HERRING, S. C. (2010). Functions of the nonverbal in CMC: Emoticons
and illocutionary force. Communication theory, 20(3), 249-268.
DRESNER, E., & HERRING, S. C. (2014). Emoticons and illocutionary force. In Perspectives
on Theory of Controversies and the Ethics of Communication (pp. 81-90): Springer.
DÜRSCHEID, C. & MELETIS, D. (2019). Emojis: A Grapholinguistic Approach. In: Y.
Haralambous (ed.), Graphemics in the 21st Century. Brest: Fluxus Edition (pp. 167–
183): Fluxus Editions.
DÜRSCHEID, C., & SIEVER, C. M. (2017). Jenseits des Alphabets–Kommunikation mit
Emojis. Zeitschrift für germanistische Linguistik, 45(2), 256-285.
GAWNE, L., & MCCULLOCH, G. (2019). Emoji as digital gestures. Language@ Internet,
17(2).
GE, J., & HERRING, S. C. (2018). Communicative functions of emoji sequences on Sina
Weibo. First Monday.
GLASER, B. G., & STRAUSS, A. L. (2006). Discovery of grounded theory: Strategies for
qualitative research: Routledge.
HALLIDAY, M. A. K. (1978). Language as social semiotic: The social interpretation of
language and meaning. Edward Arnold.
HALLIDAY, M. A. K. (1979). Modes of meaning and modes of expression: types of
grammatical structure, and their determination by different semantic functions. In D. J.
ALLERTON, E. CARNEY , & D. HOLDCROFT (Eds.), Function context in linguistic
analysis: Essays offered to William Haas (Vol. 1, pp. 57-79). Cambridge University
Press.
HALLIDAY, M. A. K., & MATTHIESSEN, C. M. I. M. (2014). An introduction to functional
grammar (Vol. 4th). Edward Arnold.
HAO, J. (2015). Construing biology: An ideational perspective. University of Sydney,
HERRING, S. C., & DAINAS, A. (2017). “Nice picture comment!” Graphicons in Facebook
comment threads. Paper presented at the Proceedings of the 50th Hawaii International
Conference on System Sciences.
HERRING, S. C., & DAINAS, A. R. (2020). Gender and Age Influences on Interpretation of
Emoji Functions. ACM Transactions on Social Computing, 3(2), 1-26.
KNIGHT, N. K. (2013). Evaluating experience in funny ways: how friends bond through
A social semiotic perspective on emoji 31
conversational hum. Text & Talk, 33(4-5), 553.
KONRAD, A., HERRING, S. C., & CHOI, D. (2020). Sticker and Emoji Use in Facebook
Messenger: Implications for Graphicon Change. Journal of Computer-Mediated
Communication, 25(3), 217-235.
KNOX, J. S. (2009). Punctuating the home page: image as language in an online newspaper.
Discourse & Communication, 3(2), 145-172.
KRESS, G. R., & VAN LEEUWEN, T. (2001). Multimodal discourse: The modes and media
of contemporary communication: Edward Arnold.
LEMKE, J. L. (1998). Multiplying meaning. IN J. R. MARTIN & R. VEEL (Eds.), Reading
science: Critical and functional perspectives on discourses of science (pp. 87-113):
Psychology Press.
MILLER, H., THEBAULT-SPIEKER, J., CHANG, S., JOHNSON, I., TERVEEN, L., &
HECHT, B. (2016). “Blissfully Happy” or “Ready toFight”: Varying Interpretations
of Emoji. Paper presented at the Proceedings of the International AAAI Conference on
Web and Social Media.
MARTIN, J. R. (1992). English text: system and structure. Philadelphia: John Benjamins
MARTIN, J. R. (1995). Text and clause: Fractal resonance. Text-Interdisciplinary Journal for
the Study of Discourse, 15(1), 5-42.
MARTIN, J. R. (1996). Types of structure: deconstructing notions of constituency in clause
and text. In Computational and conversational discourse (pp. 39-66): Springer.
MARTIN, J. R. (2004). Mourning: How We Get Aligned. Discourse & Society, 15(2-3), 321-
344.
MARTIN, J. R. (2008). Tenderness: realisation and instantiation in a Botswanan town. Paper
presented at the 34th International Systemic Functional Congress, Syddansk
University, Odense.
MARTIN, J. R., & ROSE, D. (2007). Working with Discourse: Meaning Beyond the Clause (2
ed.): Bloomsbury.
MARTIN, J. R., & WHITE, P. R. R. (2005). The Language of Evaluation: Appraisal in
English. Palgrave Macmillan.
MCNEILL, D. (1992). Hand and mind: What gestures reveal about thought: University of
Chicago press.
MILTNER, K. M. (2020). "“One part politics, one part technology, one part history”: Racial
representation in the Unicode 7.0 emoji set." New Media & Society.
1461444819899623.
NA’AMAN, N., PROVENZA, H., & MONTOYA, O. (2017). Varying linguistic purposes of
emoji in (twitter) context. Paper presented at the Proceedings of ACL 2017, Student
Research Workshop.
NISHIMURA, Y. (2015). A sociolinguistic analysis of emoticon usage in Japanese blogs:
Variation by age, gender, and topic. AoIR Selected Papers of Internet Research.
O'HALLORAN, K. (2004). Multimodal discourse analysis: Systemic functional perspectives:
A&C Black.
PAINTER, C., MARTIN, J. R., & UNSWORTH, L. (2013). Reading visual narratives.
Equinox.
PARKWELL, C. (2019). Emoji as social semiotic resources for meaning-making in discourse:
Mapping the functions of the toilet emoji in Cher’s tweets about Donald Trump.
Discourse, Context & Media, 30, 100307.
PROVINE, R. R., SPENCER, R. J., & MANDELL, D. L. (2007). Emotional expression online:
Emoticons punctuate website text messages. Journal of language and social
psychology, 26(3), 299-307.
REZABEK, L., & COCHENOUR, J. (1998). Visual cues in computer-mediated
A social semiotic perspective on emoji 32
communication: Supplementing text with emoticons. Journal of Visual Literacy, 18(2),
201-215.
SAMPIETRO, A. (2016). Exploring the punctuating effect of emoji in Spanish Whatsapp
chats. Lenguas modernas(47).
SKOVHOLT, K., GRØNNING, A., & KANKAANRANTA, A. (2014). The communicative
functions of emoticons in workplace e-mails. Journal of Computer-Mediated
Communication, 19(4), 780-797.
THOMPSON, D., & FILIK, R. (2016). Sarcasm in written communication: Emoticons are
efficient markers of intention. Journal of Computer-Mediated Communication, 21(2),
105-120.
VESZELSZKI, Á. (2017). Digilect: the impact of infocommunication technology on language:
Walter de Gruyter GmbH & Co KG.
WALTHER, J. B., & D’ADDARIO, K. P. (2001). The impacts of emoticons on message
interpretation in computer-mediated communication. Social Science Computer Review,
19(3), 324-347.
ZHAO, S. (2011). Learning through multimedia interaction: the construal of primary social
science knowledge in web-based digital learning materials. (Dissertation/Thesis).
Appendix
Text
#
Verbiage
7
its sad bc this is the standard for literally 90% of the media/pr/marketing internships
i see online
!qrst
”
9
Youre such a little
! ! !ttuvvwwxxyz{
10
Coffee before anything
!45678
11
Rainbow beach in Brisbane
!_`abcdef
an expensive estate. Just opposite the Great Sandy
National Park, which house a steep trail!¼½¾¿ÀÁÂà towards a hidden aboriginal craving site.
!ÄÅÆÇ
12
Love is love
!XYZ[\]^
Proud of love.
13
Finally an excuse to post this photo I took that Abigail refuses to post herself! Happy
21st birthday @abigailmarks
! !!,"-#$%&'()*+./01234
!üýþÿ
17
Happy birthday to my fave twins!!!!
! !DDEEFFGGHHII
#thosewhogetfattogetherstaytogether
22
do we really have to do this
!uvwx
27
A belated Merry Christmas and a safe and happy holidays to y'all
!°±²³´µ¶
!üýþÿ
31
Hyundai exec for Sale 🥳🥳
35
Tragic
!ÈÉÊ
40
!RSTUVW
5 PLACES LEFT!
!RSTUVW
There are only FIVE tickets remaining for BABSOC’s
Fungal Art Workshop! Secure your place now before tickets run out ! Éghijkl♀
43
Leighton Beach
🏖
44
no worries ☺ good luck !!!
45
Twinning it with this babe
! !_ø`abcùúû
52
Extremely chuffed that my first publication, an article for Discourse & Society, is
now online - a nice wrap on the first fortnight of my PhD
!Z[\]^
55
Thoroughly disappointed with the lack of milk
!}~•€•‚
56
The good stuff
!%&'
63
So i had salad
!†‡ˆ‰
64
they’ve taken it off Netflix tho
! !ŠŠ‹‹ŒŒ••ŽŽ••
65
You are welcome
!/0123
buy my tacos next week biatch
!DEFGHIJ
A social semiotic perspective on emoji 33
67
I can see how "interested" everyone is
! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯
72
Went to harry styles concert with 0 expectations but came back in awe???? What a
guy
!defghi
73
chillssssss
! !ÄÄÅÅÆÆÇÇÈÈ
@shawnmendes
77
Best day of my life so far ❤
88
Take 3 for the sea
!%&'
(more is welcome too :) ;) )
89
!5678
We are finalists in the category of Fitness Service in 2019 Melbourne North Local
Business Awards!
90
the cutest baby
! ! !ŠÞm‹nŒo•Ž•ßàpqrs
99
Me & My worst frenemy. ❤
111
Gonna miss this school
!·¸¹º»
Love every one of you boys, thanks for the memories ❤
115
been once, theyre pretty fun
!ª«¬
147
Thanks so much for helping me out (and it was lovely to catch up last week!)
!ËÌÍÎ
169
Smaller than expected but cosy none the less (: Saw
! ! !”•…–—˜™š›‰œ•žŸ•‘’“
+
! ! ¡¢£¤¥¦§¨©
. I reckon it
would be a whole 'nother scene at night. Wouldn't mind coming back in spring
sometime as well with all the flowers in bloom~
176
Is anyone selling Taylor Swift tickets?
!š›œ•
178
Embracing my skin more than ever because damn it’s bloody important; sometimes
you gotta let your skin breathe a little !ÏÐÑÒ Also, my baby hair is getting out of control,
I can literally construct a fringe out of it
!•‘’“”•
181
aww
! !ÞÞßßàà
u r so wholesome
183
What a lovely thing to see! So happy you and kiddos are safe, it must be so difficult
for you to keep things together right now, but I just wanted to commend you on your
strength!
!Þßà
!üýþÿ
!ÁÂÃ
188
Do you want to
!9:;
today?
192
new nose piercing !üýþÿ
198
If you want a break from studying and pet some animals
!_`abc
there is a petting zoo at
the Physics Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!! It’s
entirely free and there are trained handlers with lots of animals
!ƒ†‡Š‹ŒŽ!„ˆ••!…‰•‘’“
200
so many freaks on here will literally jump at peoples throats to defend objectively
shitty people just bc they make mediocre samefaced art thats vaguely popular in
obscure internet circles it's SO embarrassing. couldn't be me
! !ÓÓÔÔÕÕ
List of texts cited
