ArticlePDF Available

A social semiotic perspective on emoji: How emoji and language interact to make meaning in digital messages


Abstract and Figures

This article presents a social semiotic analysis of emoji-language semiosis. Combining the theoretical architecture of Systemic Functional Linguistics and methodology of Multimodal Discourse Analysis, we propose an analytical framework that can identify how emoji make meaning both individually and in interaction with language. Using the web-based coding software WebAnno, we apply this framework to a dataset of text messages and social media posts. The results identify typical realisations of particular semiotic features by emoji as well as noteworthy dynamics in how emoji interact with language to realise meaning. We observe (1) how emoji and language jointly construing ideational meaning realise intermodal taxonomies (where hyper/hyponyms are distributed across modes) and particular fields of discourse (domains of experiential meaning), (2) how resources in one mode can serve to foreground particular regions of meaning potential in other modes, and (3) how attitudinal meaning realised by emoji appears to differ from the prosodic patterning of linguistic attitude.
Content may be subject to copyright.
A social semiotic perspective on emoji 1
A social semiotic perspective on emoji:
How emoji and language interact to make meaning in digital messages
1. Introduction
There is currently extensive interest in linguistics and communication studies in the role
that non-linguistic features such as emoji play in digitally mediated communication. Emoji are
widespread in social media discourse, involved in the expression of emotion, conveying
stances, and negotiating interpersonal alignments. This study draws on the theoretical
orientation of social semiotics to explore the kinds of meanings that can be made as emoji
interact with language (the linguistic ‘co-text’) in a social media post or digital message.
Emoji are a subcategory of what Herring and Dainas (2017) classify as ‘graphicons’ (graphical
icons) – a classification of graphical features found on social media platforms that also includes
emoticons, stickers, GIFs, images, and videos. Emoji are small images represented using code
points (a numerical code assigned to each character) in the Unicode Standard. While the first
recorded case of emoticon use occurred in 1982 on the Carnegie Mellon bulletin board, emoji
were created in Japan in 1997. Since then, emoji have been widely adopted by computer-
mediated-communication users: in 2016 almost 50% of messages sent via messaging apps
contained emoji (Emogi Research Team, 2016). As emoji have become more popular, users
have sought to expand their meaning potential and the Unicode Consortium has accepted many
proposals to expand the emoji catalogue with more varied experiences, such as representation
of a greater variety of skin tones, genders, family types, regional foods etc, although the skin
A social semiotic perspective on emoji 2
tone modifiers have been critiqued as enacting “an institutionalized form of colour-blind racism
which insists that concerns regarding racial representation and identity are irrelevant to
“neutral” technical systems and workplaces(Miltner, 2020, p.517). In 2021, the Unicode
Standard contained over 3,300 emoji. Emoji thus constitute a vital area of linguistic
investigation, both due to their increasing usage and to their evolving meaning-making
In seeking to advance our understanding of emoji, this paper elaborates and applies an
analytical framework for exploring how emoji interact with language. Applying this framework
to the dataset suggests both typical meanings realised by specific emoji and the high degree to
which emoji meaning is context-sensitive. In regard to the latter point, this study applies Zhao’s
(2011) minimum mapping hypothesis to interpret how emoji interact with language, proposing
that the meaning jointly construed is the area of shared meaning potential across
communicative modes.
The analysis presented in this paper is underpinned by the theoretical framework of
Systemic Functional Linguistics (SFL). Relevant principles of SFL will be outlined in Section
3, however a brief glossary of terms used in the paper is provided here. These include the three
metafunctions conceptualised in SFL theory: the ideational metafunction, concerned with
experiential meaning, the interpersonal function, concerned with enacting relationships, and
the textual function, describing resources for organising meaning into a coherent text (Halliday,
1978); and the systems that describe attitudinal meaning (ATTITUDE): AFFECT (emotional states)
APPRECIATION (aesthetic evaluations) and JUDGEMENT (social/moral evaluation). Lastly, ‘co-
text’ refers to semiotic resources occurring within the same text.
2. Linguistic Research on emoji
Semiotic research on emoji to date has evolved from focusing on their attitudinal
A social semiotic perspective on emoji 3
meaning to recognising their semantic, pragmatic and phatic function within texts (cf. Danesi,
2016), often drawing parallels with how gesture contributes to semiosis (cf. Gawne &
McCulloch, 2019). Outcomes of this work relevant to this study will be reviewed below. An
alternate avenue of enquiry has investigated the sociolinguistic features of emoji, exploring
how emoji use is distributed across text genres, and by age, gender, race and cultural context
(see Miltner, 2020; Albawardi, 2018; Herring & Dainas, 2020; Nishimura, 2015). This is
undoubtedly a rich and valid field of enquiry; however its results are not immediately relevant
to this study, thus it will not be reviewed in any detail.
2.1. Pragmatic approaches
A primary issue to address in reviewing literature on pragmatic approaches to emoji
semiosis is the relationship between emoji and emoticons. Emoticons and emoji are distinct
types of graphicons; however, the terms have been used interchangeably, possibly as a
reflection of a common software feature that automatically converts some emoticons into emoji
(e.g. in Microsoft Word). While the richer pictorial affordances of Unicode afford emoji a
greater meaning potential than is available to ASCII emoticons, research which discusses
emoticons is nevertheless relevant to this study. This reflects the functional, intermodal
approach taken here, which follows Albert (2020) in grouping different categories of
graphicons as semiotic resources interacting with language.
As surveyed in Dresner and Herring (2014), early work on emoticons (Rezabek &
Cochenour, 1998; Walther & D’addario, 2001) described their semiotic function as constrained
to affective meaning, a view summed up in Crystal’s description of emoticons as
“combinations of keyboard characters designed to show an emotional facial expression” (2001,
p. 36). More recent work disputes this definition, arguing that, further to attitudinal meaning,
emoji construe phatic and pragmatic functions. Primary exponents of this view are Dresner and
Herring, who point to the multiple, context-dependent functions emoticons can realise, and
A social semiotic perspective on emoji 4
challenge the assumption that emoticons can make meaning independent of co-text. In their
(2017) work, Herring and Dainas identify six main functions of graphicon use in Facebook
comments: “reaction, tone modification, riffing, action, and narrative sequence” (2017, p.
Dresner and Herring (2017) is among the numerous works that apply a pragmatic lens
to emoji analysis. Sampietro (2016, p. 109) argues that emoji are used by writers to “align with
the interlocutor, to express informality or to enhance phatic communion and expressive speech
acts, especially greetings.” Focusing on how sarcasm is realised in written communication,
Thompson and Filik (2016, p. 116) observe that “tongue face, wink face, and ellipsis all
occurred significantly more frequently with sarcastic than literal comments”. Similarly,
Skovholt, Grønning, and Kankaanranta (2014) observe that emoticons are used as markers of
humorous intent and irony, and that they can be used to hedge accompanying speech acts, such
as face-threatening and directive acts. Another avenue of pragmatic enquiry has investigated
how emoji and emoticons can be used as discourse markers akin to punctuation, but that can
simultaneously realise other kinds of meaning (cf. Na’aman et al., 2017; Sampietro, 2016;
Dresner & Herring, 2010; Provine et al., 2007). These findings are confirmed by work on emoji
within the field of grapholinguistics (Dürscheid & Siever, 2017).
Danesi’s (2016) work marshals various strands of phatic, pragmatic and semantic
research to address the question of whether emoji constitute an independent language,
ultimately concluding that they do not (yet). Danesi distinguishes adjunctive emoji, that make
meaning alongside language, and substitutive emoji, that replace language, and observes that
emoji intelligibility declines in correlation to its independence from language. Danesi also
emphasises that unlike language, emoji interpretation does not require any formal instruction.
In describing emoji semiosis, Danesi foregrounds their ‘adding tone’ function, which is
congruent with the ‘tone modification’ function described by Herring & Dainas (2017).
A social semiotic perspective on emoji 5
2.2. Context-sensitive approaches
While work exploring how emoji make meaning independently of language has yielded
valuable insights, with sustained, widespread use, emojis’ specific pragmatic meanings may
have become diluted (Konrad, Herring & Choi, 2020). In response, some scholars have adopted
analytical frameworks that aim to describe how emoji make meaning in combination with
linguistic resources. In this vein, grapholinguistic research on emoji advocates context-specific
interpretation of their meaning (Dürscheid & Meletis, 2019). Accordingly, Dürscheid and
Meletis state, “this renders the emoji graphematically ambiguous, as the specific linguistic unit
it refers to is not fixed but variable and determined by the context or the reader’s interpretation
of a given text in which it is used” (2019, p.174). Similarly accommodating co-occurring
language into their analytical framework, Ge and Herring (2018) adapt rhetorical structure
theory to analyse how emoji sequences relate to their accompanying co-text rhetorically and
logically, concluding that in some respects, emoji appear to be “developing into an independent
language” (p. 15).
A related vein of argument that runs through analysis of emoji and emoticons is the
likening of these resources to gesture. At a surface level, this is motivated by the popularity of
emoji which iconically represent facial expressions and body gestures (i.e.: ‘Thumbs Up’
‘OK Hand’ !%&'). Describing the relationship between emoji and gesture, Dresner and Herring
(2014, p. 66) comment that their analysis “does not rule out an iconic mapping between the
function of emoticons and some bodily and facial movements”. However, a direct equivalence
of emoji and gesture risks proscription of emoji’s meaning potential – as Albert observes, “the
formal analogy between emoji faces in general and the corresponding facial expressions
provokes the misleading inference that there must also be a functional analogy” (2020, p. 68).
A more nuanced parallel drawn between these modes is that like co-speech gesture, the
A social semiotic perspective on emoji 6
meaning made by emoji is to varying degrees dependent on their linguistic co-text. This
dependent relationship to language prompts Abercrombie’s (1968) designation of modes such
as co-speech gesture as ‘paralinguistic’, a term we adopt here. Employing McNeill’s (1992)
diagnostic criteria for determining the degree to which semiotic modes can function
independently of language, Gawne and McCulloch observe that “gestures and co-speech emoji
are closely integrated into meaning with the accompanying speech/text” (2019, p. 13). They
argue that unlike language, emoji are global and synthetic, non-combinatoric, context-sensitive
and do not have standards of form. In so doing, they integrate pragmatic/phatic/attitudinal
findings of earlier work into a single model for emoji semiosis.
In treating emoji as a paralinguistic modality and allowing interpretation of their
meaning to be guided by co-text, Gawne and McCulloch’s work aligns with a social semiotic
approach such as that adopted by [Author B] and Parkwell (2019). This approach maps how
emoji realise meaning across the three metafunctions described in the social semiotic theory of
SFL. Accordingly, [Author B]’s analysis of emoticons used in Twitter posts described their
function of construing (generally positive) interpersonal meaning among interactants, but also
noted their textual function as discourse markers. In line with social semiotic work on
intermodal semiosis (cf. Martin, 2008), [Author B]’s analysis interpreted emoticons in context,
observing that “Viewed alone, emoticons display a high degree of ‘fuzziness’… interpersonal
meaning is more readily studied when we look at how emoticons work in tandem with
evaluative meanings made in the verbiage” (2012, p. 80). A further consequence of this
‘fuzziness’ is that meaning construed by emoticons cannot be described by more delicately
differentiated features of semiotic systems. Whereas in SFL theory linguistic resources
realising ATTITUDE can be subdivided into AFFECT, APPRECIATION, and JUDGEMENT, [Author
B] found that viewed in isolation, emoticons can only construe generalised ATTITUDE.
Moreover, with regards to distinguishing how emoticons construe solidarity and attitude,
A social semiotic perspective on emoji 7
[Author B] notes that “it is not possible to neatly compartmentalize emotion and solidarity.
Emotion is involved in the way solidarity is expressed since alignment with others is generally
a positive emotional experience” (2012, p. 68).
Also approaching emoji semiosis from a social semiotic perspective, Parkwell’s (2019)
analysis elaborated a framework for describing how meaning is made across modes. Parkwell
drew on social semiotic work on multimodality (Kress & van Leeuwen, 2001; O'Halloran,
2004), and on [Author C]’s analysis of the discourse of social media, in particular their
metafunctional analysis of hashtags. Parkwell’s analysis explored how a single emoji (‘toilet’
!()*+) can construe meaning in each of the metafunctions, concluding that “emoji are a highly
contextual, flexible modality, likely to continue to shift and morph with the changing needs
and contexts of social media users” (Parkwell, 2019, p. 9).
3. Systemic Functional Linguistics (SFL)
3.1. A Social Semiotic Approach to Exploring Emoji
As attested by the work of [Author B] and Parkwell (2019), a social semiotic approach
to exploring emoji offers a robust analytical framework for describing intermodal semiosis.
This framework is underpinned by the theoretical architecture of SFL as elaborated by Halliday
(1978) and colleagues. A key theoretical principle of SFL relevant to this study is the
complementarity of paradigmatic and syntagmatic relations as perspectives on semiosis.
Language users make meaning by instantiating choices from systems of language, and these
choices interact as a text unfolds. Paradigmatic relations describe the relationship between all
the potential options available to a language user at a particular point in a text, or as Halliday
and Matthiessen (2014, p. 22) indicate, “what could go instead of what”. Syntagmatic relations,
on the other hand, are concerned with “what goes together with what” (Halliday & Matthiessen,
2014, p. 22), offering insight into the patterns and regularities that occur as language unfolds.
A social semiotic perspective on emoji 8
In this research into the semiotic affordances of emoji, paradigmatic relations provide a view
on how emoji are organised into systems of options, while syntagmatic relations account for
their contribution to a text’s meaning as it unfolds as discourse.
Another key dimension of SFL useful for understanding emoji is the concept of
semiotic metafunctions, whereby semiotic resources simultaneously enact an ideational
function of construing experience, an interpersonal function of enacting relationships and a
textual function of organising discourse into coherent text. The choices in meaning most
relevant to this study are at the level of discourse semantics, concerning the ideational system,
IDEATION, the interpersonal systems, APPRAISAL and INVOLVEMENT, and the textual system,
PERIODICITY. These systems are explained in more detail in the method section where they form
the basis of the coding rubric applied to annotate the dataset.
3.2. Intermodal semiosis
Within the social semiotic tradition, multimodal discourse analysis (MDA) has
emerged as a robust framework for analysing how non-linguistic modes realise meaning, both
independently and in conjunction with language (Kress & van Leeuwen, 2001). MDA is
underpinned by the SFL theoretical architecture described above. Accordingly, multimodal
discourse analysts have elaborated systems that describe how paradigmatic and syntagmatic
choices unfold in modalities other than language, and how resources in these modes realise
meaning across metafunctions. A further, crucial region of enquiry of MDA is how meaning
enacted across modalities interacts, given that many genres employ multiple modalities within
a single text. This approach is informed by what Lemke terms the ‘combinatorial principle’ for
interpreting intermodal semiosis, which “shows us how we can mean more, mean new kinds
of meanings never before meant and not otherwise mean-able, when this process occurs both
within and across different semiotic modalities” (1998, p. 92). Lemke’s approach to describing
A social semiotic perspective on emoji 9
intermodality thus proposes that the most fruitful analysis of intermodal texts accounts for both
meaning made within modes and across them. The term ‘intermodality’ is favoured over
‘multimodality’ here as it foregrounds the relation among modes as the site of enquiry, rather
than the description of their semiosis in parallel (see Bateman, 2014). Reflecting on work
exploring the interaction among linguistic, paralinguistic and pictorial resources in pedagogic
contexts, Lemke notes that despite the analytical utility of separating these “into different
‘channels,’”, this approach “neglects the inherent unity of communicative meaning-making
which makes the co-ordination among channels not only possible, but normal” (1998, p. 94).
Accordingly, while the method adopted here begins by analysing linguistic and emoji semiosis
monomodally, description of meanings construed across texts reflects a unitary interpretation.
In this regard, the MDA approach parallels the context-sensitive approaches described in the
previous section, such as the rhetorical structure theory-informed approach employed by Ge
and Herring (2018) and the grapholinguistic approach taken by Dürscheid and Meletis (2019).
As the combinatorial principle has been applied to analyses of intermodality in other
contexts and genres, a number of more specific intermodal relationships have been observed.
Those most relevant to this study are intermodal coupling and minimum mapping. A social
semiotic multimodal discourse analysis approach is also relevant to the polemic regarding
whether emoji can be considered a stand-alone language (cf. Danesi, 2016). This approach
sidesteps the question by considering emoji a semiotic resource (like images, sounds, etc.) with
particular semiotic affordances. As such, while we refer to SFL’s ‘supramodal’ grammar to
describe these affordances, we recognise that in some regions of meaningmaking emoji afford
fewer and thus less delicate options for making meaning than language, but in others they afford
3.2.1. Intermodal coupling
A social semiotic perspective on emoji 10
Intermodal coupling provides the core unit of analysis for describing intermodal
semiosis. An intermodal coupling occurs when resources across communicative modes
interact, yielding a new semiotic unit. This definition follows the approach outlined by Painter,
Martin, and Unsworth (2013). While Painter et al.’s work was limited to interactions within
metafunctions (constrained by the principle of convergence/divergence), they acknowledge
that intermodal couplings can also occur across metafunctions. Work on how resources interact
across metafunctions (but within a single modality) has been especially productive in
describing the interaction of attitudinal and ideational resources (e.g.: Knight, 2013; Martin,
[Author A]; [Author D]). These resources frequently interact to construe language users’
evaluations about ideational targets; as Martin states, “feelings are always about something –
they are always interpersonal attitudes to ideational experience” (2004, p. 337). For example,
in the linguistic component of text #77 in the dataset for this study,
Best day of my life so far
the positive ATTITUDE (APPRECIATION) Best is targeting the ideational ENTITY day of my life so
far. Consequently, we would describe the resulting meaning as an evaluative coupling, notated
(following Martin et al., 2013) as [day of my life so far / +ve APPRECIATION] (‘+ve’, ‘-ve’ are
used as shorthand for ‘positive’, ‘negative’; APPRECIATIONis the subcategory of ATTITUDE
describing resources for construing aesthetic evaluations). Accounting for the role of emoji
resources in this text introduces an additional layer of complexity, as the positive ATTITUDE
realized by the emoji interacts with the positive APPRECIATION of Best, thus also
participating in the evaluative coupling. A principled description of this interaction requires us
to account for how meaning is distributed across these modes, as well as how their individual
meanings combine within the coupling. In response, we incorporate the principle of minimum
mapping into our analytical framework.
A social semiotic perspective on emoji 11
3.2.2. Minimum mapping
Complementing Painter et al.’s work, Zhao (2011) proposes the minimum mapping
hypothesis as part of a more principled framework for the analysis of meaning across modes.
This hypothesis states that “if verbiage and image can co-construe one aspect of a social action,
e.g. process (what is going on), participant (the participants that engage in the process), etc.,
they form a verbiage-image coupling” (Zhao, 2011, p. 171). Thus, while individual resources
might potentially construe generalized or multiplied meanings, when they co-occur in a text
their meaning potential is constrained to the region of meaning shared among them. See, for
instance, text #44:
Incheon Airport
Here, the emoji in isolation does not necessarily denote the ‘airport’ ENTITY (or rather, it is
a less salient option of its meaning potential); however, when combined with the lexical item
‘airport’ as a coupling, the meaning across these modes is specified as construing that ENTITY
(cf. Dainas & Herring’s, 2017 ‘mention’ function of graphicons).
4. Dataset and Method
The dataset considered in this study is a corpus of 1000 digital messages (SMS
messages and social media posts) collected in 2019 and containing at least one emoji, collected
from 50 undergraduate media students at an Australian University, with each student
contributing 20 messages. Each of these students was interviewed for one hour about the emoji
they used in these messages. Due to the scope of the study and the necessity for close textual
analysis of both the emoji and the co-text across multiple dimensions of meaning, a random
selection of 200 texts was analysed. Of this selection, approximately 40 (see Appendix) are
discussed in this study. The decision of which to include was guided by the methodological
principle of theoretical sampling as proposed by Glaser & Strauss (2006), whereby an “analyst
A social semiotic perspective on emoji 12
jointly collects, codes, and analyzes his data and decides what data to collect next and where
to find them, in order to develop his theory as it emerges” (p.45). Accordingly, the examples
presented in this paper were selected for their value in illustrating particular theoretical
phenomena noted by the analysts.
The method for analysing the data was qualitative, with individual messages manually
annotated according to a coding rubric. This rubric was designed to code for the kinds of
meanings that emoji can make intramodally (independent of language) and intermodally via
various kinds of relations with the linguistic component of the message. Categories of meaning
were drawn from Martin (1992) and Martin and White’s (2005) discourse semantics systems
of IDEATION (construing experience), APPRAISAL (expressing emotion and evaluation),
INVOLVEMENT (enacting solidarity) and PERIODICITY (organising text). Linguistic realisations
of features within these systems are shown in Table 1. Relationships between individual
features were annotated as couplings, drawing on the relational principles of intermodal
semiosis explained earlier in this paper. Accordingly, four kinds of intermodal couplings were
identified: convergent IDEATION, convergent ATTITUDE, convergent GRADUATION and
intermodal evaluative couplings. By coding the meaning realised by emoji first in isolation and
then in conjunction with linguistic co-text, this method allowed the researchers to identify
instances where the interaction between modalities impacted emoji meaning. In turn, these
instances form the basis of the results and discussion.
Things, activities and semiotic
coffee (#10 from dataset),
helping (#147),
publication (#52).
Activities or processes
involving one or more entity
they’ve taken it off Netflix
(#64), I had salad (#63)
States or changes affecting
one or more entities
Hyundai exec for Sale
Emotion or evaluation
AFFECT (emotion): happy
(#183), sad (#7);
A social semiotic perspective on emoji 13
wholesome (#181), freaks
Up or downscales attitudinal
cutest (#90).
Non-gradable resources for
construing solidarity among
interactants, including
resources that mark
interactants as part of a shared
community, such as
swearing/taboo lexis, slang,
and naming
twinning (#45) (slang
term for resembling
another person’s
Organisation of text in terms of
information structure
sometimes you gotta let
your skin breathe a little
(#200) (temporal
circumstance in Theme
Table 1: Coding Rubric for discourse semantic systems for annotation of layers in WebAnno.
The annotation software, WebAnno, was employed to systematically code the data.
WebAnno is a Java-based, web-hosted annotation program that accommodates Unicode
characters such as emoji and has a flexible, customizable interface for designing coding rubrics.
Data was first converted to .csv file format and uploaded into a new WebAnno project,
formatted for one entry per line. The software enabled two kinds of annotation to occur:
annotation of features in discrete layers (coding meaning for isolated semiotic resources, such
as a single emoji or lexical item) and annotation of types of relations between features. The
discourse semantic systems shown in Table 1 formed the annotation ‘Layers’ within WebAnno,
forming ‘tags’ with ‘tagsets’.
An example of how these layer features and couplings are shown when coded in the
WebAnno interface is found in Figure 1 – an excerpt of the coding for text #13. We can see
that each individual feature and coupling is labelled in the interface and, whilst it cannot be
shown in a static image such as this one, each label is highlighted in bold font when the mouse
pointer is held over it (like ‘cumulative’ in this image). The program abbreviates the features
A social semiotic perspective on emoji 14
in the visualisation: Att = Attitude; Ide = Ideation; Pos = Position (relating to the location in
the text where an emoji occurs); Pol = Polarity (specifying whether instances of ATTITUDE are
positive or negative). Coding of emoji resources for these features was conducted by
interpreting emoji intramodally (within a single mode) and then intermodally in batches of 40-
50 texts. After each batch the researchers would confer to discuss any issues in the coding
rubric or points of interest. In light of revisions to the coding rubric or how it should be applied
to particular cases, coding was repeated/updated where necessary.
Figure 1: example of WebAnno coding
Considering the importance of the context of communication for interpreting the
meaning of emoji, the authors of texts were interviewed and asked about the tenor of the text
(the relationship between the author and recipient or audience), the field of the text (the topic
or domain), and the mode (the platform via which the text was produced and disseminated).
Annotation results were compared to these descriptions. The degree of convergence
(agreement) between the annotation and the authors’ descriptions were coded as either ‘fully
agree’, ‘partially agree’ or ‘disagree’. The annotation fully agreed with the author descriptions
in 81% of cases, and if including partial agreement, accorded with 93% of instances.
5. Results and Discussion
Given the goal of mapping emoji semiosis according to SFL theory, a primary
outcome of the analysis was the identification of typical emoji realisations of semiotic choices
in SFL discourse semantic systems. Once this had been concluded, these realisations informed
A social semiotic perspective on emoji 15
analysis of more complex instances of intermodal semiosis. Accordingly, the results of this
study can be divided between the elaboration of a social semiotic analytical framework for
analysing emoji semiosis and the description of particular regions of intermodal meaning
realised by the interaction between emoji and language.
5.1. Typical emoji realisation of discourse semantic features
The first relevant result of the analysis is the description of typical meanings realised
by emoji. Identification of typical emoji realisations of discourse semantic features was guided
by an understanding of how intermodal semiosis acts to foreground particular regions of emoji
meaning potential. We are aware that capturing the dynamic, context-dependent semiosis of
emoji in a static representation such as a table inevitably simplifies and omits aspects of
individual emojis’ meaning potential; however given the repeated use of some emoji to
construe particular semiotic features we feel it is justified to note these typical realisations.
These realisations are summarised below and form the basis for the descriptions of semiosis
realised through the interaction between emoji and language discussed in the following
ATTITUDE is the discourse semantic system within the interpersonal metafunction
concerned with describing resources for construing evaluative meaning. Typical emoji
realisations of ATTITUDE include iconic representations of facial expressions, emblematic
gestures and ideational entities/occurrences/states with attitudinal connotations (Table 2).
Negative Polarity
facial expressions
!•ƒ !†ˆ ŒŽ !• !–˜
œ Ÿ ¡ £¤¥¦§¨© «¬ !-®¯°
emblematic gestures
A social semiotic perspective on emoji 16
Table 2: ATTITUDE realised by emoji.
GRADUATION is the discourse semantic system for upscaling or downscaling ATTITUDE.
In the data analysed here, emoji appear to only realise upscaling of ATTITUDE. This is achieved
either through repetition of emoji construing ATTITUDE, through emoji realising stylised icons
of intensified facial expressions (as compared to a baseline realisation such as the Smiling Face
!,-.), or through iconic realisation of ideational entities that connote paralinguistic features that
realise upscaled emphasis (Table 3). An example of the latter option is found in text #40, where
the raised volume in spoken language connoted by the ‘loudspeaker’ emoji !RSTUVW serves to
upscale the written language in the text.
! ! !ŠŠŠ‹‹‹ŒŒŒ•••ŽŽŽ•••
intensification of facial
‘grinning face with big eyes’
‘weary face’
iconic representation of
paralinguistic emphasis
There are
only FIVE tickets remaining (#40)
Table 3: GRADUATION realised by emoji.
INVOLVEMENT is the interpersonal system for expressing solidarity by employing
semiotic resources to encode meaning that can only be interpreted by members of a particular
community. In the context of emoji semiosis, these include emoji that are used as emblems
within particular communities and emoji that connote taboo meaning which is acceptable when
used among solidary interactants (Table 4).
encoding solidarity among members of
the LGBTQ+ community in text #12:
Love is love
Proud of love.
taboo semiosis
‘Pile of Poo’ emoji encoding positive ATTITUDE
among interactants in text #17:
A social semiotic perspective on emoji 17
Happy birthday to my fave twins!!!!
Table 4: INVOLVEMENT realised by emoji.
IDEATION is the discourse semantic system describing resources for construing
experience. Emoji typically encode ideational meaning through iconic representations of
physical entities, processes and qualities (Table 5). While emoji were observed to realise
entities and occurrences even independently of language, all instances where emoji realise state
figures occurred as intermodal couplings.
Iconic representation of entities
and processes
‘desert island’
!_`abcdef (#11)
‘person running’
! Éghijkl
Table 5: IDEATION realised by emoji.
The PERIODICITY system describes choices in how information is ordered within a text.
As emoji are less formally bound by syntagmatic relations to surrounding co-text than linguistic
resources, the coding rubric for PERIODICITY choices has been simplified to three features
corresponding to emoji location within a text: INITIATING (occurring before linguistic
resources), INTEGRATED (occurring after some and before other linguistic resources) or
CUMULATIVE (occurring after linguistic resources) (Table 6).
Where the
emoji is
located in the
!5678 We are finalists
in the category of
Fitness Service in
2019 Melbourne
North Local
Business Awards!
chillssssss ! !ÄÄÅÅÆÆÇÇÈÈ
do we really
have to do this
!uvwx (#22)
Table 6: PERIODICITY realised by emoji
A social semiotic perspective on emoji 18
5.2. Annotation of relations between features as couplings
The second relevant result of the analysis concerns the categorisation of coupling
relations found in the data. Couplings among semiotic resources can occur either within or
across metafunctions and modalities, thus coupling options can be represented as a matrix
organised around two variables: intramodal vs intermodal and intrametafunctional vs
intermetafunctional, as shown in Table 7 referring to text #90:
the cutest baby ! ! !ŠÞmnŒoŽßàpqrsÉ
Coupling of interpersonal
metafunction resources for
ATTITUDE (cutest) and
GRADUATION (cutest) within the
linguistic mode of text #90.
Coupling of interpersonal metafunction
resources for ATTITUDE (cutest) and
IDEATIONAL (baby) within the linguistic
mode of text #90.
Coupling of interpersonal
metafunction resources for
ATTITUDE in the linguistic mode
(cutest) and in emoji (ßà) in text
Coupling of interpersonal metafunction
resource of ATTITUDE in the linguistic
mode (cutest) with the ideational
metafunction resource of IDEATION in
the emoji mode (‘Cat Face’
Table 7: Applying a semiotic couplings matrix to text #90
Describing intermodal semiosis within a metafunction is substantially different to
describing intermodal semiosis across metafunctions, thus we have divided intermodal
couplings into two kinds: convergent intermodal couplings and evaluative intermodal
couplings. In turn, convergent intermodal couplings can be further specified by metafunction
or system, for instance IDEATIONAL convergent intermodal coupling or ATTITUDINAL
convergent intermodal coupling (illustrated in Table 8Error! Reference source not found.).
Realisation Examples
Leighton Beach 🏖 (#43)
Leighton Beach converges
with ‘beach’ 🏖
Do you want to
‘flexing biceps’
converges with the mental
process clause do you
want to to construe the
occurrence ‘lift weights’
new nose piercing ýþÿ
‘sparkles’ ýþÿ converges
A social semiotic perspective on emoji 19
with nose piercing to
construe the state
‘sparkling nose piercing’
its sad bc this is the standard
for literally 90% of the
internships i see online !qrst
sad converges with
‘persevering face’ !qrst
The good stuff !%&' ((#56)
Interaction between good
and ‘Ok hand’
Went to harry styles concert
with 0 expectations but
came back in awe???? What
a guy !defghi (#72)
what a guy converges with
‘Star-Struck’ !defghi
Youre such a little
! ! !ttuvvwwxxyz{ (#9)
such converges with
repetition of ‘mouse’ !tvwx
Is anyone selling Taylor
Swift tickets? œ. (#176)
Is anyone selling Taylor
Swift tickets? realises an
ideational OCCURRENCE,
while œ realises negative
ATTITUDE targeting this
Table 8: Discourse semantic features realised by intermodal couplings
However, it should be noted here that in instances where emoji and language converge
to construe ATTITUDE, that ATTITUDE will always have an IDEATIONAL target, thus an evaluative
coupling will also be construed. This is the case in text #56:
The good stuff !%&'
Here, emoji and language converge to construe positive APPRECIATION for the
IDEATIONAL target stuff, thus forming the evaluative coupling [stuff / +ve APPRECIATION]. As
posited by the minimum mapping hypothesis, the intermodal relationship exists between the
ATTITUDINAL meaning construed by each mode as well as between ATTITUDINAL and
IDEATIONAL meaning divided by mode, thus this text contains both a convergent attitudinal
coupling (good + !%&'| and an evaluative coupling (stuff + !%&'|.
5.3. Applying principles of minimum mapping to the dataset
A social semiotic perspective on emoji 20
The third relevant result of the analysis concerns the application of the minimum
mapping hypothesis to the interpretation of meaning made by the interaction between emoji
and language. Application of the principle of minimum mapping introduced in Section 3.2.2 to
emoji-language semiosis revealed that the semiotic relations between interacting resources are
temporary. Thus individual resources could act to specify differing aspects of their meaning
potential depending on what they were coupled with, the coupling’s co-text, and the text’s
context. For example, the emoji !%&' was used in texts #56 and #88:
#56: The good stuff !%&'
#88: Take 3 for the sea !%&' (more is welcome too :) ;) ).
In text #56 the emoji’s interaction with the verbiage good foregrounds its ATTITUDINAL
meaning (as the emoji iconically represents the hand gesture emblematic for ‘OK’), with the
resulting intermodal coupling construing positive APPRECIATION. Contrastingly in text #88
interaction with the verbiage three foregrounds the ideational aspect of the emoji realising the
ENTITY ‘three’ (as the emoji iconically represents a hand holding up three fingers). In this
dataset no instances were found of individual resources being repeated within a text with
divergent meanings foregrounded, most likely due to the short length of texts. However, the
underlying principle of temporary interaction allows for such divergence, especially in longer
Minimum mapping was useful in guiding analysis of interaction across modes within a
single metafunction, but it did not help us describe intermodal couplings where each mode is
realising meaning in a different metafunction. Take, for example, text #176:
Is anyone selling Taylor Swift tickets? œ.
In this text, IDEATIONAL meaning is realised by the linguistic component of the text, while
ATTITUDINAL resources are realised by the emoji component. We can interpret the emoji as
construing generalised negative ATTITUDE, and the field of the IDEATIONAL meaning is
A social semiotic perspective on emoji 21
established as ‘obtaining tickets to a Taylor Swift concert’. But with no interacting resources
within the interpersonal metafunction to minimally map onto, the ATTITUDE of the intermodal
evaluative coupling remains unspecified (cf. [Author B]’s description of ‘fuzzy’ meaning).
Plausible interpretations of the emoji include negative AFFECT, reflecting the author’s concern
about finding a ticket, or negative APPRECIATION for the absence of tickets. Consequently, in
cases such as this we refrain from specifying more delicate ATTITUDE features and notate the
coupling as [obtaining tickets to a Taylor Swift concert / -ve ATTITUDE].
5.4. Taxonomic relations within convergent IDEATION ENTITY couplings
The fourth relevant result of the analysis concerns the distribution of meaning across
modes in intermodal, convergent ideational couplings. A particular region of intermodal
semiosis that was observed in the data is that convergent ideational meaning realising ENTITIES
across modes often follows a pattern of hypo/hypernymy, whereby meaning construed by one
mode will be a superordinate or subordinate to meaning construed by the other. For example,
text #55:
Thoroughly disappointed with the lack of milk !}~
Here the verbiage milk realises one primary ENTITY – ‘milk’ (other potential interpretations are
possible, as [etc.] in Figure 2 allows for). Conversely, the ‘baby bottle’ entity iconically
realised by the !}~ emoji can be divided into the entities that constitute the graphicon: ‘bottle’,
‘milk’, ‘teat’, and so forth. Applying the minimum mapping hypothesis, we propose that the
intermodal meaning jointly construed by the emoji and language is the ‘milk’ entity construed
by both modes, while other entities potentially construed by only one mode are not realised.
The intermodal semiosis of this text is summarised in Figure 2, with linguistic semiotic
resources highlighted in red, emoji in blue, and intermodal semiosis in green. In turn, the
intermodally construed entity ‘milk’ is nested within a linguistically realised STATE, ‘lack of
A social semiotic perspective on emoji 22
Figure 2: intermodal ideational semiosis in text #55
A more complex example of shared IDEATIONAL meaning mapped across modes can be
seen in text #198:
If you want a break from studying and pet some animals !_`abc there is a petting zoo at
the Physics Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!! It’s
entirely free and there are trained handlers with lots of animals !!!ƒ„…‡ˆ‰ŠŒ•Ž•
Here we can see how the linguistically construed IDEATIONAL ENTITIES some animals, petting
zoo, trained handlers and lots of animals can be organised into a taxonomy that intersects with
the meaning of the ‘cow’ !ƒŠŒŽ, ‘rabbit’ !ˆ and ‘goat’ ! emojis, as shown in Figure 3.
Collectively, language and emoji realise a taxonomic category coded as ‘petting zoo animals’.
This relationship echoes Zhao’s (2011) description of ‘metonomysing’ relations in language-
image texts.
A social semiotic perspective on emoji 23
Figure 3: intermodal ideational semiosis in text #198
5.5. Intermodal specification
A fifth noteworthy pattern observed in the data is that intermodal interaction can serve
to specify particular dimensions of the meaning potential of each mode. Consider, for example,
text #169:
Smaller than expected but cosy none the less (: Saw ! ! !”•…˜š›‰œžŸ+ ! ! ¡¢£¤¥¦§¨©. I reckon it
would be a whole 'nother scene at night. Wouldn't mind coming back in spring
sometime as well with all the flowers in bloom~
Here we can see how Saw ! ! !”•…˜š›‰œžŸ construes an intermodal OCCURRENCE which can be
rendered linguistically as ‘I saw pigs, chickens and goats’. However, following from the
syntactical patterning of the preceding co-text, rather than construing its typical, monomodal
meaning of ‘knife and fork’ or ‘cutlery’, the ! ¢¤¥ emoji construes the process ‘eating’ as part of
the OCCURRENCE which can be rendered as ‘ate rabbit’: ! ! ¡¢£¤¥¦§¨©. This interpretation also sustains
continuity in the IDEATIONAL field of the text, with farm animals remaining the targets of the
author’s activities (as opposed to incongruously suggesting that the author saw pigs, chickens,
A social semiotic perspective on emoji 24
goats, cutlery and rabbits). From a minimum mapping perspective, the verbal group dimension
of the ! ¢¤¥ emoji has been foregrounded because it intersects with the meaning realised by the
co-text, while the ENTITY dimension is backgrounded.
Another instance of linguistic co-text shifting the meaning of emoji from an
IDEATIONAL ENTITY to an OCCURRENCE occurs in text #67:
I can see how "interested" everyone is ! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯
Here the quotation marks around interested’ suggest this word could be interpreted
ironically/sarcastically, implying that ‘everyone’ is in fact not interested. This opens a potential
region of meaning with which the !ª«¬-®¯ emoji which follows can be cross-referenced, inviting a
reader to consider what relationship might exist between the STATE ‘everyone is (not)
interested’ and the iconically realised ENTITY ‘mobile phone’. Accordingly, a plausible
interpretation of the ! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯ emoji sequence is that it construes an OCCURRENCE rendered
as ‘looking at mobile phones’ which in many contexts is associated with lack of interest. In this
case, it is noteworthy that the interaction among modes functions to specify meaning bi-
directionally, as both the quoted verbiage, “interested” and the !ª«¬-®¯ emoji are examples of
semiotic resources that are to some degree indeterminate and context dependent. By applying
a minimum mapping approach to interpreting intermodal semiosis, we can see how the
dimensions of individual semiotic resources that are shared across modes are foregrounded,
while those that are not shared are backgrounded. The intermodal semiosis occurring in this
text is summarised in Figure 4.
A social semiotic perspective on emoji 25
Figure 4: intermodal ideational semiosis in text #67
5.6. Proximal attitudinal prosody
A final observed pattern in attitudinal meaning mapped across modes is that ATTITUDE
realised by emoji typically ‘washes’ over adjacent co-text both forwards and backwards. As
such we have termed this proximal attitudinal prosody. As Halliday (1979) and Martin (1995,
1996) have described, different kinds of meaning unfold according to different structural
configurations. Halliday describes interpersonal meaning as “strung throughout the clause as a
continuous motif or colouring” whose “effect is cumulative” (1979, p. 67). For example, in text
#27 we can see how the positive ATTITUDE construed by the words Merry, safe, and happy
accumulates and intensifies over the course of the text:
A belated Merry Christmas and a safe and happy holidays to y'all !°±²³´µýþÿ
It is thus unsurprising to see ATTITUDE construed by emoji contributing to prosodic meaning,
for instance in text #52:
Extremely chuffed that my first publication, an article for Discourse & Society, is now
online - a nice wrap on the first fortnight of my PhD !Z[\]^
Here the general positive ATTITUDE construed by ‘Grinning Face with Big Eyes’ !Z[\]^ builds on
the positive ATTITUDE realised by chuffed and nice, combining to realise convergent ATTITUDE
couplings with these lexical items of AFFECT and APPRECIATION, respectively.
A social semiotic perspective on emoji 26
A more unexpected observation in the interaction between intermodal ATTITUDE
resources is that while the prosodic unfolding of attitudinal meaning in the linguistic mode is
largely unidirectional, attitudinal emoji appear to demonstrate a freer relation to their co-text.
An example of this can be found in text #99:
Me & My worst frenemy.
In this text, the first ATTITUDE resource is the lexical item worst, which construes upscaled
negative APPRECIATION. This is followed by frenemy a portmanteau of the lexical items
‘friend’ and ‘enemy’ – which denotes both positive and negative AFFECT. Looking at only the
linguistic resources, the attitudinal prosody of this text would appear to be either negative or
ambivalent. However, interaction with the positive ATTITUDE construed by the ‘Red Heart’
emoji foregrounds the positive ATTITUDE of the verbiage while backgrounding the negative,
resulting in an overall ATTITUDE construed in the text being positive. This suggests that the
ATTITUDE construed by the emoji is ‘washing’ backwards over the preceding co-text in a kind
of reverse prosody. This aspect of convergent attitudinal couplings parallels the ‘tone
modification’ function observed in pragmatics-informed descriptions of emoji semiosis such
as Herring & Dainas (2017).
While the relation of emoji to attitudinal prosody does appear to be freer than that of
linguistic resources, attitudinal emoji can nonetheless be governed by syntagmatic relations
within a text. For instance, in text #111 the ‘Crying Face’ !·¸¹º» and ‘Red Heart’ emoji only
interact with the clauses that precede them, acting as culminations for the linguistic ATTITUDE
construed therein:
Gonna miss this school !·¸¹º» Love every one of you boys, thanks for the memories
This suggests attitudinal meaning realised by emoji behaves somewhat differently from
linguistic ATTITUDE, both in how it interacts with proximal attitudinal resources and how it can
serve to conclude stretches of prosody. This observation parallels work both within social
A social semiotic perspective on emoji 27
semiotics (such as Knox, 2009) and in other theoretical traditions that notes how emoji and
emoticons can function like clause-final punctuation (cf. Na’aman et al., 2017; Sampietro,
2016; Dresner & Herring, 2010; Provine et al., 2007).
6. Conclusion
This paper has presented a social semiotic perspective on emoji semiosis. Informed by
the theoretical architecture of SFL and social semiotic work on MDA, we have proposed an
analytical framework that describes interactions in meaning realised by emoji and linguistic
resources. This framework was applied to the data to map typical emoji realisations of choices
from discourse semantic systems across metafunctions, thus revealing patterns in how meaning
is construed across modalities. It was observed that intermodal convergent IDEATIONAL
meaning can be described as a shared IDEATIONAL taxonomy wherein meaning realised in one
modality is sub- or superordinate to the other, thus construing an intermodal field of discourse.
A second noteworthy interaction between emoji and language is that a minimum mapping
approach to intermodality reveals that the region of meaning potential that overlaps across
modes corresponds with the region of meaning realised by each mode. This supports the
productivity of an MDA-informed approach to interpreting emoji semiosis and provides a
valuable guiding principle for emoji-language intermodality. A third pattern observed in the
interaction between emoji and language is that the ATTITUDE realised by emoji behaves
somewhat differently from linguistically construed ATTITUDE insofar as it appears to interact
prosodically with proximal linguistic resources occurring both before and after in the co-text.
These results constitute a modest advancement towards mapping the relations
governing how emoji and language jointly construe meaning. They take us beyond the correct
but vague observations regarding the fuzziness and context-dependency of emoji prevalent in
the area of study to date, and offer the potential for a more principled study of these relations
A social semiotic perspective on emoji 28
in future. It might be especially fruitful to combine the social semiotic approach of this study
with quantitative approaches in Artificial Intelligence (cf. Miller et al., 2016) and theories of
communication (cf. Veszelszki, 2017) to explore how these relations unfold across larger
While the theoretical framework of SFL and the MDA methodology differ from more
prevalent pragmatics-informed explorations of emoji semiosis, a number of points of
intersection between the two approaches were found. Methodologically, an MDA approach to
analysing how emoji and language interact to make meaning is congruent with work informed
by similarly intermodal frameworks such as the adapted rhetorical structure theory employed
by Ge and Herring (2017) and the grapholinguistic approach of Dürscheid and Meletis (2019).
In terms of results, convergent ideational couplings appear to realise the ‘mention’ function
described by Herring and Dainas (2017), while the dynamics of proximal attitudinal prosody,
where attitude realised by emoji ‘washes’ across co-occurring language, largely corresponds
to Danesi’s (2016) ‘adding tone’ function. These intersections both validate the findings of this
work and suggest the two traditions might be fruitfully cross-referenced in future research.
A salient limitation of this study is its limited dataset. Consequently, numerous semiotic
functions observed by other researchers working with more varied data were not observed in
our data. For instance, it would be valuable to apply the analytical framework proposed here to
texts composed solely of emoji so as to explore how emoji construe meaning in the absence of
language. In light of recent work on emoji sequences such as (Ge & Herring, 2018), this
dimension of emoji use merits closer attention. A further limitation of this study is that the data
and method employed are unable to account for the role the widely used emoji prediction
features in digital keyboards play in suggesting particular combinations of emoji and written
language. Integrating results of work in this area such as Barbieri, Ballesteros, Ronzano &
Saggion (2018) into the design of future research would expand the descriptive power of
A social semiotic perspective on emoji 29
resulting models.
7. Acknowledgements
The research presented in this study was funded by the Australian government.
Emoji glosses are sourced from (accessed 11.11.2020)
A social semiotic perspective on emoji 30
8. References
ABERCROMBIE, D. (1968). Paralanguage. British journal of disorders of communication,
3(1), 55-59.
ALBAWARDI, A. H. (2018). Digital literacy practices of Saudi female university students.
University of Reading,
ALBERT, G. (2020). Beyond the binary: Emoji as a challenge to the image-word distinction.
In C. Thurlow, C. Dürscheid, & F. Diémoz (Eds.), Visualizing Digital Discourse (pp.
65-80): De Gruyter Mouton.
BARBIERI, F., BALLESTEROS, M., RONZANO, F., & SAGGION, H. (2018). Multimodal
emoji prediction. arXiv preprint arXiv:1803.02392.
BATEMAN, J. A. (2014). Text and image: A critical introduction to the visual/verbal divide:
CRYSTAL, D. (2001). Language and the Internet. Cambridge University Press.
DAINAS, A., & HERRING, S. C. (2021). Interpreting emoji pragmatics. In C. Xie, F. Yus, &
H. H. (Eds.), Internet pragmatics: Theory and practice. Amsterdam: John Benjamins.
DANESI, M. (2016). The semiotics of emoji: The rise of visual language in the age of the
internet: Bloomsbury Publishing.
DRESNER, E., & HERRING, S. C. (2010). Functions of the nonverbal in CMC: Emoticons
and illocutionary force. Communication theory, 20(3), 249-268.
DRESNER, E., & HERRING, S. C. (2014). Emoticons and illocutionary force. In Perspectives
on Theory of Controversies and the Ethics of Communication (pp. 81-90): Springer.
DÜRSCHEID, C. & MELETIS, D. (2019). Emojis: A Grapholinguistic Approach. In: Y.
Haralambous (ed.), Graphemics in the 21st Century. Brest: Fluxus Edition (pp. 167–
183): Fluxus Editions.
DÜRSCHEID, C., & SIEVER, C. M. (2017). Jenseits des Alphabets–Kommunikation mit
Emojis. Zeitschrift für germanistische Linguistik, 45(2), 256-285.
GAWNE, L., & MCCULLOCH, G. (2019). Emoji as digital gestures. Language@ Internet,
GE, J., & HERRING, S. C. (2018). Communicative functions of emoji sequences on Sina
Weibo. First Monday.
GLASER, B. G., & STRAUSS, A. L. (2006). Discovery of grounded theory: Strategies for
qualitative research: Routledge.
HALLIDAY, M. A. K. (1978). Language as social semiotic: The social interpretation of
language and meaning. Edward Arnold.
HALLIDAY, M. A. K. (1979). Modes of meaning and modes of expression: types of
grammatical structure, and their determination by different semantic functions. In D. J.
ALLERTON, E. CARNEY , & D. HOLDCROFT (Eds.), Function context in linguistic
analysis: Essays offered to William Haas (Vol. 1, pp. 57-79). Cambridge University
HALLIDAY, M. A. K., & MATTHIESSEN, C. M. I. M. (2014). An introduction to functional
grammar (Vol. 4th). Edward Arnold.
HAO, J. (2015). Construing biology: An ideational perspective. University of Sydney,
HERRING, S. C., & DAINAS, A. (2017). “Nice picture comment!” Graphicons in Facebook
comment threads. Paper presented at the Proceedings of the 50th Hawaii International
Conference on System Sciences.
HERRING, S. C., & DAINAS, A. R. (2020). Gender and Age Influences on Interpretation of
Emoji Functions. ACM Transactions on Social Computing, 3(2), 1-26.
KNIGHT, N. K. (2013). Evaluating experience in funny ways: how friends bond through
A social semiotic perspective on emoji 31
conversational hum. Text & Talk, 33(4-5), 553.
KONRAD, A., HERRING, S. C., & CHOI, D. (2020). Sticker and Emoji Use in Facebook
Messenger: Implications for Graphicon Change. Journal of Computer-Mediated
Communication, 25(3), 217-235.
KNOX, J. S. (2009). Punctuating the home page: image as language in an online newspaper.
Discourse & Communication, 3(2), 145-172.
KRESS, G. R., & VAN LEEUWEN, T. (2001). Multimodal discourse: The modes and media
of contemporary communication: Edward Arnold.
LEMKE, J. L. (1998). Multiplying meaning. IN J. R. MARTIN & R. VEEL (Eds.), Reading
science: Critical and functional perspectives on discourses of science (pp. 87-113):
Psychology Press.
HECHT, B. (2016). “Blissfully Happy” or “Ready toFight”: Varying Interpretations
of Emoji. Paper presented at the Proceedings of the International AAAI Conference on
Web and Social Media.
MARTIN, J. R. (1992). English text: system and structure. Philadelphia: John Benjamins
MARTIN, J. R. (1995). Text and clause: Fractal resonance. Text-Interdisciplinary Journal for
the Study of Discourse, 15(1), 5-42.
MARTIN, J. R. (1996). Types of structure: deconstructing notions of constituency in clause
and text. In Computational and conversational discourse (pp. 39-66): Springer.
MARTIN, J. R. (2004). Mourning: How We Get Aligned. Discourse & Society, 15(2-3), 321-
MARTIN, J. R. (2008). Tenderness: realisation and instantiation in a Botswanan town. Paper
presented at the 34th International Systemic Functional Congress, Syddansk
University, Odense.
MARTIN, J. R., & ROSE, D. (2007). Working with Discourse: Meaning Beyond the Clause (2
ed.): Bloomsbury.
MARTIN, J. R., & WHITE, P. R. R. (2005). The Language of Evaluation: Appraisal in
English. Palgrave Macmillan.
MCNEILL, D. (1992). Hand and mind: What gestures reveal about thought: University of
Chicago press.
MILTNER, K. M. (2020). "“One part politics, one part technology, one part history”: Racial
representation in the Unicode 7.0 emoji set." New Media & Society.
NA’AMAN, N., PROVENZA, H., & MONTOYA, O. (2017). Varying linguistic purposes of
emoji in (twitter) context. Paper presented at the Proceedings of ACL 2017, Student
Research Workshop.
NISHIMURA, Y. (2015). A sociolinguistic analysis of emoticon usage in Japanese blogs:
Variation by age, gender, and topic. AoIR Selected Papers of Internet Research.
O'HALLORAN, K. (2004). Multimodal discourse analysis: Systemic functional perspectives:
A&C Black.
PAINTER, C., MARTIN, J. R., & UNSWORTH, L. (2013). Reading visual narratives.
PARKWELL, C. (2019). Emoji as social semiotic resources for meaning-making in discourse:
Mapping the functions of the toilet emoji in Cher’s tweets about Donald Trump.
Discourse, Context & Media, 30, 100307.
PROVINE, R. R., SPENCER, R. J., & MANDELL, D. L. (2007). Emotional expression online:
Emoticons punctuate website text messages. Journal of language and social
psychology, 26(3), 299-307.
REZABEK, L., & COCHENOUR, J. (1998). Visual cues in computer-mediated
A social semiotic perspective on emoji 32
communication: Supplementing text with emoticons. Journal of Visual Literacy, 18(2),
SAMPIETRO, A. (2016). Exploring the punctuating effect of emoji in Spanish Whatsapp
chats. Lenguas modernas(47).
SKOVHOLT, K., GRØNNING, A., & KANKAANRANTA, A. (2014). The communicative
functions of emoticons in workplace e-mails. Journal of Computer-Mediated
Communication, 19(4), 780-797.
THOMPSON, D., & FILIK, R. (2016). Sarcasm in written communication: Emoticons are
efficient markers of intention. Journal of Computer-Mediated Communication, 21(2),
VESZELSZKI, Á. (2017). Digilect: the impact of infocommunication technology on language:
Walter de Gruyter GmbH & Co KG.
WALTHER, J. B., & D’ADDARIO, K. P. (2001). The impacts of emoticons on message
interpretation in computer-mediated communication. Social Science Computer Review,
19(3), 324-347.
ZHAO, S. (2011). Learning through multimedia interaction: the construal of primary social
science knowledge in web-based digital learning materials. (Dissertation/Thesis).
its sad bc this is the standard for literally 90% of the media/pr/marketing internships
i see online
Youre such a little
! ! !ttuvvwwxxyz{
Coffee before anything
Rainbow beach in Brisbane
an expensive estate. Just opposite the Great Sandy
National Park, which house a steep trail!¼½¾¿ÀÁÂà towards a hidden aboriginal craving site.
Love is love
Proud of love.
Finally an excuse to post this photo I took that Abigail refuses to post herself! Happy
21st birthday @abigailmarks
! !!,"-#$%&'()*+./01234
Happy birthday to my fave twins!!!!
do we really have to do this
A belated Merry Christmas and a safe and happy holidays to y'all
Hyundai exec for Sale 🥳🥳
There are only FIVE tickets remaining for BABSOC’s
Fungal Art Workshop! Secure your place now before tickets run out ! Éghijkl
Leighton Beach
no worries good luck !!!
Twinning it with this babe
! !`abcùúû
Extremely chuffed that my first publication, an article for Discourse & Society, is
now online - a nice wrap on the first fortnight of my PhD
Thoroughly disappointed with the lack of milk
The good stuff
So i had salad
they’ve taken it off Netflix tho
! !ŠŠ‹‹ŒŒ••ŽŽ••
You are welcome
buy my tacos next week biatch
A social semiotic perspective on emoji 33
I can see how "interested" everyone is
! ! ! !ªªªª««««¬¬¬¬----®®®®¯¯¯¯
Went to harry styles concert with 0 expectations but came back in awe???? What a
Best day of my life so far
Take 3 for the sea
(more is welcome too :) ;) )
We are finalists in the category of Fitness Service in 2019 Melbourne North Local
Business Awards!
the cutest baby
! ! !ŠÞmnŒoŽßàpqrs
Me & My worst frenemy.
Gonna miss this school
Love every one of you boys, thanks for the memories
been once, theyre pretty fun
Thanks so much for helping me out (and it was lovely to catch up last week!)
Smaller than expected but cosy none the less (: Saw
! ! !”•…˜š›‰œžŸ
! ! ¡¢£¤¥¦§¨©
. I reckon it
would be a whole 'nother scene at night. Wouldn't mind coming back in spring
sometime as well with all the flowers in bloom~
Is anyone selling Taylor Swift tickets?
Embracing my skin more than ever because damn it’s bloody important; sometimes
you gotta let your skin breathe a little !ÏÐÑÒ Also, my baby hair is getting out of control,
I can literally construct a fringe out of it
! !ÞÞßßàà
u r so wholesome
What a lovely thing to see! So happy you and kiddos are safe, it must be so difficult
for you to keep things together right now, but I just wanted to commend you on your
Do you want to
new nose piercing ýþÿ
If you want a break from studying and pet some animals
there is a petting zoo at
the Physics Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!! It’s
entirely free and there are trained handlers with lots of animals
so many freaks on here will literally jump at peoples throats to defend objectively
shitty people just bc they make mediocre samefaced art thats vaguely popular in
obscure internet circles it's SO embarrassing. couldn't be me
List of texts cited
... according to Logi, L. & Zappavigna, M (2021) is called co-text. The purpose of this paper is an attempt to understand the role of the interaction of non-linguistic elements in media communications, as well as the contextual meaning they reflect in specific communicative situations. ...
... Over time, as well as with the intensity of the use of written communication through individual social networks, the use of pictographic images would know an increased development, such as, that their presence in any communication has become so natural, that it seems that the replacement of linguistic and non-linguistic means is fully acceptable and accessible. Emoji are a subcategory of what Herring and Dainas (2017) classify as 'graphicons' (graphical icons) a classification of graphical features found on social media platforms, while the first recorded case of emoticon use occurred in 1982 on the Carnegie Mellon bulletin board, emoji was created in Japan in 1997 (Logi, L. & Zappavigna, M: 2021). In 2015, a truly remarkable event occurred. ...
... It seems that their diversity in use is passing the phase from pictographic image to ideographic image, as the meaning and semantic shades of the images are increasing added everyday to communication. Emoji typically encode ideational meaning through iconic representations of physical entities, processes and qualities (Logi & Zappavigna, 2021). From the semantic point of view, we are interested in observing the relationships that are created between visual representation and references, including objects, things and ideas (Fan, 2006) For example: in one of the posts of the heads of European counturies announcing the construction of a new stadium for fans of the local team, which has achieved consecutive victories, the communication is accompanied by pictographic images that create ideographic relationships with the written text (I know that football is a very important topic for the residents of Silesia. ...
... However, since emojis not only include expressive faces but also iconic representations of real-world entities and activities, emojis can have a referential function (Logi & Zappavigna, 2021). Emojis can also be used to visually structure the text, for example, by introducing the message or substituting the punctuation marks (Escouflaire, 2021). ...
... Emojis can also be used to visually structure the text, for example, by introducing the message or substituting the punctuation marks (Escouflaire, 2021). The involving function applies specifically to emojis that are used as emblems of particular communities like the of the lesbian, gay, bisexual, transgender, queer, and others (LGBTQ+) community (Logi & Zappavigna, 2021). ...
... Third, political actors could seek to retain their seriousness and respectability (Lalancette & Raynauld, 2019) by avoiding emotional emojis. Due to the wide range of emojis, parties can integrate less expressive and more neutral emojis and apply emoji functions like the referential, structural, or involving functions (Escouflaire, 2021;Logi & Zappavigna, 2021). Given these three potentially competing hypotheses, we ask the following research questions: (2023) topics, it can be assumed that the national flag emoji is more likely to be used. ...
Full-text available
Emojis have become ubiquitous in digital communication, but we know relatively little about how they are used in political and campaigning contexts. To address this deficit, we analyze the use of emojis in the Facebook communication of parties in 11 European countries during the 2019 European election campaign. Results indicate that the use of emojis by political parties differs significantly from general online communication. Political parties more often use neutral and representational (such as flags) emojis than emotional and facial emojis to draw users' attention while maintaining a serious appearance of their content. Based on our empirical results, we develop a typology to characterize the mixture
... A transitivity analysis contributes to the ideational metafunction by investigating the syntax, participants, and circumstances of an utterance to comprehend the experience expressed by the speakers [40]. SFL is applicable to healthcare interactions [41], and only a few researchers have used it to investigate digital communication, such as emoji use [42]. ...
Full-text available
Background Although healthcare professionals (HCP) undergo communicative skills training, these are sometimes unsatisfactory for patients (empathy, discussion managing). Existing coding tools overlook the interaction and patients' responses. Meanwhile, remote consultations are redefining communication channels. While some researchers adapt those tools to telehealth, few investigate written interactions. Objective To identify and evaluate coding tools for healthcare interactions and examine their suitability for written interactions. Methods We conducted a meta-narrative review in PubMed, PsycINFO, Embase, Web of Science, CINAHL, and Scopus databases up to December 2022 with Communicati* AND Human* AND Linguistic* AND Professional-Patient Relation* as search terms. We extracted data regarding methodology, unit of analysis (UoA), coding categories, reliability, strengths, weaknesses, and inter-rater reliability (IRR). Results We identified 11 mixed-methods tools. Qualitatively, coding dimension was focused (n = 6) or comprehensive (n = 5). Main quantitative methods were descriptive statistics (n = 4) and cross-tabulations (n = 4). Main UoA was utterance (n = 7). Relevant categories were processes (n = 4), content (n = 3), emotional expressions and responses (n = 3), and grammatical format (n = 2). IRR ranged from 0.68 to 0.85 for coding categories. Conclusion Despite similarities, category terminologies were inconsistent, one-sided, and mostly covered conversation topics and behaviours. A tool with emotional and grammar categories could bridge the gap between a speaker's intended meaning and the receiver's interpretation to enhance patient-HCP communication. Furthermore, we need empirical research to determine whether these tools are suitable for written interactions. Innovation This review presents a comprehensive and state-of-the-art overview of healthcare interactions' coding tools and identifies their barriers. Our findings will support communication researchers in selecting appropriate coding tools for evaluating health interactions and enhancing HCP training.
... Emojis refer to various faces that describe emotions but most of them do not inherently have to do anything with emotion(Otemuyiwa, 2017). Emojis play an important part in showing emotion and shifting stand(Zappavigna and Logi, 2021). ...
Full-text available
This research highlights the differences of texting behaviour shown by male and female university students. Textism features consist of interesting characteristics used in text messaging such as abbreviations, emojis, stickers and GIF (Graphic Interchange Format). This study was conducted to analyse the usage of abbreviation and emojis among male and female students. The study was conducted on male and female university students aged from 19 to 25 years old through Google Form that acts as a replication of texting applications such as WhatsApp and Telegram messenger. It was found that male students prefer to reply in a simple form compared to female students. Moreover, there were five additional new features identified in the study which were added letters, Malay stylisation, double words, missing space and expression-based spellings. Besides, in terms of the differences of emojis used, female students applied wider types of emojis compared to male students.
... According to Coats (2016), emojis can be associated with pragmatic functions such as interpersonal value, semantic value, and metaphoric value. Logi and Zappavigna (2021) analysed emojis by applying Systemic Functional Linguistics because they can represent lexical, affective, and textual meanings. Luzón (2023) identifies emojis as non-linguistic markers of stance (attitude markers) and engagement (attention-getting resources, appeals to shared knowledge, and discipline-related humour). ...
Full-text available
This study examines the emerging digital genre of the Twitter conference, which remediates the traditional academic conference. Twitter conference presentations (TCPs) are composed of six-tweet threads where academics share their ongoing research projects. In the context of Open Science, this paper aims to analyse how academics craft these presentations to reach diverse audiences and increase the visibility and impact of scientific knowledge. The study analyses a corpus of 55 TCPs (330 tweets) to identify textual and multimodal markers of digital academic discourse that can function as stance and engagement markers. The findings show that engagement markers were more frequent than stance markers, particularly in terms of appeals to shared knowledge and attention-getting resources. Appeals to shared knowledge are conveyed through specialised terminology, abbreviations, references, and hashtags, while attention-getting resources consist mostly of symbols, images, emojis, and mentions. The results highlight the importance of effectively orchestrating modes and capitalising on Twitter affordances to balance academic discourse conventions with the informal register. This approach can aid in disseminating scientific knowledge on this platform to a wider audience, thus contributing to the democratisation of science.
... For example, a combination of multiple approaches could be applied to a larger dataset that covers various forms of comments, such as forum comments (as seen in Yiqiong Zhang et al., 2022). Additionally, more in-depth analysis could be conducted, focusing on specific practices (such as intermodal coupling, as seen in Logi and Zappavigna, 2021) to advance multimodal theories. Furthermore, exploring more advanced forms of danmu art, which resemble ASCII art in text-based communication, would be interesting. ...
Full-text available
50 days of free access until June 23, 2023: Danmu (anonymous superimposed video comments) is a popular form of communication on Chinese and Japanese video sharing sites. While previous studies primarily focused on the verbal aspects of danmu comments, there is a growing interest in exploring their multimodal features. This study investigates the unique potential of danmu comments to communicate visual meaning, interact with on-screen content, and thereby shape audience perception. Informed by a social semiotic approach to multimodality and relevant pragmatic theories, the study analyzed 50 screenshots of visually significant danmu comments to understand the resources used by commenters to craft visual comments and the relationship between these comments and the screen. Our findings revealed that four key resources were utilized to create visual comments: arrows, kaomoji, context-specific special characters and symbols, and ASCII art. Additionally, five types of relationships were identified between visual danmu comments and the screen, including deictic, emphasizing, complementing, extending, and independent. This study provides an up-to-date examination of the possibilities for visual expression in textual communication and extends previous research on semiotic resources in social media. It also discusses the role of danmu visual play as internet memes and the emergence of danmu visual grammar.
In January 2021, the shares of the brick-and-mortar video games retail chain GameStop exploded in value. At the same time, billboards on highways and ads in Times Square in New York City used cryptic visuals and seemingly meaningless emojis with diamonds and rockets. Mainstream media soon had an explanation: This was the story of how the subreddit r/wallstreetbets had mobilized thousands of retail investors in a fight against evil hedge funds. Based on a case study of r/wallstreetbets and the GameStop incident, we analyze how idiosyncratic internet culture was incorporated into a broadly resonant and emotionally inflected narrative that lionized the ‘little man’, focusing on both individual profits and collective grievances. Through a theoretical framework combining sociological theories of internet culture and framing analysis, we identify an overall communication structure that drew on three interconnected discursive layers: idiocultural memes, investment-specific information, and a moralized, collectivized injustice frame with heroes and villains. We further argue that the GameStop (GME) incident instantiates a case of the politicization of personal finance, where the investment practices and strategies of ordinary people were transformed into a political issue. As such, the article makes two contributions to the existing literature. First, we contribute to the nascent literature on internet cultures related to personal finance by looking at a specific subreddit devoted to stock trading and investing. Second, we show how idiocultural elements, such as emojis and memes, can function both as contested and exclusionary material aimed at insiders and as flexible components of communications framed for broad mobilization through emotionally resonant notions of grievance and injustice.
Full-text available
“多模态”一词尚未有清晰的界定, 可以泛指理论、 视角、方法以及领域 (Jewitt, 2014: 127)。Bateman (2022: 43) 提出 “多模态” 可以视作一个领域的发展阶段, 因为几乎所有学科都涉及研究表意活动, 而且几乎所有研究对象都在逐渐多模态化。 因此, 将 “多模态” 视作研究的一个发展阶段更为包罗万象, 可以囊括几乎所有的表意活动。不同模态具备不同的潜力和不足, 在一个具体的交际活动中多种模态互相协作达成交际目的。这意味着多模态研究涉及的领域繁多, 研究目的不一, 方法各异, 缺乏一个统一的标准对多模态研究进行分类。 本文根据多模态研究的分析目的和侧重点将多模态研究的分析方法分为三类: (1) 模态意义分析, 即分析某个特定模态的意义建构方式, 建立模态的意义分析框架; (2) 多模态语类分析, 即分析特定语类中的多模态资源分配, 梳理这种语类的多模态特征; (3) 多模态批评话语分析, 即通过分析多模态的细节选择来揭示话语中的意识形态目的。文章着重梳理了近十年来多模态分析方法的新发展, 并在此基础上指出了国内现有的多模态研究的问题和未来的发展方向, 并重点讨论了现有的多模态分析所面临的主要争议和挑战。
This study examines the impact of graphicons (emoticons, emojis, and stickers) on the use of sentence-final particles (SFPs) in Chinese based on a 13-year longitudinal corpus of 941,020 comments posted on the popular Chinese social media platform Bilibili. Quantitative analysis shows that graphicon frequencies increase while SFP frequencies decrease over time, and that the correlation between these two trends is statistically significant. However, the more an SFP encodes a grammatical function or has a negative connotation, the less likely it is to be replaced by graphicons. Qualitative analysis suggests that the relationship between graphicons and SFPs is evolving from syntagmatic, where the two co-occur in the same sentence, to paradigmatic, where either can fulfill the function of expressing (positive) attitude or sentiment. This suggests that the functions of graphicons are shifting from compensation to competition with language, as an alternative to SFPs.
Full-text available
The present article stands at the interface of CMC research and grapholinguistics. After outlining which features are typical of the writing of pri­ vate text messages, the focus of the first part of the paper (Sections 2 and 3) lies on the use of emojis. Notably, emoji use is not-as is commonly done-analyzed under a pragmatic perspective, but grapholinguistically, at the graphetic and graphematic levels: emojis are conceptualized as visual shapes that may assume graphematic functions within a given writing system. In the second part (Sec­tion 4), it is underlined that all variants of written digital communication (such as the use of emojis, but also all other characters) are made possible only due to the Unicode Consortium's decisions; this, finally, is argued to have far­reaching consequences for the future of writing.
This chapter describes the methods and the overall findings of the Understanding Emoji Survey, which we administered online in early 2018 to determine how social media users interpret the pragmatic functions of popular emoji types in the discourse context of comments posted to public Facebook groups. The findings generally validate Herring and Dainas’s (2017) taxonomy of graphicon functions for emoji, although survey respondents (n=523) overwhelmingly preferred one function, tone modification, over the others. Moreover, preferred interpretations of pragmatic function varied according to emoji type. Based on these findings, we argue for the importance of analyzing emoji meaning from the perspective of pragmatics.
This study posits that graphicon use follows an evolutionary trajectory characterized by stages. Drawing on evidence that the uses and functions of emoticons have changed over time and that the introduction of emoji affected the popularity and usage of emoticons, we examine the uses of the newer types, emoji and stickers, and consider the relationship of stickers to emoji. Adapting the apparent-time method from the sociolinguistic study of language change, we compare sticker and emoji use by English-speaking Facebook Messenger users, exploring how they are used and under what conditions using semi-structured interviews and a large-scale survey. Stickers are argued to be more pragmatically marked for emotional intensity, positivity, and intimacy, characteristic of a more recent stage of evolution, while emoji use exhibits signs of conventionalization and pragmatic unmarking. The identification of patterns that characterize evolutionary stages has implications for future graphicon use.
An online survey, the Understanding Emoji Survey , was conducted to assess how English-speaking social media users interpret the pragmatic functions of emoji in examples adapted from public Facebook comments, based on a modified version of [15]’s taxonomy of functions. Of the responses received (N = 519; 351 females, 120 males, 48 “other”; 354 under 30, 165 over 30, age range 18--70+), tone modification was the preferred interpretation overall, followed by virtual action , although interpretations varied significantly by emoji type. Female and male interpretations were generally similar, while “other” gender respondents differed significantly in dispreferring tone and preferring multiple functions . Respondents over 30 often did not understand the functions or interpreted the emoji literally, while younger users interpreted them in more conventionalized ways. Older males were most likely, and younger females were least likely, to not understand emoji functions and to find emoji confusing or annoying, consistent with previously reported gender and age differences in attitudes toward, and frequency of, emoji use.
Emoji are miniature pictographs that have taken over text messages, emails, and Tweets worldwide. Although contemporary emoji represent a variety of races, genders, and sexual orientations, the original emoji set came under fire for its racial homogeneity: minus two “ethnic” characters, the people emoji featured in Unicode 7.0 were represented as White. This article investigates the set of circumstances that gave rise to this state of affairs, and explores the implications for users of color whose full participation in the emoji phenomenon is constrained by their exclusion. This project reveals that the lack of racial representation within the emoji set is the result of colorblind racism as evidenced through two related factors: aversion to, and avoidance of, the politics of technical systems and a refusal to recognize that the racial homogeneity of the original emoji set was problematic in the first place.
Emoji have emerged as a significant feature of modern discourse through the proliferation of social media. Noted for their ability to create meaning across linguistic boundaries, emoji are strongly associated with informal interaction where they serve to build and maintain social bonds. Following in the footsteps of Zappavigna (2011, 2015, 2018) I use a social semiotic approach to analyze how the celebrity, Cher, creates meaning across the strata of discourse and builds affinity with her followers through the use of the toilet emoji in relation to Donald Trump. The approach is successful at elucidating the complex layers of meaning which a single emoji can enact, providing further evidence of the value of the social semiotic approach in the context of global media. The affinity that is created by the toilet emoji can be understood through attitude-ideation couplings, where the coupling is made more 'bondable' to her ambient audience through the system of promotion. The possibility of an emoji enacting both attitude and ideation in a coupling is suggested and implications for how we understand certain semiotic choices discussed.