Chapter

Contemporary Landscapes of Visual and Digital Communication: The Interplay of Social, Semiotic, and Technological Change

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Our aim is to describe what we regard as nothing less than a paradigm shift – in societies where this change is taking place. It is a move from a paradigm in which the mode of writing (‘language’, in traditional parlance) was assumed to provide the ontological and epistemological basis for describing, understanding, and accounting for ‘the world’, to a situation where the visual mode is – more and more – assuming that role. The mode of image underpins the new paradigm, and the semiotic resources of that mode provide the tools, the descriptions, the analyses, and the accounts as means for understanding the world. In this chapter, we can do no more than ‘fly that kite’. Proceeding, in the main, via examples, we point to some telling instances, suggesting what kinds of tools might serve for documenting and for analysing that world. In this, we aim to provide some of the outlines of the paradigm and consequences of that shift.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Por otro lado, la ubicuidad de las pantallas ha dado paso a nuevas formas de socializar y de mostrarse frente a otros; prácticas instaladas en las vidas de muchos a través de distintas tecnologías (Gualberto & Kress, 2020), debiendo adaptarse a plataformas digitales y a clases virtuales, que incluso trasladaron las desigualdades de la escuela presencial a la escuela virtual (Núñez, 2020). La mirada semiótica y social a las formas de comunicación señala que los soportes para comunicar ofrecen maneras distintas de crear signi#cados, no traducibles entre sí, que dan forma y, a la vez, están in%uenciados por los aspectos sociales, culturales, económicos y políticos (Bezemer & Kress, 2016). ...
... La mirada semiótica y social a las formas de comunicación señala que los soportes para comunicar ofrecen maneras distintas de crear signi#cados, no traducibles entre sí, que dan forma y, a la vez, están in%uenciados por los aspectos sociales, culturales, económicos y políticos (Bezemer & Kress, 2016). Dado que la pandemia obligó a mantener los vínculos de manera digital, se torna urgente comprender y visibilizar los cambios dentro de los espacios de participación virtual en cuanto a las posibilidades que ofrecen (Gualberto & Kress, 2020;Kress & van Leeuwen, 2006). Por mucho tiempo la manera de entender la participación se ha restringido a la comunicación hablada o escrita, como si fuera la única y exclusiva manera de expresar nuestras ideas y posiciones en el mundo, de signi#car para otros. ...
... Lo social empuja y está en la base de la creación de los signi#cados (Gualberto & Kress, 2020); por esta razón se reconoce el importante lugar que ocupan las tecnologías de la información y comunicación en la vida cotidiana contemporánea, especialmente entre la juventud (Cantor-Silva et al., 2018;Del Prete & Redón, 2020). García-Canclini y Urteaga (2017) señalan que los procesos sociales relacionados con las transformaciones de las estructuras comunicacionales y la construcción de redes evidencian relaciones de poder en las prácticas de comunicación, las cuales sobrepasan las prácticas de hablar y recibir. ...
Full-text available
Article
(analítico)Las y los estudiantes carecen de espacios para valorar sus experiencias de inclusión/exclusión escolar. La crisis sanitaria obligó a vincularnos virtualmente, sin conocer las posibilidades para el intercambio de significados digitales, representación de experiencias y movilización de relaciones interpersonales. Desde una perspectiva pedagógica, semiótica y social, este estudio analiza una sesión de un taller virtual con nueve estudiantes secundarios de tres escuelas chilenas, preguntándose sobre cómo se materializó la participación de las y los jóvenes. Con herramientas de la multimodalidad se analiza un corpus audiovisual de una cartografía virtual de espacios educativos. Se visibilizan los recursos disponibles para la participación virtual y las maneras en que fueron usados para compartir significados individuales y generar solidaridad co-construyendo relatos de inclusión/ exclusión escolar. Palabras clave: Juventud; interacción; participación estudiantil.
... Destacaremos tres ideas del enfoque multimodal desde la semiótica social. La primera es que lo social impulsa y está en la base de lo semiótico (Gualberto & Kress, 2019). Esto se ha hecho evidente en el contexto de pandemia, ya que los actores sociales han visto en la educación remota la posibilidad de sostener la escuela (Navarro et al., 2021). ...
Full-text available
Article
Profesoras/es y aprendices recrean significados constantemente, condicionados por los medios digitales usados en la enseñanza remota durante la pandemia. Los profesores diseñan significados como juegos de aprendizaje e involucran a los estudiantes en la simulación de prácticas culturales, en las cuales las secuencias didácticas de escritura se materializan en cadenas que desplazan los significados entre diversos modos semióticos, dando origen a momentos transmodales. Este estudio integra tres enfoques teóricos: socio-semiótico, antropológico-didáctico y didáctico-disciplinar. Nuestro objetivo es explorar la semiosis en los momentos transmodales ocurridos en la interacción virtual durante una secuencia didáctica. Desde lo cualitativo, este estudio de caso profundiza en la simulación de prácticas culturales de una estudiante de 4o año de educación básica mediante el análisis integrado de cadenas semióticas y SHTIS (Sistema-Híbrido-Texto-Imagen-Sonido). Los hallazgos ofrecen una descripción de los juegos de aprendizaje vinculados con dos prácticas culturales cuyos núcleos de significado se desplazan entre seis eslabones. El rastreo de núcleos refleja la tensión entre la agencia de ambos creadores, que se materializa en ensamblajes semióticos de la enseñanza y aprendizaje mediados a través de una plataforma digital, en la que destaca el reciclaje de videos y documentos. No obstante, la retroalimentación revela que los significados visuales siguen invisibilizados.
Full-text available
Article
Este artigo apresenta uma proposta de matriz de práticas de letramento para a produção de infográfico por meio da ferramenta on-line e gratuita de editoração gráfica Canva. Com base em Paiva (2009, 2013), analisou-se o infográfico como um gênero de texto visual informativo muito utilizado em visualização de informações em mídia impressa e digital, por isso considera-se ser proveitoso seu ensino e aprendizagem com estudantes da educação básica. Para analisar o seu layout, partiu-se da abordagem da multimodalidade de Kress e van Leeuwen (2006) e dos estudos de layout na perspectiva semiótica social multimodal de Kress (2010) e Paiva (2021). Depois, foram apresentadas e discutidas três pesquisas relevantes para a produção do infográfico na escola, quais sejam Ribeiro (2016), Rodrigues (2018) e Gomes (2019). Em seguida, com base em Dias e Novais (2009), desenvolveu-se a proposta de matriz, para organizar o ensino e a aprendizagem da produção de infográficos no Canva. Ainda que não seja uma metodologia de ensino, acredita-se que a matriz possa auxiliar o professor e a professora no seu trabalho de ensino e de aprendizagem de produção de infográficos, porque os estudantes podem desenvolver projetos de visualização de informação utilizando o Canva, inclusive em seus próprios smartphones.
Full-text available
Article
In Chile, discourses on educational quality and inclusion are defined in a competitive framework, highlighting best performing schools in the national standardized tests, which tend to be private schools belonging to the country’s capital. This ranking defines “excellence” and reflects inequality and segregation. Nevertheless, in 2018, the first place was taken by a regional public school. From Social Semiotics and a multimodal perspective, we seek to understand the meanings created in mass media and internet by different social actors about this high-performing public school. This study contrasts the meaning making by the Education Quality Agency, National Television (TVN) and the school principal. A multimodal discourse analysis is applied to an audio-visual corpus of three videos. Three emerging categories transverse the voices’ discourses, defining educational quality in terms of: community, art and languages, and keys to success. These meanings are materialized in dynamic multimodal ensembles which associate the meanings of quality to management, effort, and humanization as rebellion.
Full-text available
Article
In this article, we reflect on some impacts of the pandemic on interactions and communication drawing from Gunther Kress' multimodal and sociosemiotic thinking. To do so, we relate examples with fundamental concepts of Social Semiotics, such as identity, power, motivated sign and interest. In addition, we propose a small tribute to this author who greatly influenced us. We trace a brief history of his academic career in which we highlight some of his main publications and discuss concepts and ideas developed by him throughout his career.Keywords: Social Semiotics. Gunther Kress. Multimodality. Pandemic
Full-text available
Article
In this paper we provide a multimodal account of historical changes in secondary school textbooks in England and their social significance. Adopting a social semiotic approach to text and text making we review learning resources across core subjects of the English national curriculum, English, Science and Mathematics. Comparing textbooks from the 1930s, 1980s and 2000s, we show that a) all modes operating in textbooks -typography, image, writing and layout- contribute to meaning and potential for learning b) that the use of these modes has changed between 1930 and now, in ways significant for social relations between and across makers and users of textbooks. Designers and readers / learners now take responsibility for coherence, which was previously the exclusive domain of authors. Where previously reading paths were fixed by makers it may now be left to learners to establish these according to their interests. For users of textbooks the changes in design demand new forms of ‘literacy’; a fluency not only in ‘reading’ writing, image, typography and layout jointly, but in the overall design of learning environments. We place these changes against the backdrop of wider social changes and features of the contemporary media landscape, recognizing a shift from stability, canonicity and vertical power structures to ‘horizontal’, more open, participatory relations in the production of knowledge.
Full-text available
Article
Frequently writing is now no longer the central mode of representation in learning materials—textbooks, Web-based resources, teacher-produced materials. Still (as well as moving) images are increasingly prominent as carriers of meaning. Uses and forms of writing have undergone profound changes over the last decades, which calls for a social, pedagogical, and semiotic explanation. Two trends mark that history. The digital media, rather than the (text) book, are more and more the site of appearance and distribution of learning resources, and writing is being displaced by image as the central mode for representation. This poses sharp questions about present and future roles and forms of writing. For text, design and principles of composition move into the foreground. Here we sketch a social semiotic account that aims to elucidate such principles and permits consideration of their epistemological as well as social/pedagogic significance. Linking representation with social factors, we put forward terms to explore two issues: the principles underlying the design of multimodal ensembles and the potential epistemological and pedagogic effects of multimodal designs. Our investigation is set within a research project with a corpus of learning resources for secondary school in Science, Mathematics, and English from the 1930s, the 1980s, and from the first decade of the 21st century, as well as digitally represented and online learning resources from the year 2000 onward.
Full-text available
Article
In this article, the authors provide an empirically based, social semiotic account of changes in textbook design between 1930 and the present day. They look at the multimodal design of textbooks rather than at image or any other mode in isolation. Their review of 23 textbooks for secondary education in English shows that profound changes have taken place not just in the use of image but equally in writing, typography and layout. Design is no longer exclusively organized by the principles of the organization of writing, but also, and increasingly so, by graphic, visual principles. They explore what these semiotic changes mean for the social organization of design and knowledge production, asking: What is 'English', a subject that supposedly concerns itself with the modes of writing and speech? What has changed in the environment that is set up by the textbook makers for teachers and students to engage in?
Chapter
This chapter details how affect theory can guide analysis of social media's visual content.
Article
The 21st century is awash with ever more mixed and remixed images, writing, layout, sound, gesture, speech, and 3D objects. Multimodality looks beyond language and examines these multiple modes of communication and meaning making. Multimodality: A Social Semiotic Approach to Contemporary Communication represents a long-awaited and much anticipated addition to the study of multimodality from the scholar who pioneered and continues to play a decisive role in shaping the field. Written in an accessible manner and illustrated with a wealth of photos and illustrations to clearly demonstrate the points made, Multimodality: A Social Semiotic Approach to Contemporary Communication deliberately sets out to locate communication in the everyday, covering topics and issues not usually discussed in books of this kind, from traffic signs to mobile phones. In this book, Gunther Kress presents a contemporary, distinctive and widely applicable approach to communication. He provides the framework necessary for understanding the attempt to bring all modes of meaning-making together under one unified theoretical roof. This exploration of an increasingly vital area of language and communication studies will be of interest to advanced undergraduate and postgraduate students in the fields of English language and applied linguistics, media and communication studies and education.
Book
This state-of-the-art account of research and theorizing brings together multimodality, learning and communication through detailed analyses of signmakers and their meaning-making in museums, hospitals, schools and the home environment. By analyzing video recordings, photographs, screenshots and print materials, Jeff Bezemer and Gunther Kress go well beyond the comfortable domains of traditional sites of (social) semiotic and multimodal research. They steer away from spurious invention and naming of ever more new and exciting domains, focusing instead on fundamentals in assembling a set of tools for current tasks: Namely, describing and analyzing learning and communication in the contemporary world as one integrated field. The theory outlined in the book is grounded in the findings of the authors wide-ranging empirical investigations. Each chapter evaluates the work that is being done and has been done, challenging accepted wisdom and standing much of it on its head. With extensive illustrations and many examples presented to show the reach and applicability of the theory, this book is essential reading for all those working in multimodality, semiotics, applied linguistics and related areas.
Type in Motion: Innovations in Digital Graphics
  • Jeff Bellantoni
  • Matt Woolman
Bellantoni, Jeff and Woolman, Matt (2000) Type in Motion: Innovations in Digital Graphics. London: Thames & Hudson.
  • George Bethell
  • David Coppock
Bethell, George and Coppock, David (1988) Integrated Science 1. Oxford: Oxford University Press.
Design: The rhetorical work of shaping the semiotic world
  • Gunther Kress
Kress, Gunther (2014) 'Design: The rhetorical work of shaping the semiotic world', in A. Archer and D. Newfield (eds), Multimodal Approaches to Research BK-SAGE-PAUWELS_MANNAY-190229-Chp36.indd 589 9/18/19 11:18 AM and Pedagogy: Recognition, Resources, and Access. London: Routledge, pp. 131-52.
Available from: www.theguardian.com/technology/2018/feb/ 12/is-facebook-for-old-people-over-55s-flockin-as-the-young-leave
  • Mark Sweney
Sweney, Mark (2018) 'Is Facebook for old people? Over-55s flock in as the young leave', Guardian, US edition, February 12. Available from: www.theguardian.com/technology/2018/feb/ 12/is-facebook-for-old-people-over-55s-flockin-as-the-young-leave [Accessed 27 January 2019].