Conference PaperPDF Available

Affordances as Possible Actions: Elements for a Semiotic Approach



As far as semiotic theories are concerned, new digital texts and objects emphasize the need to refocus on the intimate connection between user and interface, considered as a space where deploying and manipulating contents through repeated contacts with a sensitive area (Fontanille & Zinna, 2004; Zinna2004). In this case, the concept of «affordance» becomes of central importance.Originally developed in the framework of Gestalt theory, the term «affordance» was subsequently reworked and made famous through the ecological approach to perception conceived by James Gibson(1966, 1979). However, the concept has successively been integrated into a more binary conception of cognition (Norman, 1988, 1998, 1999, 2007; Zhang & Patel, 2006), which seems to be responsible for the loss of much of its heuristic power. In this paper, I intend to go back to the genesis of the notion (Koffka,1935, Visetti & Rosenthal, 1999) and propose a semiotic and dynamic reinterpretation of this concept,where affordances can be seen as dispositions to act and patterns of expectation (De Souza, 2005; Eco,1997, 2007; Paolucci, 2007; Peirce, 1931-58; Quéré, 1999). Using Graphic User Interfaces as my object of study and developing an ecological approach to cognition, where the environment and the subject cannot be considered on the basis of binary distinction (Clark 1997, 2006, 2008; Noë 2004, 2009), I will show how semiotic activity takes place in a not infinitely brief present-time (Rosenthal, 2004, 2005),which is necessary for the development of a microgenetic activity of perception and cognition that will be intrinsically cultural. In this context, affordances may be explained as responses to a conceivable practical action made possible by habits that subjects consider on the basis of their inclusion in a system of practices and knowledge which foreshadow a specific and located horizon of action.
Proceedings of the 10th World Congress of the International Association for Semiotic Studies (IASS/AIS)
Universidade da Coruña (España / Spain), 2012. ISBN: 978-84-9749-522-6 Pp. 867-878
Affordances as Possible Actions:
Elements for a Semiotic Approach
siMone MorGaGni
Ecole des Hautes Etudes en Sciences Sociales, Paris (France)
& Università di Bologna (Italy)
As far as semiotic theories are concerned, new digital texts and objects emphasize the need to refocus
on the intimate connection between user and interface, considered as a space where deploying and
manipulating contents through repeated contacts with a sensitive area (Fontanille & Zinna, 2004; Zinna
2004). In this case, the concept of «affordance» becomes of central importance.
Originally developed in the framework of Gestalt theory, the term «affordance» was subsequently
reworked and made famous through the ecological approach to perception conceived by James Gibson
(1966, 1979). However, the concept has successively been integrated into a more binary conception of
cognition (Norman, 1988, 1998, 1999, 2007; Zhang & Patel, 2006), which seems to be responsible for the
loss of much of its heuristic power. In this paper, I intend to go back to the genesis of the notion (Koffka,
1935, Visetti & Rosenthal, 1999) and propose a semiotic and dynamic reinterpretation of this concept,
where affordances can be seen as dispositions to act and patterns of expectation (De Souza, 2005; Eco,
1997, 2007; Paolucci, 2007; Peirce, 1931-58; Quéré, 1999). Using Graphic User Interfaces as my object
of study and developing an ecological approach to cognition, where the environment and the subject
cannot be considered on the basis of binary distinction (Clark 1997, 2006, 2008; Noë 2004, 2009), I will
show how semiotic activity takes place in a not innitely brief present-time (Rosenthal, 2004, 2005),
which is necessary for the development of a microgenetic activity of perception and cognition that will be
intrinsically cultural. In this context, affordances may be explained as responses to a conceivable practical
action made possible by habits that subjects consider on the basis of their inclusion in a system of practices
and knowledge which foreshadow a specic and located horizon of action.
What happens when we begin using a computer? How does the interface communicate, how
does it show the actions that will be put at our disposal and what are the points on which
the user can act? If exploring the way an object communicates its functions can be consid-
ered a classical approach, the development of new objects and forms of digital narrative
involves more complex interaction practices. This increasing complexity, combined with the
need of a more intuitive interaction, led in recent years, to a new centrality of the concept
of «affordance», seen as a manifestation of a readiness for action. These last developments
make it necessary to refocus semiotic theories on the intimate connections established between
user and interface, intended as a space where it is possible to deploy and manipulate contents
through repeated contacts with a sensitive area (Fontanille & Zinna, 2004; Morgagni, 2008;
Zinna, 2004).
Together with the concept of «ambient optic array», the concept of «affordance» has been
developed by and is one of the theoretical pillars of the ecological approach to visual percep-
tion developed by James Gibson (Gibson, 1966 & 1979). It is however important to note that
in both cases the American psychologist keeps developing themes that were already central
in the tradition of Gestalt psychology (Ash, 1982 and 1998, Koffka 1935, Köhler, 1929 and
1969), trying to clearly mark the differences with the theoretical approach he proposes. Today,
since affordances have been integrated and have become very popular in the context of a binary
conception of cognition (Norman, 1988, 1998, 1999 and 2007), I think that a renewed reection
about this tradition and these concepts has become inevitable. I will try to quickly go through
Gibson’s position, drawing a parallel with the one exposed in the Gestaltist approaches. I will
then show how a reinterpretation of this concept in the context of an ecological approach to
cognition, crossing a Gestalt microgenetic perspective (Rosenthal, 2004 and 2005; Visetti &
Rosenthal, 1999 and 2003) with an interpretative semiotic approach (Eco, 2007; Paolucci, 2007;
Peirce, 1931-1958; Stjernfelt, 2007), can allow us to overcome the main theoretical impasses
of the Gibsonian concept of affordance, and in particular of the impoverished version of this
concept developed by some cognitive approaches.
in The Ecological Approach to Visual Perception, Gibson starts from the Gestalt assumption
that the meaning or value of things is constituted by an immediate perception, as would be,
for example, the recognition of their colour. To clarify this position, he mentions Kurt Koffka,
arguing that: «Each thing says what it is [...] a fruit says ‘Eat me’; Water says ‘Drink me’;
[1] The author would like to thank Nadège Lechevrel for the constructive critiques and discussions of
previous versions of this paper.
siMone MorGaGni
thunder says ‘Fear me’, and woman says ‘Love me’» (Koffka, 1935: 7). According to Koffka,
those values are essential and visible results of the experience itself, because this kind of sense
could not be explained as a pale context of memory images or as an unconscious set of response
tendencies. To resume, using his famous example of the mailbox, he says that it’s the box in
itself which «invite» us to introduce mail and, more generally, that the things in themselves
tell us what we can do with them, thanks to what he calls a demand character. Then, Koffka
calls us back to the concept of Aufforderungscharakter used by Kurt Lewin (Lewin, 1926) and
translated as «valence» (for the history of the English translation of this term, cf. Marrow, 1969:
56). This concept implies the existence of vectors that can attract or repel the subject from
the object of observation. Koffka questions himself about the role they can play and how they
can correctly be explained. As a member of the Gestalt school, he could obviously not reduce
them to an exclusive and physical monist existence, so he seems to accept the conclusion of the
necessity of a dual position. He therefore considers them as the result of a phenomenological
dynamics, following which the valence of an object shall be established on the basis of the
experience and of the necessity a subject can have of it, thus accepting some kind of variation
dependent on the needs of the subject himself. As a consequence, a mailbox will not have this
valence; it will not invite us to mail a letter if the subject does not nd himself in the necessary
condition of feeling the need to do it.
The concept of affordance developed by Gibson, however, is not sensitive to subject’s
changing needs. It is invariant, always available to the perception that may put or not the sub-
ject in the necessary condition for this recognition activity. For Gibson, the affordances of an
object are not constructed by the needs of a subject and by his perceptive action, but are just
offered by the object, they are a constituent part of this one. Gibson explains that this state-
ment has to be considered in the context of an ecological approach and not in the context of
the classical physical theory: everything here derives from the interactional environment to
which subject and object belong.
Gibson doesn’t accept Koffka’s dualist position, following which it is the phenomenal
mailbox and not the physical one which gives us the opportunity to send a letter. He prefers to
afrm the existence of a single mailbox, a real mailbox that can afford letter mailing to a letter
writing human being living in a community with a postal system. This would be understood
in the exact moment in which the mailbox is recognized as such, beyond its actual presence in
the visual eld. To resume, we can say that the Gestalt theory explains the «direct and immedi-
ate» experience of valences, assuming that ego as well is an object of experience, and that, in
the dynamic relation of the subject’s experience, some tensions arise between his phenomenal
ego and the object. Gibson rejects this explanation and proposes an easier way to show why
the values of things seem to be perceived «immediately and directly». According to him, this
perception is obvious because the affordances of things are specied in stimulus information
itself, and so in the ambient optic array. They seem to be perceived directly, he said, because
they are perceived directly.
If the Gestalt theory has seen the light as a reaction to the elementarist psychology that
considers the value of things as something that is perceived indirectly, for Gibson it fails to
explain how these components are to be considered part of the objects and how they are under-
stood by subjects. For this reason, he dismisses its phenomenological foundations, transform-
affordances as possible actions: eleMents for a seMiotic approach
ing it, with a clear background of behaviourism, in a kind of externalist realism that permits
to say that Gestalt psychologists, while criticizing the classical theories of perception, «never
managed to go beyond them» (Gibson, 1979: 140).
According to Gibson, the environment contains a medium and some substances that form
objects and subjects, and allow them to «afford» different interactions. Gibson uses the term
«affordance» to represent any possible interaction offered by an environment to the subjects
living in it: «The affordances of the environment are what it offers the animal, what it provides
or furnishes, either for good or ill». In this way, the air will afford vision and breathing, the
land will afford walking, a stone will afford gripping and so on, in a relationship exclusively
depending on the physical characteristics of objects connected with the physical characteristics
of subjects of perception. Water, for example, has a «holding on» affordance for a nite number
of animals (a lot of insects), that are able to maintain the relationship between their weight and
the contact surface below the breaking point of equilibrium; for all other animals water will
have the «not holding on» affordance. However, in special occasions, water may provide an
affordance such as «walk in» and so on. With the concept of environmental niche, Gibson sug-
gests that species live in an environment which is an interplay of affordances, complementary to
their bodily characteristics and which cannot be modied without a complete reorganization of
the entire environmental niche. Men are the clearest example: they change or create affordances
to make their environment more livable. So, affordances depend on the environmental niche
in which the animal lives: they are not to be conceived as subjective creations, but as physical
properties of this environment on the subject, regardless his expectations or his actual needs.
Water supports the «holding on» and «walk in» affordances, also in the absence of a subject
actually performing the action. Moreover, Gibson believes that affordances are meaningful to
the subject and, for this reason (and only in this limited sense), they should be understood as
subjective. Affordances would thus go beyond the classic subjective / objective[2] dichotomy
and the limits it poses.
In Gibson’s theory, the link between visual perception and the concept of affordance
is constitutive. To be able to guide and supervise the behaviour of subjects, affordances must
have a strong meaning and should be continuously perceived. It is important to explain this
perception. For Gibson, perceiving affordances is something direct, i.e. they are seized on the
perceptual eld thanks to the recognition of specic invariants. They are not the result of an
inference or a mental representation, but the simple recognition of information being part of
the ambient optic array. In this context, analysing the concept of information used by Gibson
becomes necessary because, as shown by the Carello & Turvey (Turvey & Carello, 1985) and
[2] «An affordance cuts across the dichotomy of subjective-objective and helps us to understand its
inadequacy» (Gibson, 1979: 129).
siMone MorGaGni
the Coulter & Sharrock critics (Sharrock & Coulter, 1998), the ecological approach to visual
perception seems to pass over the double mediation of sensations and cognitive processes
only thanks to the reduction of the gestaltist direct perception to a simple recognition of some
eco(logical) structures already existent in the environment. In a world where all animals live
governed by fundamental elements called substances, media and surfaces[3], no difference can
be made between the affordances of social objects and human behaviour and those of the natu-
ral environment. This lack of differentiation, however, seems not to modify the problem of the
support, the detection and the concepts that will be necessary to get information from the envi-
ronment. As noted by Louis Quéré (Quéré, 1999), echoing peircean terminology, even a purely
factual statement, an index, will not signify anything in the absence of a third term, an interpretant
that is not to be intended as an agent giving meaning, but as an habit, a mind’s law.
Two fundamental and complementary problems arise at this point of our dissertation if
we will not accept Gibson’s argument without discussing it. On the one hand, it is necessary
to recognize the role of practices and the continuing changes they cause in the socio-cultural
sphere complementary to the natural environment. On the other hand, the affordances of a
socio-cultural environment seem to require the mediation of interpretants to be accessed. So, to
speak of affordances as an immediate information perception in the framework of an ecological
approach ask to specify in greater detail both the content of this ecological environment and
the notion of objectivity we can apply to the affordances we nd in it.
For the moment, we will limit ourselves to summarize the three main points that have
emerged from the comparison between Gibson’s theory of affordances and his Gestaltist
1. Affordances have not to be seen as full properties of the object, but as actions that
the object may afford in the context of a larger environment, which includes the subjects of
perception. The value of an object depends on the actions made available in an ecological
framework including both the object and the subject of the interaction.
2. This rst point has an obvious Gestaltist source but, at the same time, Gibson deeply
criticizes this theory, saying that it is dualistic and does not provide explications about the
human competence of identifying and detecting these actions.
3. Gibson’s use of the terms «directly» and «immediately» is obviously derived from the
synonymous use typical of Gestalt theory. However, despite his rejection of the phenomenologi-
cal dualism and his systematic use of the term «directly» referring to the information transfer
known as affordance, he never explains how all this can be integrated in a theory considered
innovative because it rejected any notion of aprioristic meaning in a perception that, in the
human case, cannot not be limited to the natural environment, but it is also constituted by a
fundamental socio-cultural milieu.
[3] In Gibson’s theory the matter composing the environment can be divided in three different categories:
media, substances and surfaces. The media is the matter in which animals move; all the matter which in not
included in it represent the substances forming the objects and the surfaces are the boundaries between substances
and media.
affordances as possible actions: eleMents for a seMiotic approach
Technological evolution and the consequent renewed interest in practices put this notion again
in the spotlight. Donald Norman, for example proposes a renewed conception of the affordance
notion from a psychological and cognitive point of view, giving it a really different meaning
if compared to the original Gibson’s notion. In The Psychology of Everyday Things (Norman,
1988), he applies the concept of affordance to object design, proposing in particular a new
Human Computer Interaction approach that has experienced a great success and has been one
of the key terms of the contemporary discussion in both design and cognitive science.
For Norman, the concept of affordance refers to the detected and present properties of
objects, and in particular to some fundamental properties, which illustrate their possible uses.
Affordances can then be used to make interaction easier for users by transmitting, during an
exclusively perceptual contact, the instructions they need for the interaction. Here we assist to
a revival of the concept of affordance, not involving its insertion in an ecological environment
like the one developed by Gibson.
In Norman’s theory, a second distinction is made between perceived and real affordances,
distinction that would not have reason to exist in the original Gibson’s theory. During the design
process, perceived affordances are more important than real affordances, because they show
the user what is the action to do to correctly interact with the object (Norman, 1998: 123). In
this theory, everyday situations are determined by a combination of internal cultural knowledge
and informational external constraints. We see a clear distinction between cultural conventions,
which are learned and embedded in memory processes, and affordances, viewed as natural
potentials and limits of action, belonging to the sphere of external cognition (Norman, 1988:
55). In these two psychological spheres (psychology of everyday things and psychology of
cognitive processes), affordances are everything that can be collected as information without
the need of specic understanding and cognitive processes (Norman, 1998: 42).
We can see how a position like that, which considers affordances as belonging to one
of the two categories structuring mental representations, not only renounces to the idea of
direct and ecological perception, but it also establishes a strict dualism between cognition and
perceptual activity. If Gibson considered affordances as transcending the distinction between
objective and subjective, thus designing a unique system where the activity of perception was
the result of a collaborative work between its two poles, Norman[4] continues to use the concept
in connection with some psychological premises imposing a clear distinction between a world of
cultural and social conventions and a world of directly perceived affordances. In contrast with
this theoretical position, but remaining in a strictly cognitive domain, many critics show how
the interaction with objects seems to require at least an access to semantic resources (Creem
& Proftt, 2001), not to mention that a correct interaction with objects often requires a differ-
ent motor response from the one which men would normally activate (Klatzky, McCloskey,
Doherty, Pellegrino, & Smith, 1987).
[4] This seems true also if Norman, later (Norman, 1999: 123), will ambiguously speak of affordances as
«relationship that old between the object and the organism that is acting on an object».
siMone MorGaGni
Only recently, in his The Design of Future Things (Norman, 2007), Norman, considering
the work of Clarisse De Souza (De Souza, 2005), modies his position, accepting the idea that
affordances can be understood as the result of a communication process. In this new perspective,
Norman’s concept of affordance is completely shaken: from a direct relationship with the external
world, he develops a new view based on a communication process involving the intervention
of human higher cognitive levels. This change is due to a factor of utmost importance for him,
because the practical issues related to the design process are inextricably linked to the visibility
of affordances. Affordances cannot be used before the subject recognises them. The ability to
discover and learn to use them will be one of the explanations for conceiving easier interactions
with objects that have not been experienced yet. The need to build objects with perceptible and
visible affordances has always been important, but it is still more important today in an era
where objects are becoming digital and automated, he says, because they must communicate
and interact effectively with both ourselves and the world. Norman concludes saying that the
aim of affordances is to implicitly show where and how to go. They just show us a privileged
way to optimize our interaction, without even the need for the subject to notice it.
In Norman’s theory evolution, we clearly see the need to nd a way to go beyond a xed
and aprioristic time frame, an extremely problematic element in the process of understanding the
affordances surrounding us. If Gibson does not seem to be able to explain how the immediacy
of visual perception was able to directly make us understand affordances, Norman feels the
need to partially abandon his dichotomy between the spheres of human cognition to look for
a communication process that can implicitly show to men how to intervene in the world. The
positions of the two researchers, very different in the beginning, failed at the same point: the
need to reintroduce inherent temporality in the emergence of the percept and in the develop-
ment of human action. These two processes seem intimately interrelated and cannot easily be
explained out of a framework that is from its beginning inherently semiotic.
I will start from the deadlocks that have been highlighted to propose a denition of affordance
as possible action. A denition of this kind might allow us to overcome the impasses of the
original concept, thanks to a broader approach, which includes the effects of taking into account
time and intersubjective values.
I will immediately embrace the thesis of an ecological approach to cognition that does
not allow describing the environment and the subject according to a binary distinction. Several
cognitive approaches, the so-called «second generation approaches»[5] appear to have undertaken
[5] The term «second generation cognitive sciences» refers to the most recent cognitive approaches leaving
aside the original binary conception between body and mind in favour of a progressive embodiment of cognitive
processes. Instead of thinking cognition as a sum of some internal and abstract processes lling a material and
physical body, these new perspectives try to explain these signicant dynamics in the interaction between their
sensorimotor, conceptual and intersubjectives components.
affordances as possible actions: eleMents for a seMiotic approach
such a work by progressively extending cognitive processes outside the brain and the body (Clark,
1997 & 2008; Hollan, Hutchins & Kirsh, 2000; 1995 Hutchins, Noë, 2004 and 2009, O’Regan
& Noë, 2001; Varela, Thomson & Rosch, 1991). Despite these recent advances and the devel-
opment of some externalist and at least partially anti-representational theories, the most known
contemporary approaches still appear to present the classical problems about perception and the
emergence of meanings and values that have already been identied. Actually, they do not seem
to be able to give us a convincing restatement (see Lechevrel & Morgagni, submitted) of these
problems, something that could highlight the need for a complete embodied semiotics neces-
sary to show us the double dependence existing between our body, especially with regard to our
sensorimotor abilities, and the selection of relevant elements made by our cognitive system.
A more structured and convincing approach can be built up on the idea that a sensation,
and some sensorimotor qualities linked to it, can be reciprocally involved within the frame-
work of a perception process, implying at the same time some semiotic organization of forms,
as suggested by a recent reformulation of the Gestalt theory in a dynamic key (Rosenthal &
Visetti, 1999 and 2003) and of the semiotic theory of Charles Sanders Peirce (Paolucci, 2007;
Stjernfelt, 2007). The problems of this sensorimotor coupling, the role of the physical environ-
ment and the contribution of language and culture can be explained from the perspective of a
renewed phenomenological approach, in which all these elements are redistributed in a dynamic
loop. This dynamic loop would permit to take into account, from the beginning, all the complex
interrelations existing between these elements, showing that no distinction is possible between
the phenomenological and the physical level of human perception. In doing so, we can once
more make reference to the almost classical example of the mailbox, seeing a single box-sign
that can afford some actions by simply making them possible for someone, in a complex situ-
ation that does not elude the knowledge and the intersubjective practices previously acquired
by the subject of perception. Moreover, if we want to develop a comprehensive approach to the
domain of cognitive organization and form perception, we have to recognize, at the same time,
that any cognitive activity should be deployed in time within the framework of a non immediate
present, necessary to the microgenetic development of the activity of perception and cognition
(Rosenthal 2004 and 2005). In its dynamic process of categorization, this temporal space opens a
horizon of action, composed by provisions for act which forms the plot of an active anticipation
of what will build up meaning on the basis of the whole process in progress. In this context,
affordances will be just actions made possible by the cognitive activity in itself and could thus
be conceived as answers to practical actions, following the pragmatic maxim of CS Peirce:
Consider what effects that might conceivably have practical bearings you conceive the
objects of your conception to have. Then, your conception of those effects is the whole of
your conception of the object.
I will restate this in other words, since ofttimes one can thus eliminate some unsuspected
source of perplexity to the reader. This time it shall be in the indicative mood, as follows:
The entire intellectual purport of any symbol consists in the total of all general modes of
rational conduct which, conditionally upon all the possible different circumstances and
desires, would ensue upon the acceptance of the symbol.
(‘Issues of Pragmaticism’, CP 5.438, 1905)
siMone MorGaGni
The meaning will be established by answers to pragmatic actions made possible by the
habits people may develop on the basis of a diagram, which represents a network of temporary
relations based on past experiences. Not knowing the postal service, mailboxes do not say
anything about the affordances they can make available for us, while our social and previous
experience make us possible to receive such actions as conceivable by giving us access to this
particular kind of affordance. This will potentially be possible until the mailbox, the postal
service or another player acting in this social context announces that these affordances are no
longer available, leading us to produce a new abduction that will take us to a new diagram-
matic reasoning, with the possibility to develop a different conclusion and thereafter generate
new possible actions. This is the fundamental difference, as well remembered by Claudio
Vandi (Vandi, 2009), between James’ pragmatism, often mentioned in ethnomethodological
approaches, and Peirce’s pragmaticism. The rst states that any truth has to be built on a practi-
cal action, while the second afrms that the meaning is acquired thanks to the development of
a habit, a predisposition to act that can interrupt the continuous reorganization of the network
diagram. This interruption is in principle, and it is important to remember it, independent from
the practical success of the action that can be accomplished.
In the context we have outlined, perception, cognition and practices are developed in a parallel
and complementary way, always considering both the affordances of traditional and materials
objects and the affordances of digital objects such as computer interfaces. In this last case, the
access points to activate, the icons to identify and to select, the path to run trough to accom-
plish a goal seem to show us in a particularly striking way the process just described. Despite
the stability and the invariance of the support on which they are presented, interfaces allow
the establishment of extremely changing and varied interactions, where perception binds itself
with practical usages, as showed for example by some current research in semiotics, usability
or in the eld of the situated cognition (Bar, 2004; Fusaroli & Morgagni, 2009; Greeno, Moore
& Smith, 1993; Kirsh, 2009; Queen, 2006; Vandi, 2009). Computer icons, but more generally
each interactive graphic element of an interface, can be used in a multitude of different ways,
and can «afford» some actions by their colour, their form, their accompanying or embedded
text, or even (which is the most common case) a complex interplay between all these and many
more others elements. In any case, all these properties remain potential, and cannot be selected,
used and even perceived out of a set of immediately semiotic practices formed in parallel with
the object’s primary perception. The affordances of objects should therefore be intended as
potential actions triggered by a particular elaboration of a diagrammatic reasoning, that allow
their selection as expressive qualities, as Firstnesses of the object in question. What in our
immediate perception appears to be an organized opposition, a constituted meaning, is not
necessarily a prove of a direct transmission of information, or, to express it in Peircean terms,
just a Firstness. It always emerges from the bottom of a previous Thirdness. Each element
belonging to the world ts into a system of practices and knowledge that always foreshadows
a located and specic horizon of action. These practices, taken together with the progressive
affordances as possible actions: eleMents for a seMiotic approach
development of our perceptual activity, are what permits the emergence of a certain number
of meanings, which can certainly be seen as immediate, but that are not for this reason to be
considered directly determined by a physical world. By proposing an approach of this type,
I tried to show that the affordances of objects are nothing more than the specic manifesta-
tion of a process which is actually far more extensive, and that they have therefore to be
approached more broadly by a global semiotic theory. To proceed further in the elaboration
of this notion it will, on one hand, be necessary to have more empirical supports in cognitive
science’s empirical experiments trying to put together interpretation and action processes. On
the other hand it will also be necessary to better comprehend how objects can be conceived as
instruments capable to acquire and redistribute cognitive structures intended as interpretative
habits culturally shaped.
Ash, Mitchell (1982): The emergence of Gestalt Theory: experimental psychology in Germany, 1890-1920.
PhD thesis in History. Cambridge Massachusetts: Harvard University.
(1998): Gestalt Psychology in German Culture, 1890-1967. Cambridge University Press.
Bar, Moshe (2004): «Visual objects in context», Nature Reviews: Neuroscience, n.5: 617-629.
Clark, Andy (1997): Being There. Cambridge Massachusetts: MIT Press.
(2008): Supersizing the Mind. Embodiment, Action and Cognitive Extention. Oxford: Oxford
University Press.
Coulter, Jeff & Sharrock, Wes (1998): «On What we Can See», Theory and Psychology, 8 (2): 147-164.
Creem, Sarah & Proftt, Dennis (2001): «Grasping Objects by their Handles: A Necessary Interaction
between Cognition and Action», Journal of Experimental Psychology: Human Perception and
Performance, 27 (1): 218-228.
De Souza, Clarisse (2005): The Semiotic Engineering of Human Computer Interaction. Cambridge
Massachusetts: MIT Press.
Eco, Umberto (2007): «La soglia e l’innito». In Paolucci, Claudio (ed.) Studi di semiotica interpretativa
(pp. 145-176). Milano: Bompiani
Fontanille, Jacques & Zinna, Alessandro, eds. (2004): Les objets au quotidien, Nouveaux Actes Sémiotiques
– recueil. Limoges: PULIM
Fornel, Michel de & Quéré, Louis, eds. (1999): La logique des situations. Paris: Editions de l’EHESS.
Fusaroli, Riccardo & Morgagni, Simone (2009): «Enacting Computer Icons. The Dynamics of Interpretation
between Forms and Diagrams», Proceedings of The Arco’09 Conference: Interprétation et
problématiques du sens.
Gibson, J. James (1966): The senses Considered as Perceptual Systems. Boston: Hougton Mifin.
(1979): The ecological Approach to Visual Perception. Boston: Houghton Mifin.
Greeno, James, Moore, Joyce & Smith, David (1993): «Transfer of situated learning». In Detterman,
Douglas & Sternberg, Robert (eds.), Transfer on Trial: Intelligence, Cognition and Instruction (pp.
99-167). Norwood: Ablex.
siMone MorGaGni
Hollan, James, Hutchins Edwin & Kirsh, David (2000): «Distributed Cognition: Toward a New Foundation
for Human-Computer Interaction Research», ACM Transaction on Computer-Human Interaction,
7 (2): 174-196.
Hutchins, Edwin (1995): Cognition in the Wild. Cambridge Massachusetts: MIT Press.
Kirsh, David (2009): «Problem Solving and Situated Cognition». In Robbins, Philip & Aydede, Murat
(eds.), The Cambridge Handbook of Situated Cognition (pp. 264-306). Cambridge: Cambridge
University Press.
Klatzky, Roberta, McCloskey, Brian, Doherty, Sally, Pellegrino, James & Smith (1987): «Knowledge about
Hand Movements and Knowledge about Objects», Journal of Motor Behavior, 19: 187-213.
Koffka, Kurt (1935): Principles of Gestalt Psychology. New-York: Harcourt Brace.
Köhler, Wolfgang (1929): Gestalt psychology. New York: Liveright.
(1969): The task of Gestalt Psychology. Princeton: Princeton University Press.
Lechevrel, Nadège & Morgagni, Simone (submitted). Ecology and Cognition: on some new contributions
to the epistemological debate over biological metaphors.
Lewin, Kurt (1926): «Vorbemerkungen über die psychischen Kräfte un Energien und über die Struktur der
Seele», Psychologische Forschungen., 7, 294-329.
Marrow, Alfred (1969): The Practical Theorist: The Life and Work of Kurt Lewin. New York: Basic
Morgagni, Simone (2008): «Montages signiants: les icônes d’Apple iPhone entre texte et image», Signes,
Discours et Sociétés, n.2. Available on
Noë, Alva (2004): Action in Perception. Cambridge Massachusetts: MIT Press.
(2009): Out of Our Heads. New York: Hill and Wang.
Norman, Donald (1988): The Psychology of Everyday Things. New York: Basic Books.
(1998): The Invisible Computer. Cambridge Massachusetts: MIT Press.
(2004): Emotional Design. Why we Love (or hate) Everyday Things. New York: Basic Books.
(2007): The Design of Future Things. New York: Basic Books.
O’Regan, J. Kevin & Noë, Alva (2001): «A sensorimotor account of vision and visual consciousness»,
Behavioral and Brain Sciences, 24 (5), 939-1011.
Paolucci, Claudio, ed. (2007): Studi di semiotica interpretativa. Milano: Bompiani.
Peirce, Charles Sanders (1931-58): Collected Papers of Charles Sanders Peirce, voll. I-VI edited by
C. Hartshorne and P. Weiss, 1931-1935, voll. VII-VIII edited by AW Burks, 1958. Cambridge,
Massachusetts, Belknap Press.
Queen, Matt (2006): «Icon Analysis», Boxes and arrows. Available on: http://www.boxesandarrows.
Quéré, Louis (1999): «Action située et perception du sens». In Fornel, Michel de & Quéré, Louis, (eds.), La
logique des situations (pp. 301-338). Paris: Editions de l’EHESS.
Robbins, Philip & Aydede, Murat, eds. (2009): The Cambridge Handbook of Situated Cognition. Cambridge:
Cambridge University Press.
Rosenthal, Victor (2004): «Perception comme anticipation: vie perceptive et microgenèse». In Sock,
Rudolph & Vaxelaire, Béatrice (eds.). L’anticipation à l’horizon du présent . Liège: Mardaga.
(2005): « Formes, sens et développement: quelques aperçus de la microgenèse », Texto!, mars
affordances as possible actions: eleMents for a seMiotic approach
Rosenthal, Victor & Visetti, Yves-Marie (1999): «Sens et temps de la Gestalt»: Intellectica, 28, 147-229.
Rosenthal, Victor & Visetti, Yves-Marie (2003): Köhler. Paris: Les Belles Lettres.
Stjernfelt, Frederik (2007): Diagrammatology . New York: Springer-Verlag.
Turvey, Michael & Carello, Claudia (1985): «The Equation of Information and Meaning from the Perspective
of Situation Semantics and Gibson’s Ecological Realism», Linguistic and Philosophy, 8 (1): 81-90.
Vandi, Claudio (2009): «La strategia di Google: abiti e pratiche», E/C , 3-4: 163-172.
Varela, Francisco, Thompson, Evan & Rosch, Eleanor (1991): The Embodied Mind. Cambridge
Massachusetts: MIT Press.
Zinna, Alessandro (2004): Le interfacce degli oggetti di scrittura. Roma: Meltemi.
siMone MorGaGni
... Some authors claim that affordances are a part of ontological ecology and should hence be considered independently of the perceiver needs and intentions (Michaels, 2003) while others argue that they should be understood as properties of the animal-environment relationship (Chemero, 2003;Stoffregen, 2003). More recently, Morgagni (2009) discussed the possibility that affordances may emerge from phenomenological dynamics implying that affordances are responses to a conceivable practicable action for the subject and thus, depend on his practical knowledges. Other works have sought to broaden the understanding of the concept for heuristic purposes, notably by questioning the extent to which affordances invite actions for a subject. ...
Full-text available
Classically investigated in the context of judgment tasks about achievable actions, affordances have also been investigated in the context of the stimulus–response compatibility paradigm. Earlier work showed that perceptual categorization performance is significantly faster and more accurate when the orientation of the graspable part of a presented object, and the orientation of the participant’s response are compatible, suggesting that the main function of affordances is restricted to action preparation. Here, we investigate the potential role of affordances in the categorization of ambiguous stimuli through a stimulus–response compatibility paradigm. In other words, we investigate if in ambiguous situations, such as ones in which a stimulus may give rise to two percepts, affordances would stabilize perception on one of these two and, therefore, helps in the subsequent categorizations. Two experiments were run, based on the forced-choice stimulus–response compatibility (SRC) paradigm, with a progressive series of ambiguous (bistable) lateral-graspable objects. In Experiment 1, subjects responded by pressing horizontally opposite keyboard keys, while in Experiment 2, the keyboard keys were vertically separated. Experiment 1 found that subjects perceived the initial object in a bistable series for longer, and exhibited greater response stability in compatible than incompatible situations. In Experiment 2, none of these modulations were significant. Overall, our results show that affordances operationalized through a SRC paradigm modulated how subjects categorized ambiguous stimuli. We argue that affordances may play a substantial role in ambiguous contexts by reducing the uncertainty of such situations.
Conference Paper
Full-text available
L’obiettivo del presente lavoro è quello di offrire un supporto all’ipotesi che gli individui con Sindrome di Rett (RS) possano beneficiare di interventi riabilitativi che combinano il potenziamento cognitivo con i nuovi dispositivi di tracciamento oculare, come il Tobii I-Series. Il tobii è un dispositivo portatile, simile a un laptop, dotato di raggi infrarossi, posizionati alla base dello schermo che consentono di leggere il movimento oculare e di decifrare le scelte dei soggetti in risposta agli stimoli cui di volta in volta vengono sottoposti. Nonostante il profilo neuropsicologico dei pazienti con RS si collochi al terzo o quarto stadio dell’intelligenza sensomotoria, vale a dire corrispondente rispettivamente ad una età mentale di 4-8 e 8-12 mesi (Olsson & Rett, 1990) gli individui con RS mostrano comportamenti non-verbali spontanei e su richiesta, come il contatto e l’orietamento oculare verso un stimolo target, simili ai soggetti con sviluppo tipico. Tali competenze comportamentali possono giustificare l’utilizzo della tecnologia eye-tracker ed essere considerati come una valida risorsa per implementare interventi di potenziamento cognitivo. In quest’ottica, studi precedenti (Fabio et al., 2016; 2014) che hanno usato il dispositivo Tobii, indicano un effetto positivo del potenziamento cognitivo sui parametri cerebrali (EEG) e comportamentali (capacità di riconoscimento e discriminazione di oggetti) nelle bambine con SR. Inoltre, sebbene il funzionamento cognitivo dei soggetti con RS sia limitato a semplici risposte comportamentali a seguito di un stimolo sensoriale, diversi studi su soggetto singolo hanno dimostrato che le ragazze con RS, se adeguatamente allenate, possono migliorare le loro capacità attentive e di comunicazione. Lo scopo principale del presente studio è valutare l’efficacia del potenziamento cognitivo applicato attraverso l’uso del Tobii, nel quale è stato installato un sofware di CAA appositamente creato per le ragazze con SR. Al fine di raggiungere questo obiettivo, abbiamo esaminato l’abilità di comunicazione, gli indici attentivi, e il livello di gravità della sindrome attraverso strumenti strandardizzati quali: la Scala RARS, Vineland e il Questionario di Valutazione della Qualità della Vita nei soggetti con medio e grave ritardo mentale. 29 soggetti con RS, dai 2 ai 33 anni, hanno partecipato allo studio. E’ stato impiegato un disegno sperimentale ABAA: valutazione pre-test, training di potenziamento cognitivo con l’ausilio del Tobii, valutazione posttest e follow-up. Sono state misurate le capacità di riconoscimento di immagini e parole e la capacità di discriminazione e riconoscimento di oggetti, colori e forme, attraverso i parametri dell’eye-tracker. I risultati hanno indicato che la performance nei compiti di discriminazione e di riconoscimento di immagini, parole, oggetti, colori e forme migliorava significativamente nel corso del training di potenziamento cognitivo in tutti i partecipanti. Questo studio supporta il ruolo del potenziamento cognitivo in associazione alla tecnologia eye-tracker, nel favorire lo sviluppo delle abilità cognitive e comunicative anche nei soggetti con sindrome degenerativa, quali la SR.
Full-text available
The transposition of icons and interfaces in a plurality of devices and software questions the processes of meaning construction involved in their use and interpretation. The analysis of four apparently simple icons used to send an email on an iPhone leads us to investigate C. S. Peirce's conception of iconism and the Gestaltic notion of form. Icons are operationally defined as what enables us to do/understand something more. Icons are thus grounded in a bundle of practices according to which the “more” can be established. An adequate description of their meaning relies on a diagrammatical approach: through established habits users infer local configuration of meaningful relations (Gestaltic forms) to guide their action and relations that can be further manipulated and modified.
A theory of HCI that uses concepts from semiotics and computer science to focus on the communication between designers and users during interaction. In The Semiotic Engineering of Human-Computer Interaction, Clarisse Sieckenius de Souza proposes an account of HCI that draws on concepts from semiotics and computer science to investigate the relationship between user and designer. Semiotics is the study of signs, and the essence of semiotic engineering is the communication between designers and users at interaction time; designers must somehow be present in the interface to tell users how to use the signs that make up a system or program. This approach, which builds on—but goes further than—the currently dominant user-centered approach, allows designers to communicate their overall vision and therefore helps users understand designs—rather than simply which icon to click. According to de Souza's account, both designers and users are interlocutors in an overall communication process that takes place through an interface of words, graphics, and behavior. Designers must tell users what they mean by the artifact they have created, and users must understand and respond to what they are being told. By coupling semiotic theory and engineering, de Souza's approach to HCI design encompasses the principles, the materials, the processes, and the possibilities for producing meaningful interactive computer system discourse and achieves a broader perspective than cognitive, ethnographic, or ergonomic approaches. De Souza begins with a theoretical overview and detailed exposition of the semiotic engineering account of HCI. She then shows how this approach can be applied specifically to HCI evaluation and design of online help systems, customization and end-user programming, and multiuser applications. Finally, she reflects on the potential and opportunities for research in semiotic engineering.
Thesis (Ph. D.)--Harvard University, 1982. Includes bibliographical references (v. 2, leaves 536-598). Includes indexes.
  • Moshe Bar
Bar, Moshe (2004): «Visual objects in context», Nature Reviews: Neuroscience, n.5: 617-629.