Conference PaperPDF Available

Evaluating safety signage systems in train stations: a qualitative and quantitative methodology



In this paper we describe a methodology developed to test a new safety signage system before its implementation in train stations. Every year, pedestrians are involved in numerous railway-related accidents in train stations. In order to prevent such accidents, a cognition centred research project was undertaken to rethink, redesign and evaluate safety signage systems (Morgagni, Lemesle & Maurice, 2012). Three main steps have been defined, each one based on both qualitative and quantitative approaches, collecting subjective as well as objective data, and integrating methodologies from both cognitive and ergonomic domains. The results obtained helped to decide the future of safety signage systems in French train stations and could be adapted and reused elsewhere. Indeed, the proposed methodology provides complete cues for future implementation and deployment of safety signage systems and systematises a research methodology for future similar needs.
Evaluating safety signage systems in train stations: a qualitative and quantitative methodology.
1SNCF, La Plaine Saint-Denis, France
2Ergocentre, Orléans, France
Corresponding Author: Elise Grison (
In this paper we describe a methodology developed to test a new safety signage system before its
implementation in train stations. Every year, pedestrians are involved in numerous railway-related accidents in
train stations. In order to prevent such accidents, a cognition centred research project was undertaken to
rethink, redesign and evaluate safety signage systems (Morgagni, Lemesle & Maurice, 2012). Three main steps
have been defined, each one based on both qualitative and quantitative approaches, collecting subjective as
well as objective data, and integrating methodologies from both cognitive and ergonomic domains. The results
obtained helped to decide the future of safety signage systems in French train stations and could be adapted
and reused elsewhere. Indeed, the proposed methodology provides complete cues for future implementation
and deployment of safety signage systems and systematises a research methodology for future similar needs.
Keywords: experimental psychology; human factors; signs; virtual reality.
1. Introduction
The prevention of pedestrian accidents in train stations is an important subject of interest given that, every year,
numerous railway-related accidents involving pedestrians are recorded around the world. As an example, in
France, we observe around 140 pedestrian-accidents every year (EPSF, 2018). Several of these accidents may be
due to pedestrians’ poor consideration or comprehension of risks when entering a railway zone. Amongst the
various possible ways to act against these accidents, such as prevention and sensibilisation campaigns that can
be done at regional or national levels, another action might concern the development of a more effective rail
safety signage system that will be deployed in train stations.
Currently in France, safety signage deployed in train stations convey the important messages concerning risks
and recommended behaviour. However, the findings found in the human factors and cognitive literature of
these past thirty years suggest numerous improvements to safety signage, such that they be better understood,
memorised and impactful on human behaviour (Sanders & McCormick, 1987; Stanton & Edworthy, 1999).
Moreover, the literature has also consciously defined how to deliver the information, in terms of the type and
structure of the message and the format of sensorial modality (auditory, visual). The structure of the message
and the preferred format is also dependent on the temporal and spatial proximity of the danger (Tingvall,
Eckstein, & Hammer 2008; Ho et Spence, 2009). For example, it has been shown that auditory formats should
be preferred when the danger is imminent. When not so spatially close to the danger (on the platform or in the
waiting area of the train station), visual formats and verbal messages can be used. Generally, verbal messages
displayed on signs should be short, structured with a hierarchy based on the relative importance of information
(Wogalter et al., 2012). They can also benefit from adequate pictograms for an easier and quicker
comprehension. Several other recommendations can be found in the literature (Laughery et Wogalter, 2014;
Williams et Noyes, 2007; Young et Wogalter, 1990), and may be a good starting point to propose a new safety
signage system that should be well understood and efficient for pedestrians.
To our knowledge, however, no study based on human factors and cognition has already documented the
process of creating a new safety signage system for the context of railways and, especially, for train stations.
Indeed, it can be easily hypothesised that, once applied in a specific context, some of recommendations found
in the literature will not be simple to follow or to be really adapted. Moreover, we did not find in the literature
any precise methodology developed to test the efficiency of a global safety signage system on human detection,
understanding and memorisation.
Thus, the main aim of this paper is to report on the process of the development of a new safety signage system
based on recommendations found in the literature. We will specifically focus our interest on the methodologies
developed from classical human factors, ergonomics and experimental psychology, to test the efficiency of the
new signage system developed.
2. Methodology
Based on the literature, we developed several concepts, including 3 different graphical styles and 2 spatial
arrangements (one that displays a sign and message on the ground, and another on mast). In all concepts, the
first version of the verbal messages has been worked to be as short and simple as possible, and pictograms have
been developed to illustrate each important information.
2.1 Step 1: Test and validation of general principles
After the first conception phase based on state-of-the-art technical and scientific knowledge, an online
questionnaire was proposed to objectively evaluate the understanding of the various elements that could be
used to compose the new signage system, such as pictograms (and their combination, Bordon, 2004), text
messages, and spatial arrangements.
106 participants responded to the online questionnaire (mean age = 40 years; 66 women, 40 men). They were
first presented the pictogram one-by-one, without any context, and asked to describe the pictogram’s
signification. Next, the same question was asked showing the pictograms in context. These two steps were used
to evaluate combinations between pictograms and their describing texts. Figure 1 below presents examples, for
two concepts, of the task of the combination of sign and text in train station contexts.
Figure 1: Example signage in the train station context used for the questionnaire.
Two additional workshops, with 18 participants (mean age = 35; 9 women, 9 men) were conducted to
complete the questionnaire with subjective and qualitative data to suggest improvements for elements that
appeared to be hard to understand for the participants.
The tasks developed for the questionnaire were used as a basis for the discussions. Participants were asked to
reflect on and propose improvements, individually as well as in groups of 3-4. They were also asked to place
every element of the new signage system in a small mock-up train station to have a naïve feedback on spatial
arrangements. All interactions between participants were recorded and analysed to understand the reasons for
their placement choices. The results of these workshops helped to select the better concept for the next steps,
and to improve message phrasing and pictogram representations. An example of how participants remarks and
recommendations have been considered is in presented in Figure 2.
Figure 2: Example of steps of conception before and after workshop with participants, and of improvements
made on the pictogram. In a) the initial version proposed to explain that pedestrians should keep their
distance from the edge of the platform as they might be hit by the train. In b) some modifications proposed by
participants. In c) two new propositions.
2.2 Step 2: Selection
To assess their cognitive understanding and efficiency compared to the existing safety signage system, two
studies were conducted on two new concepts which were developed based on data collected in step 1. The
purpose of this step was to acquire objective and quantitative data but combined to ecological data to ensure a
good selection of the better concept.
A first study, based on a virtual reality environment, was conducted in a real-like situation but providing a safe
and controlled environment to study human behaviour. 10 participants took part in the study (mean age = 38.9;
4 women, 6 men).
Figure 3: The virtual train station used for the test.
A virtual environment of a small train station was developed (Figure 3). Three versions of the environment were
developed, one for each safety signage to be tested (see Figure 4 for examples of the actual signage system and
a new system which was tested). Participants were immersed in the virtual environment through a virtual reality
headset equipped with an eye-tracking system (HTC Vive pro eye). To encourage participants to walk naturally
into the train station and observe their environment without revealing to them the real purpose of the study (to
avoid inducing any bias), a scenario in which participants had to review new services in train stations was
proposed. Participants were asked to find the new services and make a judgment on them. Participants were
thus exposed to the safety signage systems and the observation of their natural behaviour was guaranteed.
Figure 4: The virtual train station in the actual signage system condition on the left, and with a new system on
the right.
Different behavioural and physiological measures were recorded during the task (gaze behaviour and kinematic
data). Additionally, the participants subjective points of view were addressed through interviews to collect their
experience in virtual reality and recommendations to improve the new signage systems.
The second study (an online study) was implemented to evaluate using a larger number of participants and
through more classical cognitive tasks, the efficiency of the new safety signages systems compared to the
existing one on attention, visual search, comprehension, and memorisation. This was also the opportunity to
adjust some minor elements, such as colour, of the new signage system. 922 participants took part in the study
(mean age = 30.9; 355 men and 567 women).
The first task, based on a “change detection paradigm” (Luck & Vogel, 1997), was conceived to evaluate if and
how the signages are detected in a train station environment. Pairs of images featuring a real train station
environment were presented the second one could include some modifications (presence, absence,
displacement) concerning the signage systems to be evaluated. Participants had to indicate if the two images
were the same or not. The second task aimed at defining how easy it is to find safety signage and information
depending on their visual salience. We used a classical visual search task (Treisman & Gelade, 1980), in which
participants had to search for a sign in a visual scene (train station or fictive scene). Finally, a visual story of a
person navigating in a train station and seeing a variety of the safety signage signs in the environment was
presented to participants. In a following step, participants had to choose amongst various signages, those that
were previously presented.
Results obtained in this phase helped to select the right colour for the new signage system that will allow it to
be detected more clearly in the environment. It also helped to validate the better spatial arrangement; for
example, it appears that deploying all signs and messages on the ground is not relevant as it will be harder to
detect the information from far away.
2.3 Step 3: Validation
This final step was conceived to appreciate in ecological context, the advantages of the winning system in step
2, on human safety behaviour as well as on passenger understanding. 43 participants took part in the study
(mean age = 41.5; 22 men, 21 women).
We draw on the methodology developed in virtual reality to develop a realistic scenario for participants.
Participants were asked to navigate through a train station equipped with the current or new safety signage
system, as if they were waiting for a friend arriving on the next train. During their navigation, natural gaze
behaviour was recorded through Tobii Pro Glasses 2 (See Figure 5). In a second part of the study, participants
were asked to find specific signages in their environment (search time and gaze behaviour were recorded).
Finally, a questionnaire about the understanding of safety signage was proposed and their global appreciation
was recorded.
Figure 5: A participant reading a sign with the eye-tracking glasses.
This final study allows to compare the understanding of the old and new safety signage systems in real context
(see Figure 6 for an example), and their spatial arrangements, and provides final feedback before general
deployment. For example, results confirmed that the new signage system is perceived better by participants
when naturally walking in the train station. Indeed, they look at safety signs significantly more in the new safety
signage system condition than in the actual safety signage system condition. It also validates the relevance of
sign deployed on the ground right before the pedestrian track crossing (Figure 6).
Figure 6: Example of new safety signage system on the left and the actual one on the right in the real train
station during the test.
3. Discussion and conclusion
This paper describes the steps involved in developing a new safety signage system. The originality of the
methodology presented, was not only to ground the conception on actual knowledge existing in human factors,
ergonomics, and cognitive psychology to develop a new safety signage system that should be better understood
by pedestrians, but mainly to evaluate each proposition using adaptations of classical and well-known
methodologies and paradigms to ensure improvements.
In this matter, qualitative as well as quantitative methodologies were deployed at different steps of the process
depending on their relevance in accordance with the kind of evaluation needed. For example, in the early phases
it was interesting to quantitatively measure the understanding of each pictogram and sentence to acquire
statistical reliability. But without any feedback acquired by qualitative methods such as workshops, with poor
results at the quantitative tests, it would have been harder to know how to improve the proposition. Then, the
further the process progressed the more it became interesting to validate the whole safety signage system in
real-like (virtual reality) and real (train station) situations to ensure the right adequation with the ecological
context. Here again, a double check with quantitative data was relevant to statistically validate the results. Thus,
we believe that the presented research highlights the complementarities of the methods, which, to the best of
our knowledge, had not previously been realised in this context.
To sum up, one major finding of this research is that we propose to go beyond the initial development of a safety
signage system - we develop an entire guide to test the understanding, perception, and memorisation of the
new developed concepts. Moreover, we explain how the various methodologies used in the human factors and
experimental psychology domains can be used and combined to optimize the test. We do believe that the
entirety or parts of the methodology described in this paper would be of interest to anyone wishing to develop
a system (for safety purposes or not), that can be well understood by humans.
Finally, the results obtained at each step of the development helped to define a better safety signage system
which can be well perceived, understood and memorised, and, therefore, implemented in train stations. We
hope that this new system will help to prevent pedestrian accidents in the future.
[1] Bordon, E. (2004). L'interprétation des pictogrammes: approche interactionnelle d'une sémiotique. Editions
[2] EPSF. (2018). Rapport annuel sur la sécurité des circulations ferroviaires.
[3] Ho, C., & Spence, C. (2009). Using personal warning signals to orient a driver’s gaze. Human Factors, 51(4),
539-556. doi: 10.1177/0018720809341735.
[4] Laughery, K. R., & Wogalter, M. S. (2014). A three-stage model summarizes product warning and
environmental sign research. Safety science, 61, 3-10.
[5] Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and
conjunctions. Nature, 390(6657), 279-281.
[6] Morgagni, S., Lemesle, P., & Maurice, M. Charte signalétique 2012 : Guide d’application, Société
Nationale des Chemins de Fer Français (SNCF), 2012 (2nd version, 2013).
[7] Sanders, M. S., & McCormick, E. J. (1987). Human factors in engineering and design (6e éd.). New-York :
[8] Stanton. N. A., & Edworthy, J. (eds.) (1999). Human factors in auditory warnings. Aldershot, Great Britain :
[9] Tingvall, C., Eckstein, L., Hammer, M. (2008). Government and Industry Perspectives on Driver Distraction.
In Regan, M., Lee, J. and Young, K. (eds.) Driver distraction, theory, effects, and mitigation. CRC Press,
[10] Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive
psychology, 12(1), 97-136.
[11] Williams, D. J., & Noyes, J. M. (2007). How does our perception of risk influence decision-making?
Implications for the design of risk information. Theoretical issues in ergonomics science, 8(1), 1-35.
[12] Wogalter, M. S., Laughery, K. R., Mayhorn, C. B., & Salvendy, G. (2012). Warnings and hazard
communications. In G. Salvendy (Dir.), Handbook of human factors and ergonomics (6e éd.) (pp. 868-894).
Hoboken, United States of America : John Wiley & Sons, Inc.
[13] Young, S. L., & Wogalter, M. S. (1990). Comprehension and Memory of Instruction Manual Warnings:
Conspicuous Print and Pictorial Icons. Human Factors, 32(6), 637649.
... In the study of Grison et al., (2022), they explored the possibility of using pictograms, an animated virtual reality simulation and a realistic virtual reality simulation of the train station as safety signage system training to protect pedestrians from railway-related accidents [16]. The main concepts analyzed in this study were attention, visual search, comprehension, and memorization. ...
Technical Report
Full-text available
This research examines the emerging role of eye-tracking in virtual reality. Still, it is necessary to clarify precisely what eye-tracking means and its origins. Research into eye-tracking has a long history. According to Wade (2010) (p.34), eye tracking research comes “from Aristotle to Yarbus” [46]. In 1823, interesting medical research was conducted when Charles Bell put forth that the eye’s movement is controlled by the brain leading to visual orientation. Furthermore, he described that the participants could have active or passive eye movement after watching an image. From these historical beliefs, eye-tracking is grounded on the theoretical notion that eyes move in different directions to bring high resolution within our visual field. This notion facilitates the individuals to look through fine image detailing, thereby attaining the centre of the direction span. For this purpose, eye-tracking technologies and the developers of these technologies must have a thorough understanding of the anatomy of the eye so that these technologies can act as an accomplished facilitator of recording the eye’s movement and stare direction across periods and its engagement to perform various tasks.
Full-text available
Two experiments examined the effects of increasing the noticeability of instruction manual warnings on subsequent comprehension and memory performance. Participants read one of four instruction manuals for a gas-powered electric generator (Experiment 1) or a natural-gas oven (Experiment 2) on the assumption that they would later operate the equipment. The appearance of eight different warning messages in the manuals was altered in two ways: (1) the verbal messages were printed either in conspicuous print (larger text with color highlighting) or in plain print (same as the other text), and (2) either the verbal warning messages were accompanied by compatible pictorial icons or the icons were absent. Results showed that participants who received the conspicuous print, icons present manual better comprehended and recalled the verbal warning messages (Experiments 1 and 2) and better identified the semantic meaning of the icons (Experiment 1) than did participants who received the other three manuals. Implications for the design of instruction manual warnings are discussed.
Full-text available
The past 25years have experienced a substantial amount of research on safety communications, more specifically, warnings. This article focuses on the most important factors in designing effective warnings. The warning process is modeled in three stages. Effective warnings attract attention; elicit knowledge, and enable compliance behavior. Two main categories are design factors and non-design factors. Non-design factors include the effects of target audience and situational factors. For attention, important design factors are size, color/contrast, signal word, graphics, and format. The non-design factors for attention include context, location, and distraction. Design factors affecting the knowledge and compliance stages include explicit wording and pictorials to provide hazard, consequences and instructional content. Non-design factors for the knowledge stage include familiarity and perceived hazard. Non-design factors for the compliance stage include modeling the behavior of others and cost of compliance. The research literature offers practical design recommendations to aid application decisions.
Perceptions of risk are an inherent part of the decision-making process. What is more, risk perception can be understood as an individual's assessment of risk, and the adequacy of any risk assessment is reliant on the adequacy of the accessible risk information. Consequently, one way to understand the effect of risk perception on decision-making, and the approach taken in this literature review, is to understand how risk information is communicated and received by an individual. A number of factors are identified that have been found to influence perceptions of risk, which are related to the design of risk messages: the message (colour, signal word, surround shape, and the framing effect), the source of the message (credibility and trust), and the target of the message (risk target). It is concluded that, in order to design effective risk communications, and to facilitate decision-making and safe behaviour, these factors need to be considered, in a context-dependent manner.
We report a series of three experiments designed to assess the relative speed with which people can initiate speeded head-orienting responses following the presentation of spatial warning signals. Recent cognitive neuroscience findings have shown that the human brain tends to treat stimuli occurring in peripersonal space as being somehow more behaviorally relevant and attention demanding than stimuli occurring in extrapersonal space. These brain mechanisms may be exploited in the design of warning signals. Experiment 1 assessed the effectiveness of various different unisensory warning signals in eliciting a head-turning response to look at the potential source of danger requiring participants' immediate attention; Experiment 2 assessed the latency of a driver's responses to events occurring in the cued direction; Experiment 3 assessed the relative effectiveness of various warning signals in reorienting a person's gaze back to a central driving task while he or she was distracted by a secondary task. The results show that participants initiated head-turning movements and made speeded discrimination or braking responses significantly more rapidly following the presentation of a close rear auditory warning signal than following the presentation of either a far frontal auditory warning signal, a vibrotactile warning signal presented to their waist, or a peripheral visual warning signal. These results support the claim that the introduction of peripersonal warning signals results in a significant performance advantage relative to traditionally designed warnings. Warning systems that have been designed around the constraints of the human brain offer great potential in the future design ofmultisensory interfaces.
A new hypothesis about the role of focused attention is proposed. The feature-integration theory of attention suggests that attention must be directed serially to each stimulus in a display whenever conjunctions of more than one separable feature are needed to characterize or distinguish the possible objects presented. A number of predictions were tested in a variety of paradigms including visual search, texture segregation, identification and localization, and using both separable dimensions (shape and color) and local elements or parts of figures (lines, curves, etc. in letters) as the features to be integrated into complex wholes. The results were in general consistent with the hypothesis. They offer a new set of criteria for distinguishing separable from integral features and a new rationale for predicting which tasks will show attention limits and which will not.
Short-term memory storage can be divided into separate subsystems for verbal information and visual information, and recent studies have begun to delineate the neural substrates of these working-memory systems. Although the verbal storage system has been well characterized, the storage capacity of visual working memory has not yet been established for simple, suprathreshold features or for conjunctions of features. Here we demonstrate that it is possible to retain information about only four colours or orientations in visual working memory at one time. However, it is also possible to retain both the colour and the orientation of four objects, indicating that visual working memory stores integrated objects rather than individual features. Indeed, objects defined by a conjunction of four features can be retained in working memory just as well as single-feature objects, allowing sixteen individual features to be retained when distributed across four objects. Thus, the capacity of visual working memory must be understood in terms of integrated objects rather than individual features, which places significant constraints on cognitive and neurobiological models of the temporary storage of visual information.
L'interprétation des pictogrammes: approche interactionnelle d'une sémiotique
  • E Bordon
Bordon, E. (2004). L'interprétation des pictogrammes: approche interactionnelle d'une sémiotique. Editions L'Harmattan