Evaluating safety signage systems in train stations: a qualitative and quantitative methodology.
Elise GRISON1, Samuel AUPETIT2, Sara ESCAICH2, Simone MORGAGNI1
1SNCF, La Plaine Saint-Denis, France
2Ergocentre, Orléans, France
Corresponding Author: Elise Grison (firstname.lastname@example.org)
In this paper we describe a methodology developed to test a new safety signage system before its
implementation in train stations. Every year, pedestrians are involved in numerous railway-related accidents in
train stations. In order to prevent such accidents, a cognition centred research project was undertaken to
rethink, redesign and evaluate safety signage systems (Morgagni, Lemesle & Maurice, 2012). Three main steps
have been defined, each one based on both qualitative and quantitative approaches, collecting subjective as
well as objective data, and integrating methodologies from both cognitive and ergonomic domains. The results
obtained helped to decide the future of safety signage systems in French train stations and could be adapted
and reused elsewhere. Indeed, the proposed methodology provides complete cues for future implementation
and deployment of safety signage systems and systematises a research methodology for future similar needs.
Keywords: experimental psychology; human factors; signs; virtual reality.
The prevention of pedestrian accidents in train stations is an important subject of interest given that, every year,
numerous railway-related accidents involving pedestrians are recorded around the world. As an example, in
France, we observe around 140 pedestrian-accidents every year (EPSF, 2018). Several of these accidents may be
due to pedestrians’ poor consideration or comprehension of risks when entering a railway zone. Amongst the
various possible ways to act against these accidents, such as prevention and sensibilisation campaigns that can
be done at regional or national levels, another action might concern the development of a more effective rail
safety signage system that will be deployed in train stations.
Currently in France, safety signage deployed in train stations convey the important messages concerning risks
and recommended behaviour. However, the findings found in the human factors and cognitive literature of
these past thirty years suggest numerous improvements to safety signage, such that they be better understood,
memorised and impactful on human behaviour (Sanders & McCormick, 1987; Stanton & Edworthy, 1999).
Moreover, the literature has also consciously defined how to deliver the information, in terms of the type and
structure of the message and the format of sensorial modality (auditory, visual). The structure of the message
and the preferred format is also dependent on the temporal and spatial proximity of the danger (Tingvall,
Eckstein, & Hammer 2008; Ho et Spence, 2009). For example, it has been shown that auditory formats should
be preferred when the danger is imminent. When not so spatially close to the danger (on the platform or in the
waiting area of the train station), visual formats and verbal messages can be used. Generally, verbal messages
displayed on signs should be short, structured with a hierarchy based on the relative importance of information
(Wogalter et al., 2012). They can also benefit from adequate pictograms for an easier and quicker
comprehension. Several other recommendations can be found in the literature (Laughery et Wogalter, 2014;
Williams et Noyes, 2007; Young et Wogalter, 1990), and may be a good starting point to propose a new safety
signage system that should be well understood and efficient for pedestrians.
To our knowledge, however, no study based on human factors and cognition has already documented the
process of creating a new safety signage system for the context of railways and, especially, for train stations.
Indeed, it can be easily hypothesised that, once applied in a specific context, some of recommendations found
in the literature will not be simple to follow or to be really adapted. Moreover, we did not find in the literature
any precise methodology developed to test the efficiency of a global safety signage system on human detection,
understanding and memorisation.
Thus, the main aim of this paper is to report on the process of the development of a new safety signage system
based on recommendations found in the literature. We will specifically focus our interest on the methodologies
developed from classical human factors, ergonomics and experimental psychology, to test the efficiency of the
new signage system developed.
Based on the literature, we developed several concepts, including 3 different graphical styles and 2 spatial
arrangements (one that displays a sign and message on the ground, and another on mast). In all concepts, the
first version of the verbal messages has been worked to be as short and simple as possible, and pictograms have
been developed to illustrate each important information.
2.1 Step 1: Test and validation of general principles
After the first conception phase based on state-of-the-art technical and scientific knowledge, an online
questionnaire was proposed to objectively evaluate the understanding of the various elements that could be
used to compose the new signage system, such as pictograms (and their combination, Bordon, 2004), text
messages, and spatial arrangements.
106 participants responded to the online questionnaire (mean age = 40 years; 66 women, 40 men). They were
first presented the pictogram one-by-one, without any context, and asked to describe the pictogram’s
signification. Next, the same question was asked showing the pictograms in context. These two steps were used
to evaluate combinations between pictograms and their describing texts. Figure 1 below presents examples, for
two concepts, of the task of the combination of sign and text in train station contexts.
Figure 1: Example signage in the train station context used for the questionnaire.
Two additional workshops, with 18 participants (mean age = 35; 9 women, 9 men) were conducted to
complete the questionnaire with subjective and qualitative data to suggest improvements for elements that
appeared to be hard to understand for the participants.
The tasks developed for the questionnaire were used as a basis for the discussions. Participants were asked to
reflect on and propose improvements, individually as well as in groups of 3-4. They were also asked to place
every element of the new signage system in a small mock-up train station to have a naïve feedback on spatial
arrangements. All interactions between participants were recorded and analysed to understand the reasons for
their placement choices. The results of these workshops helped to select the better concept for the next steps,
and to improve message phrasing and pictogram representations. An example of how participants’ remarks and
recommendations have been considered is in presented in Figure 2.
Figure 2: Example of steps of conception before and after workshop with participants, and of improvements
made on the pictogram. In a) the initial version proposed to explain that pedestrians should keep their
distance from the edge of the platform as they might be hit by the train. In b) some modifications proposed by
participants. In c) two new propositions.
2.2 Step 2: Selection
To assess their cognitive understanding and efficiency compared to the existing safety signage system, two
studies were conducted on two new concepts which were developed based on data collected in step 1. The
purpose of this step was to acquire objective and quantitative data but combined to ecological data to ensure a
good selection of the better concept.
A first study, based on a virtual reality environment, was conducted in a real-like situation but providing a safe
and controlled environment to study human behaviour. 10 participants took part in the study (mean age = 38.9;
4 women, 6 men).
Figure 3: The virtual train station used for the test.
A virtual environment of a small train station was developed (Figure 3). Three versions of the environment were
developed, one for each safety signage to be tested (see Figure 4 for examples of the actual signage system and
a new system which was tested). Participants were immersed in the virtual environment through a virtual reality
headset equipped with an eye-tracking system (HTC Vive pro eye). To encourage participants to walk naturally
into the train station and observe their environment without revealing to them the real purpose of the study (to
avoid inducing any bias), a scenario in which participants had to review new services in train stations was
proposed. Participants were asked to find the new services and make a judgment on them. Participants were
thus exposed to the safety signage systems and the observation of their natural behaviour was guaranteed.
Figure 4: The virtual train station in the actual signage system condition on the left, and with a new system on
Different behavioural and physiological measures were recorded during the task (gaze behaviour and kinematic
data). Additionally, the participants’ subjective points of view were addressed through interviews to collect their
experience in virtual reality and recommendations to improve the new signage systems.
The second study (an online study) was implemented to evaluate using a larger number of participants and
through more classical cognitive tasks, the efficiency of the new safety signages systems compared to the
existing one on attention, visual search, comprehension, and memorisation. This was also the opportunity to
adjust some minor elements, such as colour, of the new signage system. 922 participants took part in the study
(mean age = 30.9; 355 men and 567 women).
The first task, based on a “change detection paradigm” (Luck & Vogel, 1997), was conceived to evaluate if and
how the signages are detected in a train station environment. Pairs of images featuring a real train station
environment were presented – the second one could include some modifications (presence, absence,
displacement) concerning the signage systems to be evaluated. Participants had to indicate if the two images
were the same or not. The second task aimed at defining how easy it is to find safety signage and information
depending on their visual salience. We used a classical visual search task (Treisman & Gelade, 1980), in which
participants had to search for a sign in a visual scene (train station or fictive scene). Finally, a visual story of a
person navigating in a train station and seeing a variety of the safety signage signs in the environment was
presented to participants. In a following step, participants had to choose amongst various signages, those that
were previously presented.
Results obtained in this phase helped to select the right colour for the new signage system that will allow it to
be detected more clearly in the environment. It also helped to validate the better spatial arrangement; for
example, it appears that deploying all signs and messages on the ground is not relevant as it will be harder to
detect the information from far away.
2.3 Step 3: Validation
This final step was conceived to appreciate in ecological context, the advantages of the winning system in step
2, on human safety behaviour as well as on passenger understanding. 43 participants took part in the study
(mean age = 41.5; 22 men, 21 women).
We draw on the methodology developed in virtual reality to develop a realistic scenario for participants.
Participants were asked to navigate through a train station equipped with the current or new safety signage
system, as if they were waiting for a friend arriving on the next train. During their navigation, natural gaze
behaviour was recorded through Tobii Pro Glasses 2 (See Figure 5). In a second part of the study, participants
were asked to find specific signages in their environment (search time and gaze behaviour were recorded).
Finally, a questionnaire about the understanding of safety signage was proposed and their global appreciation
Figure 5: A participant reading a sign with the eye-tracking glasses.
This final study allows to compare the understanding of the old and new safety signage systems in real context
(see Figure 6 for an example), and their spatial arrangements, and provides final feedback before general
deployment. For example, results confirmed that the new signage system is perceived better by participants
when naturally walking in the train station. Indeed, they look at safety signs significantly more in the new safety
signage system condition than in the actual safety signage system condition. It also validates the relevance of
sign deployed on the ground right before the pedestrian track crossing (Figure 6).
Figure 6: Example of new safety signage system on the left and the actual one on the right in the real train
station during the test.
3. Discussion and conclusion
This paper describes the steps involved in developing a new safety signage system. The originality of the
methodology presented, was not only to ground the conception on actual knowledge existing in human factors,
ergonomics, and cognitive psychology to develop a new safety signage system that should be better understood
by pedestrians, but mainly to evaluate each proposition using adaptations of classical and well-known
methodologies and paradigms to ensure improvements.
In this matter, qualitative as well as quantitative methodologies were deployed at different steps of the process
depending on their relevance in accordance with the kind of evaluation needed. For example, in the early phases
it was interesting to quantitatively measure the understanding of each pictogram and sentence to acquire
statistical reliability. But without any feedback acquired by qualitative methods such as workshops, with poor
results at the quantitative tests, it would have been harder to know how to improve the proposition. Then, the
further the process progressed the more it became interesting to validate the whole safety signage system in
real-like (virtual reality) and real (train station) situations to ensure the right adequation with the ecological
context. Here again, a double check with quantitative data was relevant to statistically validate the results. Thus,
we believe that the presented research highlights the complementarities of the methods, which, to the best of
our knowledge, had not previously been realised in this context.
To sum up, one major finding of this research is that we propose to go beyond the initial development of a safety
signage system - we develop an entire guide to test the understanding, perception, and memorisation of the
new developed concepts. Moreover, we explain how the various methodologies used in the human factors and
experimental psychology domains can be used and combined to optimize the test. We do believe that the
entirety or parts of the methodology described in this paper would be of interest to anyone wishing to develop
a system (for safety purposes or not), that can be well understood by humans.
Finally, the results obtained at each step of the development helped to define a better safety signage system
which can be well perceived, understood and memorised, and, therefore, implemented in train stations. We
hope that this new system will help to prevent pedestrian accidents in the future.
 Bordon, E. (2004). L'interprétation des pictogrammes: approche interactionnelle d'une sémiotique. Editions
 EPSF. (2018). Rapport annuel sur la sécurité des circulations ferroviaires.
 Ho, C., & Spence, C. (2009). Using personal warning signals to orient a driver’s gaze. Human Factors, 51(4),
539-556. doi: 10.1177/0018720809341735.
 Laughery, K. R., & Wogalter, M. S. (2014). A three-stage model summarizes product warning and
environmental sign research. Safety science, 61, 3-10.
 Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and
conjunctions. Nature, 390(6657), 279-281.
 Morgagni, S., Lemesle, P., & Maurice, M. Charte signalétique 2012 : Guide d’application, Société
Nationale des Chemins de Fer Français (SNCF), 2012 (2nd version, 2013).
 Sanders, M. S., & McCormick, E. J. (1987). Human factors in engineering and design (6e éd.). New-York :
 Stanton. N. A., & Edworthy, J. (eds.) (1999). Human factors in auditory warnings. Aldershot, Great Britain :
 Tingvall, C., Eckstein, L., Hammer, M. (2008). Government and Industry Perspectives on Driver Distraction.
In Regan, M., Lee, J. and Young, K. (eds.) Driver distraction, theory, effects, and mitigation. CRC Press,
 Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive
psychology, 12(1), 97-136.
 Williams, D. J., & Noyes, J. M. (2007). How does our perception of risk influence decision-making?
Implications for the design of risk information. Theoretical issues in ergonomics science, 8(1), 1-35.
 Wogalter, M. S., Laughery, K. R., Mayhorn, C. B., & Salvendy, G. (2012). Warnings and hazard
communications. In G. Salvendy (Dir.), Handbook of human factors and ergonomics (6e éd.) (pp. 868-894).
Hoboken, United States of America : John Wiley & Sons, Inc.
 Young, S. L., & Wogalter, M. S. (1990). Comprehension and Memory of Instruction Manual Warnings:
Conspicuous Print and Pictorial Icons. Human Factors, 32(6), 637–649.