PreprintPDF Available

SAFETY4RAILS EU project: Protecting railway and metro infrastructure against combined cyber-physical attacks

Authors:
  • International Union of Railways (UIC)
  • International Union of Railways (UIC)
  • International Union of Railways (UIC)
Preprints and early-stage research may not have been peer reviewed yet.

Abstract

As critical infrastructures, railways and metros, as well as related intermodal transport operators, can be considered as attractive targets for cyber and/or physical attacks. In this context, the goal of the EU H2020 SAFETY4RAILS project is to ensure resilience of rail and metro transport in the case of such crises. To do so, SAFETY4RAILS is co-developing a tool (SAFETY4RAILS Information System (S4RIS) platform) with relevant end-users. This paper focuses on the development of the evaluation framework for the S4RIS prototype that will be tested by the end-users within two series of simulation exercises based on different use cases, in an operational environment. It describes how the S4RIS platform will be evaluated by the end-users, ensuring the quality and consistency of the analysis and evaluation.
PREPRINT 11/10/2022 1
SAFETY4RAILS EU project: Protecting railway and metro infrastructure against combined
cyber-physical attacks
Marie-Hélène BONNEAU1, Laura PETERSEN1, Grigore HAVARNEANU1, Stephen Crabbe2
1UIC, Paris, France
Corresponding Author: Marie-Hélène Bonneau (bonneau@UIC.org)
2 Fraunhofer Institute for High-Speed Dynamics I Fraunhofer-Institut für Kurzzeitdynamik
Ernst-Mach-Institut · EMI, Efringen-Kirchen, Germany
Abstract
As critical infrastructures, railways and metros, as well as related intermodal transport operators, can be
considered as attractive targets for cyber and/or physical attacks. In this context, the goal of the EU H2020
SAFETY4RAILS project is to ensure resilience of rail and metro transport in the case of such crises. To do so,
SAFETY4RAILS is co-developing a tool (SAFETY4RAILS Information System (S4RIS) platform) with relevant end-
users. This paper focusses on the development of the evaluation framework for the S4RIS prototype that will be
tested by the end-users within two series of simulation exercises based on different use-cases, in an operational
environment. It describes how the S4RIS platform will be evaluated by the end-users, ensuring the quality and
consistency of the analysis and evaluation.
Keywords: railway, safety, security, cyber security, critical infrastructure, resilience
1. Introduction
SAFETY4RAILS is the acronym for the innovation EU funded project: Data-based analysis for SAFETY and security
protection FOR detection, prevention, mitigation and response in trans-modal metro and RAILway networkS. It
started in October 2020 for a period of two years.
Railways and Metros are safe, efficient, reliable and environmentally friendly mass carriers, and they are
becoming even more important means of transportation given the need to address climate change. However,
as critical infrastructures, railways and metros, as well as related intermodal transport operators, turn into
attractive targets for cyber and/or physical attacks. The SAFETY4RAILS project delivers methods and systems to
increase the safety and recovery of track-based inter-city railway and intra-city metro transportation. It
addresses both cyber-only attacks (such as impact from WannaCry infections in 2017), physical-only attacks
(such as the Madrid commuter trains bombing in 2004) and combined cyber-physical attacks, which are
important emerging scenarios given increasing Internet of Things (IoT) infrastructure integration.
Based on past failures analysis and end-users consultation, an overview of threats and risks faced by railways
and metro operators was generated within the project [1], focused on the identification of incidents type,
involved or concerned stakeholders and segments and assets targeted or impacted during both railways and
metro incidents. From the analysis of past failures, the following key trends were derived:
A rise in the proportion of cyber-attacks amongst the incidents, especially those with criminal motivation.
The railway and metro sector experience the same increasing convergence between logical and physical
security, that is observed in other Operational Technology (OT)-related environments.
The nature of the systems applied in the sector and the typical incidents will require a special attention
towards interdependencies and cascading effects.
From a security aspect, the human factor is the weakest link in the security chain mainly because the lack
of awareness; related requirements need to be defined within the project.
In a nutshell, the threats and risks considered within SAFETY4RAILS project are divided into three main
PREPRINT 11/10/2022 2
categories: physical, cyber incident and combined (physical-cyber) incidents. The below table summarises the
typology of threats identified within SAFETY4RAILS project framework:
Table 1 Typology of threats identified within SAFETY4RAILS project framework
SAFETY4RAILS concentrates on rush hour rail transport scenarios where many passengers are using mass transit
(i.e., commuter and regional trains or metro lines) to commute to work or attend mass events (e.g., large multi-
venue sporting events such as Olympic games). When an incident occurs during heavy usage, railway and metro
operators must consider several aspects to ensure passenger safety and security. For instance, they have to
carry out a threat analysis, maintain situation awareness, establish crisis communication and response, and they
have to ensure that mitigation steps are taken and communicated to travellers and other users. SAFETY4RAILS
aims to improve the handling of such events through a holistic approach. It analyses the cyber-physical resilience
of railway and metro systems and is developing mitigation strategies for an efficient response [2].
The main output of the project is the SAFETY4RAILS Information System (S4RIS) which is a web-based decision
support platform that integrates and extends 18 tools to support rail and metro operators to manage cyber and
physical risks. Most of the tools are already at a Technology Readiness Level (TRL) of 5-6, which means that they
were validated or demonstrated in a relevant environment [3]. They will be further developed to meet the scope
and objectives of the project. An evaluation of their performance will then be carried out at selected end-user
pilot sites.
This paper presents the methodology which is going to be applied for the evaluation of the prototype of the
S4RIS platform that will be tested in operational environments within four simulation exercises to be held in
2022. This evaluation mainly focuses on the impact of the S4RIS platform in enhancing resilience against
combined cyber-physical threats to railway infrastructure and metro systems. The objective of the paper is to
describe how the simulation exercises will be evaluated by the end-users, ensuring the quality and consistency
of the analysis and evaluation.
2. Methodology
Developing the SAFETY4RAILS evaluation methodology followed four steps. All steps included a transverse
engagement with real end-users from the project network. The SAFETY4RAILS Consortium is composed of 31
partners from 13 different countries (Germany, France, Spain, Turkey, Italy, Belgium, Switzerland, United
Kingdom, Greece, Finland, Hungary, Israel and the Netherlands). They represent railway operators, railway
infrastructure managers, research centres, academia and industry suppliers and bring a range of complementary
PREPRINT 11/10/2022 3
skills required for this multidisciplinary project. The end-users in the consortium are rail infrastructure managers
and operators (PRORAIL in the Netherlands, RFI in Italy, TCDD in Turkey, FGC in Barcelona), metro operators
(MDM in Madrid, EGO in Ankara), city manager (CDM in Milano) and international railways organisation (UIC in
Paris). The consortium is reinforced through an Advisory Board which further ensures that the SAFETY4RAILS
outcomes meet the needs of a wider sample of railway and metro operators. About 20 advisors from a wide
range of sectors: rail, metro, public transport, cyber security, smart city domains, transport ministries, and
security, covering 10 countries: Belgium, France, Bulgaria, Poland, Spain, Sweden, Germany, Netherlands,
Switzerland and Turkey make up the advisory board. The Advisory Board is being extended throughout the life
of the project through a permanent recruitment process of new members. Input from the end-users was
gathered through a multimethodological approach, using questionnaires, interviews and workshops/focus
groups.
First, a set of end-user requirements were established. The requirements and specifications are input into both
the S4RIS development cycle in SAFETY4RAILS and also evaluation and validation cycles. The second step was to
review the literature regarding existing evaluation frameworks. From this, the two most relevant for the
SAFETY4RAILS project were UK FSR and Design Science Research Methodology. The third step was to consult
with project end-users regarding existing evaluation methodologies for security solutions. Input was provided
by CDM, FGC, PRORAIL and PKP/UIC. Based on the end-user requirements, literature review of evaluation
methodologies, and end-user consultation, the SAFETY4RAILS evaluation methodology was developed and is
presented in the next section. Simultaneously, the fourth step consisted of the development of the use-cases
where the S4RIS prototype would be tested. This involved the identification of test scenarios, description of the
simulation (configuration, adequate scenario description/ development, different roles and tasks (“player”,
observer). The partial results of this methodological process are presented below.
3. Results
Step 1: Collection and analysis of end-user requirements
A first set of 109 high-level end users’ needs and requirements were formulated. Each requirement was
described and coded with a priority rank, corresponding threats events, and corresponding risk and threat
management phase. Six major trends were derived from this list of high-level end-user needs and requirements:
1. Improvement of both internal and external communications, in a sector where crisis management
involve a large number of stakeholders - both internally within different services of the impacted
transport company and externally to exchange and collaborate with other stakeholders (for instance,
law enforcement or first responders).
2. Secure systems and assets, as a critical infrastructure railways and metro operators must ensure safety
of passengers and goods transported via their infrastructures security (management of security
services, securitisation of access to systems and assets to encryption procedures and techniques to
ensure data protection and security).
3. Closer cooperation with authorities is a key element of crisis and incident management, via, for
instance,operation information exchanges faciliated by dedicated systems, coordination of measures
and actions implementation, joint trainings.
4. Advanced monitoring and detection capabilities, as a condition to collection necessary information for
continuous situational awareness update aiming as most as possible close to real-time.
5. Use of simulation for anticipation, prevention and/or mitigation of the impact of an incident that should
be desgined to integrate a massive amount of data and information to correctly simulate cascading
effects wihtin railways and networks and assets, characterised by their variety interdependencies and
complexity.
6. Management of data flows to support decision-making, aiming the provision of added-value
information, to improve significantly their capabilities to address proprely risks and threats before they
occur or when the crisis is emerging.
PREPRINT 11/10/2022 4
Then, further requirements and specifications for the overall S4RIS platform application were determined. About
200 requirements specific to the S4RIS platform itself and to the 18 individual tools were defined [4]. These
requirements focus on:
1. Interoperability and integration.
2. Graphical User Interface (GUI).
3. Security standards and data protection.
4. Tools specific requirements with the assignment of the following resilience-phases: prevention,
detection, response, recovery.
Step 2: Literature review
The literature review produced two main bodies of research of relevance for the SAFETY4RAILS project: the UK’s
Forensic Science Regulator (hereafter referred to as “UK FSR”) [5] and Design Science Research Method. UK FSR
states that “validation involves demonstrating that a method used for any form of analysis is fit for the specific
purpose intended, i.e., the results can be relied on [6].” The UK FSR’s guidance is focussed on validation of
technical methods and procedures used by forensic units [7], and while the SAFETY4RAILS project is not focussed
on forensics, some data collected by S4RIS in operational systems has the potential to be the basis for expert
evidence in criminal justice systems and in addition in the rail and metro domain end-users need also to be able
to rely on the results (e.g., after the project at TRL9).
Erreur ! Source du renvoi introuvable., which is taken from UK FSR, provides a simplified representation of how
end-users requirements can be perceived to be at the centre of both the first initiation of research for the
development of a method (cycle bottom left), the actual development of a method (bottom right) and the
validation of a method (top). In this representation: the increase in TRLs for a new method would be in the same
order i.e., lowest TRLs bottom left and highest TRLs top; and validation of a final method (top) is separated from
the development of a method.
Figure 1 End-user requirements importance and place in distinct cycles [8]
Design Science Research Methodology (DSRM) enables the development, design and building of new artifacts
[9]. A key part of DSRM is the evaluation of the artefact and this consists of rigorously demonstrating that the
artefact achieves its goals. While there is no one specific evaluation methodology recognised as part of DSRM,
the field has several possible ways of going forward. The Framework for Evaluation in Design Science (FEDS) was
PREPRINT 11/10/2022 5
developed to aid Design Science researchers with the evaluation phase of the methodology. FEDS is made up of
four steps: (1) explain the goals of the evaluation, (2) choose the evaluation strategy or strategies, (3) determine
the properties to evaluate, and (4) design the individual evaluation episode(s). Before the evaluation, Pries-Heje
et al. (2008) [10] recommend that the following three questions be answered: 1. What is going to be evaluated?
2. How is it going to be evaluated? 3. When is it going to be evaluated?
The proposed SAFETY4RAILs evaluation methodology is then based on i) the UK FSR guidelines on validation and
ii) the Design Science Research Methodology where there are three questions (“what, how and when”) but with
the addition of the “who”.
Step 3: End-user consultation
According to the feedback by the questioned project end-users, no standardised methodology for the evaluation
of new security technological solution in the transport sector has been identified. Existing evaluation
methodologies are mostly based on risk assessment methodologies using the evolution of the number of
occurrences of security incidents and KPIs such as time for service recovery or delays caused by the incidents.
The end-users evaluated new security solutions, by applying certain varying KPIs and their expert knowledge to
the object to be assessed. Most often, end-users tend to perform evaluations based on the set of criteria laid
out in the call for tender. However, based on the testimonies of the end-users and their experience, it can be
assumed very generally that following steps are taken to evaluate new security technology tools:
1. Current situation assessment - conduct an overview of the existing technology taking into account their
strengths and weaknesses. The focus here is to comprehend which areas need higher level of security
and/or which technologies currently used could be augmented or supported by other solutions.
2. Define needs and expectations describe the requirements towards new technology very clearly. Focus
on technological and functional aspects.
3. Determine the budget.
4. Investigate the market and available solutions.
5. Announce call for tenders (e.g., Request for Quote; Invitation to Offer) describing especially:
a. Defined requirements they should be based on what users try to achieve, not on a particular
technical solution (that way novel solutions can be found).
b. Defined evaluation criteria, e.g.,
i. Capabilities how the technology capability meets the needs?
ii. Security is the technology compliant with the latest security standards?
iii. Is it a long-term solution?
iv. Flexibility;
v. Integration capabilities to be able to integrate a program/system with the one currently used;
vi. Cultural alignment to determine how to work with a technology partner in the long-term;
vii. Cost of services to be aware of the maintenance, setup and potential licensing costs;
viii. Technology acceptance.
c. The specification and any appropriate document to the specification.
d. Test phase period to evaluate selected technologies in the test environment.
e. The level of detail used for requirements: either draw up a detailed list of requirements (including
their importance) or describe them in a general way and focus on the fundamental goals.
f. Others.
6. Set the deadline for the receipt of tenders.
Step 4: Definition of the simulation exercises Scenarios
A Use-Case is defined in the SAFETY4RAILS project as a “High-level description of the problem that needs to be
addressed by means of functionalities/technology”. On the other hand, a Scenario is defined as “An instance of
PREPRINT 11/10/2022 6
a Use-Case/Use-cases describing the concrete set of actions to be taken”. Therefore, the finalised Use-Cases will
be the foundation for the definition of the Simulation Exercise Scenarios [11].
During the end-user requirements collection phase, a preliminary description of 14 Use-Cases was prepared by
the end-users from the consortium. An in-depth analysis of these Use-Cases was then performed in compliance
with the end-user needs, aligning them with the expected contribution from each tool provider.
In a next step, the consortium achieved consensus on relevant Use-Cases where S4RIS, and the capabilities
offered by the tools brought to the project, could be applied. The table below lists the final set of use cases that
could be considered within the simulation exercises and the project partner who proposed the use-case.
Table 2 Use cases list
Type of Use case
Use cases
Partner
1
Natural disaster
Flooding during major event
CDM
2
Natural disaster
Track interception due to a landslide that causes an
immobilization of the train
FGC
3
Physical attacks
Terroristic attack using firearms inside a railway station
RFI
4
Physical attacks
(potential) terroristic attack via IED carried via baggage
RFI
5
Cyber-attack
Train Failure inside a tunnel without possibility to
communicate with the train driver
FGC
6
Physical attack
Intrusion and bomb planted
EGO
7
Physical attack
Intrusion in sensitive place
EGO
8
Physical attack
Spoofing attack on existing sensors
TCDD
9
Cyber-attack
Manipulation of data transferred to Operating Center
TCDD
10
Cyber-attack
Hacking of the signalling system causing accidents
PRORAIL
11
Combined Cyber-Physical
Combined Cyber-Physical attack during a football game
MDM
12
Cyber or physical attack
Level crossing accident: sabotage or cyber-attack on the LC
equipment
MTRS
Based on the description of the use cases, the Simulation Exercises (SE) are described with:
- The script of the scenario and the description of the contribution of each tool to each resilience stage
(Prevention / Detection / Response/ Recovery).
- The participants.
- The requirements and objectives per tool which will be the basis for the evaluation.
- Location & dates.
- The organisation of the Simulation Exercise per resilience phase and per tool.
4. The SAFETY4RAILS evaluation methodology and its further application in the project
The SAFETY4RAILS evaluation methodology based on the work reported in the results section is shown here.
While the answers to which will vary depending on the given scenario studied, at this stage of the research
potential approaches to answering are presented as expected future results.
What is going to be evaluated?
The S4RIS platform is an integrated platform that offers and combines risk assessment, monitoring, simulation
and decision support capabilities. In addition, it offers visualisation means to prevent, forecast, detect, defuse,
respond and mitigate the impact of cyber and physical threats in a holistic methodological and operational
approach resulting in a collaboration between cyber-physical security technologies and actors [12].
The SAFETY4RAILS project aims at a prototype of the S4RIS which can be demonstrated and validated in an
operational environment. The overall philosophy is to bring different technologies together and combine these
with the S4RIS, to provide various functionalities towards supporting the end-users in the railway and metro
PREPRINT 11/10/2022 7
sector in the handling of cyber, physical and combined cyber-physical threats [13].
The evaluation will be conducted as part of four simulation exercises which will take place in Madrid, Ankara,
Rome and Milan. During the exercises, the determined objectives and their associated requirements will be²
assessed by the end-users and expert / technical observers. The simulation will be carried out between February
and July 2022, with time between the simulations to implement identified potential for improvement of the
developed information system.
The evaluation will focus on three main aspects [14] :
The organisation of the exercise (as carried out) with evaluation criteria focussing on the organisation,
the format and length, the added value, the possible improvements, and the lessons learnt.
The performance of the S4RIS against pre-defined objectives related to:
o Usability of the S4RIS Graphic User Interface (GUI) with evaluation criteria focussing on the
ease to use, to find the information needed, to understand, the possible improvements.
o Specific requirements laid out by the end-users with evaluation criteria focussing on the
integration of the functions, the results of the combination of tools, the ethical aspects, the
added value for addressing combined cyber-physical threats, the possible improvements.
The scenario-based requirements/objectives: During each simulation exercise, an integrated version
of the S4RIS platform will be evaluated along with the capabilities that will be simulated in the context
of the scenario used during the specific objectives and the expected performance for each resilience
stage of the scenario with will be evaluated. The main objective of this evaluation, especially during
the exercises, is to provide feedback to the solution providers on the possible improvement of the
tools. The table below gives some examples of specific requirement for the prevention phase in the
case MDM scenario.
Table 3 Example of requirements and objectives for the prevention phase in MDM scenario
No
Req.-ID -
Short name
MDM
-PRE-
1
BB3d_01
Bomb blast loading
MDM-
PRE-2
CaESAR_02
CaESAR should
identify weak points
in the railway/metro
system
MDM-
PRE-3
CaESAR_05
Implementation and
evaluation of
mitigation measures
For each requirement/objective, the evaluation criteria will focus on the fulfilment of the objective, the added
value of the output for the prevention phase and the possible improvements.
Who is going to evaluate?
The evaluators are the potential end-users of the S4RIS platform, who are representatives from transport
operators and infrastructure managers (especially rail and metro). Local authorities and police representatives
who are the main responsible for managing the crisis in case of a terrorist attack may also be also involved in
the evaluation.
The end-users can be divided into three categories depending on their involvement in the project and in the
exercise:
PREPRINT 11/10/2022 8
- End-user representatives organising the exercise (and therefore actively using the tool): A detailed
evaluation of the results with several iterations of the simulation will be organised per tool/capability.
- End-user representatives from the project Consortium (and therefore observing the pilot case): The
evaluation will focus on the usability of the S4RIS platform and the outputs provided.
- End-user representatives outside the Consortium and mainly from the Advisory Board (and therefore
observing the pilot case): The evaluation will focus on the simulation exercise in general and the results
achieved.
How is it going to be evaluated?
The evaluation methodology will include observations, questionnaires, and debriefs to collect individual
feedback, as well as group based-evaluation techniques:
- Observations (open or structured): during the exercise the participants will be asked to perform a
systematic observation of physical quantitative measures/indicators.
- Questionnaires: For each category of requirements described above, a questionnaire, to be filled-in by
the end-users, will be prepared. In each questionnaire, the name, the company and the position of the
responders will be requested. The respondents will also be asked if they are willing to participate in a
follow-up interview. The questionnaire will use both open ended and closed questions. For closed
questions, a Likert scale is to be used.
- Debrief: right after each simulation, a debrief will be carried out to collect feedback from the
participants on strengths and areas for improvement.
- Group-Based techniques: The use of group-based techniques has also been identified as a useful
methodology for usability evaluation [15]. For the participants who agree, a Nominal Group Technique
(NGT) will be organised to collect data for the most relevant open questions (two to three questions).
The data collected through questionnaires and the NGT will be analysed and processed for a follow-up
questionnaire.
When is it going to be evaluated?
Followed by the exercises, the participants will be provided with the questionnaire on the evaluation of the
exercises, the usability of the S4RIS platform and the S4RIS platform specific requirements. A debrief will be then
organised for the open-ended questions.
The Scenario-based requirements will be evaluated after the simulation of each resilience stage. The closed-
ended questions will be raised through an online questionnaire (for example using Slido or a similar platform)
whereas the open-ended questions will be addressed through Nominal Group Techniques.
5. Conclusion
This paper introduced the S4RIS platform and the evaluation and validation methodology that will be applied
during the simulation exercises in operational environment. As the project implementation is only halfway, this
paper should be regarded as work in progress and the presented results considered within their reasonable
limitations.
There are many challenges for the evaluation of the S4RIS platform by the end-users. The first one is the scope
of S4RIS platform which is very broad: 18 tools addressing the four main stages of the resilience approach
(prevention, detection, response, recovery). Therefore, the evaluation by the end-users can only be partial. A
limited set of capabilities will be tested for each scenario given the timeframe for the simulation exercise which
is quite short. The evaluation will further highlight S4RIS novelties especially when dealing with combined cyber-
physical threats. It will be quite high level given the time and the number of tools. The second one is related to
the availability of data. Many tools are based on machine learning and need to be trained within a long time
period to become even more efficient. Moreover, railways have not experienced many cyber-physical threats
for now and therefore historical data are not available.
PREPRINT 11/10/2022 9
Despite these research challenges and limitations of this paper, the scope of the research is new since the project
addresses both physical and cyber-attack as part of emerging threats within a critical part of the transport sector.
It does so in a concrete way, by providing rail and metro operators with a prototype of the S4RIS platform that
integrates and extends 18 tools providing capabilities for the whole resilience approach from prevention,
detection, to response and recovery phases. The platform will provide end-users with results that will help them
for decision making when a disruptive event happen.
The evaluation of each exercise will further serve to improve the tools and their integration as well as the outputs
of the S4RIS platform for next exercises. This evaluation will contribute to the assessment of the Technology
Readiness level especially for TRL 6 (System/subsystem model or prototype demonstration in a relevant
environment) and TRL 7 (System prototype demonstration in an operational environment). It will be used to
assess if the demonstrations/exercises have been performed successfully in a relevant environment.
When completed, the work described in this paper is expected to have desired outcomes and positive impact
for key beneficiaries. Training activities and information material will be developed to increase crisis managers’
and decision-makers’ skills. The simulation exercises methodology will also be used for future research or
practice endeavours. All the lessons learned from these trainings and exercises will support trials and exercises
in the domain of the protection of rail and metro systems including stations and links with the European Union
Agency for Law Enforcement Training (CEPOL) are foreseen as part of the exploitation plan.
Acknowledgment
This work is supported by the H2020 SAFETY4RAILS project, which has received funding from the European
Union’s Horizon 2020 research and innovation program under grant agreement No. 883532.
References
[1]
Ferrando F., de Belloy F., "SAFETY4RAILS Deliverable D2.1", May 2021.
[2]
"SAFETY4RAILS Grant Agreement, version 2.0 with amendment", June 2020.
[3]
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-
annex-g-trl_en.pdf.
[4]
Siebold U., Crabbe S., "SAFETY4RAILS Deliverable D1.4", March 2021.
[5]
Crabbe S., "SAFETY4RAILS D1.1 Project Management Manual, V1.0", p. 9., October 2020.
[6]
"UK Forensic Science Regulator,Guidance: Validation (FSR, Issue 2, 2020)", p.5.
[7]
"UK Forensic Science Regulator,Codes of Practice and Conduct For Forensic Science Providers and
Practitioners in the Criminal Justice System" (FSR, Issue 7, 2021) p. 47.
[8]
Image from "UK Forensic Science Regulator, Guidance: Validation" (FSR, Issue 1, 2014), p.19.
[9]
Vom Brocke J., Hevner A., Maedche A. (2020), "Introduction to Design Science Research". In: vom Brocke J.,
Hevner A., Maedche A. (eds) Design Science Research. Cases, Cham. https://doi.org/10.1007/978-3-030-
46781-4_1.
[10]
J. Pries-Heje, R. Baskerville, and J.R. Venable, "Strategies for Design Science Research Evaluation". In:
Proceedings of the 16th European Conference on Information Systems (ECIS 2008), Galway, Ireland, 2008.
[11]
Villamor, E., "SAFETY4RAILS Deliverable D8.2", January 2021.
[12]
"SAFETY4RAILS Grant Agreement, version 1.0", 2020.
[13]
Siebold U., Crabbe S.," SAFETY4RAILS Deliverable D1.4", March 2021.
[14]
Bonneau M., "SAFETY4RAILS Deliverable D8.1", December 2021.
[15]
Chai, C. S., & Der-Thanq, V. C. (2004). "A review on usability evaluation methods for instructional
multimedia: an analytical framework. International Journal of Instructional Media", 31(3), 229.
ResearchGate has not been able to resolve any citations for this publication.
Chapter
Full-text available
Design Science Research (DSR) is a problem-solving paradigm that seeks to enhance human knowledge via the creation of innovative artifacts. Simply stated, DSR seeks to enhance technology and science knowledge bases via the creation of innovative artifacts that solve problems and improve the environment in which they are instantiated. The results of DSR include both the newly designed artifacts and design knowledge (DK) that provides a fuller understanding via design theories of why the artifacts enhance (or, disrupt) the relevant application contexts. The goal of this introduction chapter is to provide a brief survey of DSR concepts for better understanding of the following chapters that present DSR case studies.
Article
One common objective of instructional multimedia products is enhancement of learning. It follows that one of the primary tasks of the interface designer is the creation of a user-interface that is motivating, interactive, intuitive and at the same time imposing as minimal as possible a cognitive load on the learner. To achieve this goal, qualities such as learnability, efficiency, memorability, low error rates and high user satisfaction are essential. A well-designed interface should inform the user what do the objects on the screen stand for and what they might do and allows the user to break down a task into subtasks and map them on to the system's functions easily and naturally. By doing so, it enables the user to build a functional mental model of the system that does not require user's rote memorization. These qualities of a well-designed interface are commonly known as the "usability" of software. To date, many methods that allow systematic ways of evaluating usability in instructional multimedia have been developed. By integrating these methods into the various stages within the life cycle of instructional multimedia design and development, the usability of instructional multimedia, and hence its educational value, could be assured. However, there has been little attempt to consolidate these methods into a more integrative and self-explanatory model. This paper attempts to review these various methods under a proposed framework in the hope that it may help readers to select the most appropriate usability evaluation method according to their needs.
Conference Paper
Seminal works in the application of design science research (DSR) in IS emphasize the importance of evaluation. However, discussion of evaluation activities and methods is limited and typically assumes an ex post perspective, in which evaluation occurs after the construction of an IS artifact. Such perspectives can assume that the evaluation is an empirical process and its methods can be selected in the same way as empirical research methods. In this paper, we analyze a broader range of evaluation strategies, which includes ex ante (prior to artifact construction) evaluation. This broader view is developed as a strategic DSR evaluation framework, which expands evaluation choices for IS DSR researchers, and also adds emphasis to strategies for evaluating design processes in addition to design products, using well-known quality criteria as an important asset. The framework encompasses both ex ante and ex post orientations as well as naturalistic settings (e.g., case studies) and artificial settings (e.g., lab experiments) for DSR evaluation. The framework proposed offers a strategic view of DSR evaluation that is useful in analyzing published studies, and also in surfacing the evaluation opportunities that present themselves to IS DSR researchers.
SAFETY4RAILS D1.1 Project Management Manual
  • S Crabbe
Crabbe S., "SAFETY4RAILS D1.1 Project Management Manual, V1.0", p. 9., October 2020.