Available via license: CC BY
Content may be subject to copyright.
Safety differently: A case study in an Aviation
Maintenance-Repair-Overhaul facility
Anastasios Plioutsias1,
*
, Konstantinos Stamoulis 2, Maria Papanikou2, and Robert J. de Boer3
1Coventry University, School of Mechanical, Aerospace and Automotive Engineering, Faculty of
Engineering, Environment and Computing, UK
2Amsterdam University of Applied Sciences, Aviation Academy, Faculty of Technology, NL
3Northumbria University, Amsterdam Campus, NL
Abstract. You This paper presents the findings from a ‘Safety
Differently’ (SD) case study in aviation, and specifically in a
maintenance, repair and overhaul (MRO) organisation in Southeast
Asia. The goal of the case study was to apply a new method of safety
intervention that is part of the Safety Differently toolkit and utilises
a bottom-up approach. This research tested the extent to which these
interventions could be embedded into a continuous improvement
program in a highly controlled environment, namely an Aviation
MRO. The interventions (called micro-experiments, ME) are
considered as a flexible tool, which allows testing of process
improvements in a safe to fail way, empowering the lower levels of
the organisation, challenging safety related issues and revealing key
areas in need of transformation. The ideas for the interventions
considered in the case study were retrieved from interviews
conducted with 50 mechanics, and include issues to address aviation
safety and occupational health as well as quality. We elected to
include all three categories in this study as the ME approach is
applicable to all of these. This MRO case study showcases the
benefits and limitations of the ME in aviation, revealing the
conditions under which it may become useful. Future studies should
further explore the role of complex and heavily controlled industries
in similar bottom up approaches, so that interventions can become
part of a continuous improvement plan.
1 Introduction
Safety Differently is a new, yet under conceptualised, approach to improving safety processes
by using the experience, knowledge and creativity of the frontline employees [1]. As part of
the SD toolkit, micro-experiments (ME) have been introduced as a way to test the feasibility
*
Corresponding author: ad3903@coventry.ac.uk
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons
Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/).
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
of interventions in a safe-to-fail manner. Maintaining aircraft to the highest safety standards
requires skills of the mechanics within a Maintenance, Repair and Overhaul (MRO)
organisation. Aircraft mechanics perform daily tasks, in demanding working conditions and
whilst following procedures, designed to comply with safety and quality standards and
regulations. Hence, given their subject-matter expertise, MRO personnel is a valuable source
for the detection of issues experienced at the shop floor. Since it is important to continuously
improve processes, this paper focuses on interventions utilising the most valuable source of
an MRO organisation, its workforce. The MRO mechanics are key in sharing problems from
the shop floor and help locate the potential interventions required to address issues affecting
safety. Little is known about the results of the ME approaches and the added value to the
continuous improvement of safety. Following the principles of Safety Differently (SD), we,
therefore, explored bottom-up improvements through testing of the ME-based interventions.
Our motivation stemmed from the lack of applications in highly controlled and complex risk
industries, such as aviation. Aviation is an industry characterised by complexity, high
severity risks, and consequently a strong regulatory framework. Any such ME case study or
application in aviation is missing.
It was a challenge to examine the possibility of the ME application in an aviation
organisation because the study must deal with strict regulations, whereas the methodology
requires the elimination of constraints, possibly violating aviation rules. Our approach aimed
to assess the benefits of ME in high risk industries and safety critical tasks [2] in two phases.
The first phase involved data collection to a) identify gaps in the safety of the work processes
and b) to design the subsequent potential interventions. The qualitative data was collected
through a change-oriented research design, achieved by interviewing 50 mechanics who were
the primary source according to the bottom-up approach of ME [3]. The second phase was
the experimentation of a limited number of appropriate interventions, in line with previous
applications of the concept outside aviation. The interventions were designed for their small-
scale application by using the organisation’s active work area and employees as an in-situ
laboratory [4]. The project began in the base of the organisation with two university
representatives who stayed in the MRO’s base for months. The university agreed with the
organisation about the project constraints, described in detail in 2.3, and the working progress
of the project in the organisation’s area and bays. In the following sections, we report the
extent to which the SD-based methodology through the execution of ME could be used as
part of a continuous improvement program within MRO organisations.
2 Safety Differently: concept and methodology
Sidney Dekker defines the Safety Differently (SD) concept as following: “Imagine a space,
a space in which there are no rules, a space in which people can themselves determine the
best course of action, a space in which people spontaneously negotiate and collaborate in
order to create the safest outcomes for everyone, in this space a new type of humanity
emerges, nobody is telling them, they figure it out on the spot, we call this emergent
behaviour” [5]. In a complex environment, where behaviour is difficult to predict, we cannot
be sure that the interventions that we devise actually bring the results that we envisage, or
that unintended effects are avoided. We need an approach that allows us to try different
things, does not cause peril, and can be stopped if we don’t get the results that we want. This
is where the micro-experiments (ME) come in.
In an MRO organisation, ME can be applied if the organisational safety constraints are
removed in order to accommodate the bottom-up research design, examining how the
mechanics have to maintain safety in their tasks and operations. Typical constraints in an
MRO setting mean that wide-scale interventions are not suitable, mainly due to the number
of stakeholders involved, such as airlines and regulatory bodies. An intervention can act as a
2
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
safe-to-fail, small scale project, which uses the organisation’s workplaces and workforce [6].
The subsections that follow discuss how the micro experiments are selected and how they
were applied in the MRO organisation.
2.1 The ME concept
Captions Traditionally, aviation safety management is considered as a hard, positivist field
of research [7]. Safety Differently involves taking a more collaborative approach to safety. It
involves getting the people who do the work to decide on rules and safe working practices.
This is different from the traditional approach to safety where rules and policies are set by
managers and other third parties before being communicated to (or imposed on) the people
doing the job. ME fit into this approach as they are safe-to-fail interventions that are
monitored for success or failure, and that have been generated by the work-floor. ME are
different to ad-hoc trial-and-error interventions in that they are planned before they are
executed, and carefully monitored to see whether the intended results are realised. ME are
different to projects in that projects are not supposed to fail, whereas in micro-experiments
are temporary and we allow for unexpected, even disappointing results. In a complex
environment we cannot have the (misplaced) confidence that any intervention will surely
deliver the predefined results as expected. We need to allow for the possibility for the
intervention not to work without causing undue damage. In fact the experiment can be
considered a success if we realize an intervention “fails” and that some other solution is
required. We therefore need to be able to stop or slow the experiments. On the other hand, if
all goes well the intervention is ready to be formally implemented and possibly enlarged on
a wider (geographic or organisational) scale. Naturally we will create the ideas for the micro-
experiments in close collaboration with the task executors; many ideas will derive from them.
Micro-experiments generally require temporary safeguards because we will be deviating
from existing processes, perhaps violating the constraints that are usually in place. Care needs
to be taken to identify and only maintain those constraints that are absolutely vital to
safeguard against unacceptable risks – we value the input of safety experts here. The criteria
by which we determine whether an intervention is successful (or not) need to be determined
ahead of time, although additional insights will occur as experience is gathered. Interventions
that are deemed successful need to be incorporated in the next version of Work-as-Imagined.
Even after formalisation, we remain vigilant about unintended effects, expecting the front
line to report where exceptions are necessary. Multiple micro-experiments can run in parallel,
preferably somewhat isolated from each other. Good targets for micro-experiments are
procedures that are not delivering results as expected. Many organizations that we first
engage with are susceptible to large and potentially embarrassing gaps between Work-as-
Done and Work-as-Imagined, that are not straightforward to close. This is often a good
starting point to try out micro-experiments. Other reasons to execute micro-experiments
include adapting rules to changes in processes or equipment or for innovation. To further
define the problem, we first need to identify the “job to be done”: what operators really seeks
to accomplish in the given circumstances. Successful micro-experiments help make the job
safer and easier.
2.2 Methodology
Our methodology involved a sequential research design, comprising of two data collection
phases. In the sections that follow, we initially describe the data collections phases for ME.
Following this, we showcase the utilisation of previous findings in our data analysis. Before
we present the results of the ME application, we describe the pre-coded themes and the
analysis framework. Lastly, we discuss the limitations of these choices.
3
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
2.2.1 Research design phases
As outlined previously (cf. Section 1), the first phase involved the collection of data and the
second phase regarded the interventions. The first phase included identifying and
interviewing mechanics in semi-structured interviews. In order to identify the interviewees,
the team requested to be present at the organisation’s bays and observe the mechanics.
Through this process, fifty (50) mechanics were purposively sampled and were interviewed
individually. The main objective of this phase was to identify what gaps are prevalent
between WAD and WAI. This data collection phase was supplemented with archival data
from the MRO, including documentation from audits, investigations reports and safety
reports, as well as injury reports and maintenance error notifications. Following this phase,
the team identified a number of interventions, which formed the second phase of the study.
The aim of the interventions as interventions was to close the gap(s) found through the
findings from the first phase. During this phase, it was decided that the term interventions
was changed to micro-projects following the organisation’s requirement and in order to limit
possible defensive behaviours that would affect results.
2.2.2 Coding and analysis framework
MRO organisations are typically optimised by lean approaches in order to create more value
for their customers by utilising fewer resources [8]. Our approach hence initially required an
assessment of the MRO environment. Before our study commenced, the quality (and safety)
department at the MRO, had initiated a safety project to assess their safety performance.
Elements of the latter are observed in the organisation’s vision for the safety project itself,
striving for innovation and evolution, and finally, Genchi Genbutsu, going down to the shop
floor to fact check one’s ideas in the ‘real world’ [8]. The findings of the project led to the
establishment of a Safety Group (SG), encompassing the following pillars:
• Technical Systems: includes all the tools, equipment, and all the physical items in the
organisation, necessary to create value for the customer.
• Management Infrastructure: includes the formal structures, the processes and the systems
for controlling the resources, in association with the technical systems.
• Mindset and Behaviour: include the employees’ thoughts and feelings as well as
individual and group behaviour at the workplace.
These three pillars formed our initial classification that we used to compare the SD results
with the organisation’s SG project, and hence led to a pre-coding of the issues found at the
shop floor. All the sources used from the organisation’s safety project was provided to the
team by the SG. The coded shops, which were chosen for phase one observations and
consequent sampling and data collection (Table 1).
Table 1. Shop coding.
Shop
Code
Avionics shop
AVI
Structures shop
STR
Cabin shop
CAB
Cargo shop
CRG
Flight control shop
FLC
Landing gear shop
LDG
Engine shop
ENG
Seat shop
SS
Paint shop
PS
4
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
In the second phase, the researchers collected data, mainly from the interviews in which
the front-line mechanics assigned. These technicians answered the questions posed to them
and communicated their ideas for possible interventions. The findings from the interviews
and observations led us to a master coding according to shop floor and their bays. In addition,
the MRO’s project used numerical values to score and classify findings following the Safety
Risk Management (SRM) manual, which meets the compliance requirements according to
ICAO’s Safety Management Manual (SMM) and the data driven systems to proactively
manage safety, Safety Management Systems (SMS). The MRO uses numerical values for the
risk and taxonomies according to SMM standards [9], based on the Severity (minor to
hazardous) and the Probability (extremely improbable to frequent). The Risk Indicator (R is
calculated by multiplying the probability (P) with the Severity (S) of an occurrence (S),
forming the R = P x S equation. The result of this equation rated from low to high risk and
the values for the possibility and the severity explained detailed. The SRM was used during
the project to score and classify the findings.
2.2.3 Research rigour
The team took a series of steps to minimise bias, gain access, and to overall ensure rigour in
the research process. The mechanics were interviewed to gain information based on their
expertise and bypass the organisational-level hurdles. By doing so, the feeling of
empowerment of the mechanics increased, placing emphasis on the value of the technical and
operational experience of the mechanics. In addition, we described the interventions to the
organisation management to gain internal approval, allow execution of the interventions and
to peer-review the results. Following this, and regarding the micro-projects, acceptance by
management was ensured to reach consensus on the interventions proposed by the mechanics.
After suitable interventions were collected, the gathered intel has to be verified by the rest of
the mechanics, since implementing an idea that only suits one mechanic is not a valid micro-
experiment. Besides validating with the mechanics, it is also important to validate the
opportunities of improvement with the quality management representatives to make sure they
agree on the findings and see the same promising interventions for current tasks or processes.
In case there were multiple possible interventions, the mechanics scored them on an effort-
effect diagram (Figure 1) together with the Quality Manager. All the proposed interventions
were compared with each other in assuming the effort required to achieve each intervention
and its expected effectiveness. An example is shown in Figure 1, where intervention “1” was
preferred because it could yield the most effective result with the least amount of effort when
compared to intervention “2”. In general, interventions mapped on the top-right area of the
matrix were preferred over the rest.
Figure. 1. Effect - Effort Matrix [10]
5
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
To gain further approval, the micro-experiment approach was described to management
in detail, covering points such as the process, success criteria, constraints and anticipated
interventions as well as the area and possibilities of amplification. Regarding the execution
of the interventions, each micro-experiment had a customised period. The main factor was
the duration of the task each intervention targeted. One researcher was continuously present
during the execution of each experiment to observe the intervention, function as the point of
contact for concerns, questions, ideas, and, most importantly, collect the views of the
mechanics on each intervention. When the predetermined timeframe expired, the experiment
was concluded by the researcher, the intervention was suspended, and the task was brought
to its previous state (e.g. tools, procedures). A final template was created with the approval
intervention when it validated from the quality management department. The decision on
whether to amplify each successful intervention into an improvement plan laid with the
organisation’s management team.
2.3 Constraints
When the project was being planned, and following the principles of ME, the team observed
that there are several limitations when applying ME in an aviation organisation. The chosen
MRO organisation allowed the researchers time to organise and plan the project locally, but
the quality manager and the safety group required the following constraints to be applied to
the ME’s:
• Any actions must be according to the regulations
• Any intervention must be according the procedures
• Any interaction cannot create new risks
• Any intervention is according to the organisation’s risk assessment
• Any intervention is not a permanent solution
• Any intervention must comply with requirements set by organisation’s clients
• Any intervention is only applied to the predetermined bay with predetermined group
• Any intervention will run for fixed period of time
• Any intervention can only be executed with the present of at least one of the
university’s representative on site.
• Any intervention can only be executed in agreement with Quality Manager, VP, the bay
manager involved and the project team.
The limitations deriving from the application of ME in a highly controlled industry
quickly became the first important finding of the case study. It is clear that the first three
constraints (any actions must be according to the regulations, any intervention must be
according the procedures, and any interaction cannot create new risks) do not allow any
innovation to take place. These, as well as the required permission to execute the ME’s,
severely impeded progress at the site, and needed executive circumvention to allow the ME’s
to be applied. Although in the ME concept the interventions are described as a bottom up
process, the validation, approval and decision processes in aviation are a top down process
with assumptions and issues including not merely safety aspects but also extending to other
departments like security and quality. The discussion about the limitations follows the
presentation of our results.
6
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
3 Results
This section presents the findings from phases one and two of the sequential research design
applied following the principles of the ME concept. Table 2 below provides an overview of
the amount of data and of the sources involved in phase one of the study (i.e. interviews and
archival data). Interview findings were initially categorised in single or multiple concerns.
The categories emerged from the narratives of the research as presented in Table 2.
Table 2. Number of times that concern is raised and/or observed.
Concerns
Frequency
Number of findings
Single Concern
One time
18
Few Concerns
2 – 5 times
25
Multiple Concerns
6 – 10 times
8
A Lot of Concerns
>10 times
5
When a finding was a single concern, it meant one mechanic only raised it; such issues
were either considered minimal, according to the QM and the SG, as general remarks or seen
as irrelevant to the organisation. Valid concerns were findings reported by multiple
mechanics, and issues on which the organisation agreed, but had its intervention planned,
processed or implemented (11 in total). In addition, findings were categorised according to
the bay coding (cf. Section 2.2.2) as presented in Table 3. As shown below, the findings were
also categorised according to in- and on- the aircraft (in A/C, on A/C), and were summarised
for each of the shops. Bay 6 is not included in the study.
Table 3. Numbers of findings per bay.
In A/C
On A/C
CRG
CAB
STR
LDG
ENG
FLC
AVI
PAINT
Other
Bay 1
7
3
3
1
2
4
Bay 2
1
1
6
Bay 3
1
6
Bay 4
1
11
6
Bay 5
3
12
10
2
4
Bay 7
1
Bay 8
1
2
1
The shops that are not necessarily linked to all bays. For example, the landing gear shop
(LDG), is linked only with the bay 5 and bay 8. In total, sixty-nine findings (69) were
recorded. However, ten of these findings were finally selected as interventions, through the
validation and approval process as described previously (cf. Section 2). The accepted findings
as interventions are described in detail in 3.2.
3.1 WAD – WAI analysis
During the analysis of the interview data, it was observed that the mechanics were hiding the
gaps between how they should do their job (WAI) versus the way they perform their tasks
(WAD). Hence, we observed that the MRO organisation manifests signs of a problematic
safety climate. These limitations are further discussed in Section 4.
7
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
3.1.1 Findings Characteristics
The findings initially examined and evaluated the possible interventions for the MRO
organisation. As shown in Table 4, the intervention characteristics were classified using the
three pillars of the SG, then by area of improvement and per type of method and lastly by
using the MRO’s risk assessment scaling method.
Table 4. Evaluation of Interventions
Classification of Findings
1
Technical Systems
34
2
Management Infrastructure
21
3
Mindset and Behaviour
14
Type of method
1
Process
50
2
WAD & WAI
15
3
Researcher’s Observation
4
Risk Assessment of Findings
1
High (>40)
3
2
Medium High (30-39)
8
3
Medium (7-29)
30
4
Low (<5)
28
In detail, in the risk assessment it is observed that eleven (11) findings are in medium-
high and high risk and they have been rejected. The rest fifty-eight findings were examined
and evaluated from the project team, the quality manager and the organisation’s safety group.
From the suggested area of improvement, the twenty four of the sixty-nine findings belong
to safety area. The rest thirty-nine addressed to faciliatory and twenty-four are related with
the equipment. Fifty of the total findings are related with the process, fifteen of them are with
the WAD & WAI concept and four of them are related with the researcher’s observations.
When the team taxonomized with the same taxonomy of the previous safety group and QM
project they found thirty-four (34) findings in technical systems (48% of the findings),
twenty-one in management infrastructure (31% of the findings) and fourteen in mindset and
behaviour category (21% of the findings).
3.2 The accepted interventions
Due to the absence of findings on safety alone, most of the processes discussed were not
strictly safety-oriented but also overlapped with quality and occupational health domains.
Although mechanics were hesitant in discussing these gaps, they were eager to suggest
possible improvements and ideas. Similar to the problems of the WAD-WAI analysis, this
process manifested that trust from the mechanics was low, and that, then, there is a
problematic (safety) climate. During the process, and as the role of the researcher was
distinguished from organisation management, trust increased, and the mechanics opened up,
sharing the insights that follow in the ten (10) executed interventions, which have more
quality characteristics than safety related issues.
8
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
Table 5. The accepted interventions
Problem description
Intervention description
The mechanics are not allowed to wear their
safety shoes during final maintenance in a
cleaned cabin, prepared for delivery to the
customer. However, mechanics often make
last-minute repairs in the cabin, in which
the use of safety shoes is recommended.
Three types of covers were tested and a
silicon, flexible shoe cover was selected
due to ease of use, grip provided and anti-
static properties. The experiment proved
useful and QM extended the shoes
throughout the organisation by adding them
to the personal protection equipment (PPE).
Following installation the seat covers are
often damaged and repairing them usually
means delays in customer acceptance and is
costly.
For the business class seat covers, the
intervention is a fabric, reusable seat cover.
For small tools, such as a drill bit, a
expendables form needs to be filled in at the
tool room booth, making the tool keeper
wait for the completion of the form.
The filling in of the form elsewhere can
increase the throughput of the booth and
therefore reduce the current queue time.
Under the wing of Airbus aircrafts, a strong
presence of kerosene fumes can be noticed.
Fume-removing fans could be used to
circulate the air. A risk, however, is that the
fumes move to other work areas.
Current methods of working on the fuselage
or wings of the aircraft can be dangerous to
mechanics and can damage the aircraft.
Butterfly Suction Cups: A new system, in
the shape of a butterfly, was proposed as an
improved and safe alternative.
The safety shoes at are currently used by
mechanics are uncomfortable and are of
low quality.
The introduction of a new type/brand of
safety shoes has been proposed.
The extreme hot climate working
conditions in Southeast Asia.
To address the tropical working conditions,
lighter shirts made from a dry-fit fabric,
have been proposed for the mechanics.
When a tool is missing from one toolroom
mechanics often walk to the other toolroom
a few 100 meters away with time-
consuming.
A transportation means such as a bicycle
can significantly reduce the time needed to
get a tool from a different tool room.
Everyday mechanics have to retrieve their
standard toolboxes from the toolroom,
which creates long queues at the start and
finish of the shifts.
The Relocation of the toolboxes back in the
shop area was suggested, and only using the
toolroom as for original and limited
equipment.
Inside the cabin, mechanics use stairs made
from foam and tape, these are unstable.
A more firm and secure stair was proposed
as a safer equipment for the mechanics.
9
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
4 Discussion and Recommendations
The SD principles guided the application of the Micro Experiments in a bottom up process
to identify the problems and the solutions for safety related issues. The research objectives
focused on exploring whether the application of the ME approach could trigger interventions
for improved safety processes in an aviation organisation. Our two- phases methodology
following the principles of the ME in an aviation organisation, revealed the benefits and the
limitations of the approach in a high controlled and operationally risky environment. Below
we discuss the key findings from the ME case study at the MRO organisation noting our
contribution.
4.1 Occupational Health, Process Safety and Quality overlaps
One of the critical realisations during the project was the relationship between safety and
quality management. Through the research, the team realised that the constraints and the
limitations in an aviation organisation are strong barriers for the ME application as described
and applied elsewhere. The lack of case studies in aviation meant that the research would
enter an under-explored field, which created challenges and sometimes misunderstandings
between the researchers and the management of the organisation. We therefore report that
the ME assumptions are challenged by the organisation’s quality and safety department,
which descriptions of ME do not account for. For example, the MRO’s quality department
attempted to assess the safety findings and observations as quality markers. Our findings here
highlight the problems in the quality vs safety and occupational health misperceptions, and
how these create confusion in the safety management process and the identification of
suitable interventions for the organisation. Secondly, the safety department initially raised
constraints that paralysed the ME and needed to be lifted by top-level intervention for
progress to be made
Albeit, the revelation shows that the ME approach can reveal deeper organisational issues
intervening in the safety processes, whether managed or performed, and supports the benefits
deriving from bottom-up approaches [3]. Because of these benefits, mid-project interventions
were able to take place. Initially, the team was able to recognise the lack of ME fundamental
knowledge and thinking in the MRO organisation. Secondly, the team was able to address
this issue by filling the knowledge gap with ME workshops and additional visits on-site the
MRO. The intermediate actions aided to have most of the interventions accepted from the
organisation, yet, as reported in Section 3, with quality and occupational health aspects, and
not a clear safety orientation. The same misconception was identified in the interviews where
the participants wanted to discuss and proposed quality and not safety interventions like the
cover of the cabin seats during the work of the technicians in the aircrafts’ cabin. This finding
and change process underpins the added value of ME and the intervention possibilities in
aviation organisations.
4.2 Insights following the SD principles
Using the ME approach to inquiry, the findings from interviews in interventions achieved a
variety of results and insights into the adoption of the boader SD principles. Concerns such
as middle management isolation, fear of change, and not wanting to take responsibility,
revealed problems of safety bureaucracy and showcased the possible issues these can induce
during a (research) project and in the safety processes of an MRO organisation without
directly impacting the ME. The mechanics also reported that there are even cases in which
they were demoted after they made critical remarks about the organisation and/or shared
innovative ideas. There are clear indicators of the presence of these types of bureaucracies
10
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
within the organisation. Our ME findings hence support studies highlighting the importance
of safety climate underpinning aviation MRO practice [11]. Importantly, our study showed
that, when the organisation shows value and appreciation towards the work of the mechanics,
their opinions and ideas, the attitude towards responsibility changed. Specifically, the MRO
mechanics were appreciative of the responsibilities and chances are given to them to be part
of a change-based project. It showed them the importance of bottom-up suggestions and
challenging of organisation procedures. The interventions provided the flexibility required to
fit and try a wide selection of interventions. The first completed interventions showed the
amount of effort mechanics put in their ideas and the results they can achieve from this input.
The fact that the shoe cover intervention (Table 5) will be amplified organisation wide is a
first step towards the continuous improvement.
4.3 Evaluation of the MRO project
During the planning and execution phases of the project, several success factors and
limitations were revealed. Success factors include the MRO’s availability and willingness at
a high level to run the project in their working area. The decisions the team took in the latter
environment, aided in achieving research rigour and insights for the transferability of the
results [12]. In detail, the team followed the data to make decisions instead of aiming to
satisfy standards not relevant and not appropriate for the project (i.e. external and internal
validity, reliability and objectivity). In addition, the researchers’ professional background in
aviation organisations helped establish a common communication channel with the MRO
managers, gaining access into their MRO systems. Hence, familiarity with the culture of the
participants aided reflexivity [13]. As a bottom-up process underpinned the project, the
developing cooperation between managers and mechanics became evident. This success
factor showed the power of the ME approach and the advantages over traditional intervention
approaches (ad-hoc or projects).
Limitations of the project include the chosen context of the case study, hence addressing
our main objective. In detail, in the highly controlled MRO environment, the more
inexperienced members were not able to recognise possible safety-related issues, which
would have been useful for the project process. Moreover, the inexperienced members
unintentionally behaved like auditors, adding to a lack of trust already in the MRO, and
revealing problems of safety climate. Similarly, the managers were unprepared for this type
of project. Our findings hence support past aviation MRO safety studies, where safety climate
is seminal to developments. Additional limitations were the context of safety bureaucracy,
which surfaced organisational problems such as the lack of a positive safety culture, the lack
of trust and the lack of safety knowledge in the middle and the low-level employees. For
example, during interviews, it became evident that most employees are confused between the
failure analysis and incident investigation.
5 Conclusion and future directions
Safety Differently in aviation organisations is an underexplored field. Micro experiments are
not safety recommendations, but they comprise a tool to explore the perceptions of shop-
floor employees that guide actions and behaviours. In order to formulate interventions,
incident reports can be used, although it should be noted that incident reports do not trigger
them. Incident reports can be used as a tool to identify focus areas, assess the status of the
safety recommendations and close the loop. In addition, it is recommended that, although the
micro experiment is a bottom-up process, the organisation is assessed before any safety
differently project can begin. Our case study application revealed the bottle neck situation
11
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
which forms our contribution to further ME applications in a similar aviation organisation.
Our recommendation is hence to initially assess the organisation’s safety climate and then to
apply the framework for the Safety differently project. The assessment of safety climate is
fundamental for the SD – and therefore ME - implementation in aviation. It is a way to plan
the researcher's actions before the application of any method and tool of SD concept. The
preparation of the safety program participants is an additional necessary action, which
includes top-down training and identification of the setting the micro experiment takes place,
so that daily working load is not affected. They are no potential risks for the organisation’s
operations. Moreover, it is recommended that the organisation prioritises the organisation’s
problems, using quality tools and methods (e.g. Effect – Effort, Pareto, Fishbone diagrams).
In any aviation organisation which has many working tasks, the term ‘micro experiments’
is not a familiar one and could, therefore, threaten their acceptance. It is therefore suggested
that the term changes to ‘micro-projects interventions’, which is a less threatening term to
aviators. Future research should hence focus on tactics for the smoother application of the
micro-project interventions can give opportunities in organisations to prepare, plan and
organise better large-scale projects in the future. With the use of interviews at the shop floor,
new findings and interventions can be collected, executed and possibly amplified throughout
the organisation. However, mechanics will have to be empowered to share their concerns and
ideas openly. Addressing safety bureaucracy problems will not only transform the
intervention to an improvement plan, but it can help enhance wider safety practices (e.g.
safety promotion, reporting). Safety Differently and ME can, therefore, be used to
continuously improve following a bottom-up approach.
Acknowledgements
The authors would like to thank Juno Beckers and Niek Kuilder, for their contributions.
References
1. Dekker, S. (2015). Safety, Differently. Boca Raton: CRC Press.
2. Weber, D. E., & Dekker, S. W. (2017). Assessing the sharp end: reflections on pilot
performance assessment in the light of Safety Differently. Theoretical issues in
ergonomics science, 18(1), 59-78.
3. Hale, A., Borys, D., & Else, D. (2016). Management of safety rules and procedures: a
review of the literature. Report submitted to the IOSH Research Committee, Report 12.3.
4. Gantt, R. (2018). Safety Differently- A new view of safety excellence. Available online
at https://safetydifferently.com/wp-content/uploads/2018/10/Safety-Differently-ASSE-
Proceedings-Paper.pdf. In Dekker, S. (2018). A micro-experiment. In S. Dekker, The
Safety Anarchist (pp. 180- 185). Oxford: Routledge.
5. Dekker, S. W. A. (2017). Safety Differently, The Movie. Available online at
https://safetydifferently.com/safety-differently-the-movie/.
6. Dekker, S. (2018). A micro-experiment. In S. Dekker, The Safety Anarchist. Oxford:
Routledge.
7. Ferroff, C. V., Mavin, T. J., Bates, B. R., & Murray, P. S. (2012). A case for social
constructionism in aviation safety and human performance research. Aeronautica, 2(1),
pp. 1-12.
8. Nanova, G., Dimitrov, L., & Neshkov, T. (2012). Lean Manufacturing Approach in
Aircraft Maintenance Repair and Overhaul. Sofia: Technical University of Sofia,
Bulgaria.
12
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
9. ICAO (2013). Safety Management Manual. Available online at
https://www.icao.int/safety/fsix/Library/DOC_9859_FULL_EN.pdf.
10. Beckers, J. (2019). Process Improvements with Safety Differently. Amsterdam University
of Applied Sciences.
11. Fogarty, G. J. & Shaw, A. (2010). Safety climate and the theory of planned behaviour:
towards the prediction of unsafe behaviour. Accident Prevention and Analysis, 40(5), pp.
1455-1459.
12. Guba, E. G. & Lincoln, Y. S. (1981). Effective evaluation: Improving the usefulness of
evaluation results through responsive and naturalistic approaches. Jossey-Bass.
13. Shenton, A. K. (2004) “Strategies for ensuring trustworthiness in qualitative research
projects”, Education for information, 22(2), pp. 63-75.
13
MATEC Web of Conferences 314, 01002 (2020) https://doi.org/10.1051/matecconf/202031401002
ICSC-ISATECH 2019
