Content uploaded by Kim Phuc TRAN
Author content
All content in this area was uploaded by Kim Phuc TRAN on Jul 21, 2022
Content may be subject to copyright.
1
Detection of Poisoning Attacks with Anomaly
Detection in Federated Learning for Healthcare
Applications: A Machine Learning Approach
Ali Raza, Shujun Li, Senior Member, IEEE, Kim-Phuc Tran, and Ludovic Koehl
Abstract—The application of Federated Learning (FL) is steadily increasing, especially in privacy-aware applications, such as
healthcare. However, its applications have been limited by security concerns due to various adversarial attacks, such as poisoning
attacks (model and data poisoning). Such attacks attempt to poison the local models and data to manipulate the global models in order
to obtain undue benefits and malicious use. Traditional methods of data auditing to mitigate poisoning attacks find their limited
applications in FL because the edge devices never share their raw data directly due to privacy concerns, and are globally distributed
with no insight into their training data. Thereafter, it is challenging to develop appropriate strategies to address such attacks and
minimize their impact on the global model in federated learning. In order to address such challenges in FL, we proposed a novel
framework to detect poisoning attacks using deep neural networks and support vector machines, in the form of anomaly without
acquiring any direct access or information about the underlying training data of local edge devices. We illustrate and evaluate the
proposed framework using different state of art poisoning attacks for two different healthcare applications: Electrocardiograph
classification and human activity recognition. Our experimental analysis shows that the proposed method can efficiently detect
poisoning attacks and can remove the identified poisoned updated from the global aggregation. Thereafter can increase the
performance of the federated global.
Index Terms—Federated learning, security, privacy, anomaly detection, poisoning attacks, data poisoning, model poisoning, Byzantine
attacks, healthcare, ECG, HAR,.
F
1 INTRODUCTION
PRIVACY and security are among the top issues to be
addressed in privacy and security-sensitive applications
of machine learning (ML), such as healthcare, autonomous
vehicles, etc. Federated learning (FL) has been introduced to
enhance data privacy in machine learning applications [1].
FL attempts to provide enhanced privacy to the data owners
by collaboratively training a joint model by only sharing
parameters of locally trained models, in this way the data
owners never share raw data, but can collaboratively train a
joint robust global model. For instance, various hospitals
can train ML models jointly for healthcare applications
without directly sharing their privacy-sensitive data with
each other. Generally speaking, FL iterates in three steps:
the global server (i.e., a cloud server), which maintains the
global model, sends the global model to the edge devices;
the edge devices update the local models using their local
training data and shares the trained parameters of the
locally trained model with the global server, and the global
server updates the global model by incorporating the shared
parameters according to an aggregation rule. For example,
•A. Raza is with the University of Lille, ENSAIT, ULR 2461 - GEMTEX
- Génie et Matériaux Textiles, F-59000 Lille, France and School of
Computing & Institute of Cyber Security for Society (iCSS), University
of Kent, UK.
E-mail: ali.raza@ensait.fr
•K.P. Tran and L. Koehl are with the University of Lille, ENSAIT, ULR
2461 - GEMTEX - Génie et Matériaux Textiles, F-59000 Lille, France.
•S. Li is with School of Computing & Institute of Cyber Security for Society
(iCSS), University of Kent, UK.
Manuscript received xx xx, xx.
the mean aggregation rule which computes the average of
the shared local model’s parameters is one of the widely
used aggregation algorithm [2]. Nevertheless, in such cases,
the global model can be easily manipulated, even if a single
edge device is compromised [3, 4, 5]. The attack surface of
FL is growing due to its distributed nature. For example,
malicious peers can launch model poisoning [6], and data
poisoning [7, 8] attacks to impair the performance of the
updated global model.
FL can be divided into three phases: data and behaviour
auditing, training, and testing. FL faces different kinds of
security threats in each phase [9]. Hence, establishing secure
FL needs to take effective measures at each phase to mitigate
such threats. A solution before integrating a local model
into the global model is to audit the local data. However,
due to the privacy concerns and architecture of FL, it’s
challenging to achieve such audits [9]. A trivial method to
address model poisoning attacks could be using accuracy,
i.e., using accuracy as a measure to access the quality of
data being used to train the local model. Nevertheless, such
methods can not be generalised as accuracy solely cannot
reveal information about the underlying data. Just looking
at the accuracy it cannot be claimed that the model is trained
on benign or malicious data. Furthermore, models can be
designed to have high accuracy for the testing samples
by including them in the training dataset. A model can
have low accuracy even if it has been trained on benign
data depending on the amount of training data, training
epochs, hyper-parameters tuning, optimization, etc. Hence,
new solutions are required to avoid such model and data
arXiv:2207.08486v1 [cs.LG] 18 Jul 2022
2
poisoning attacks. Methods should be developed to verify
that the shared local model gradients are not trained on
anomalous data (noisy, featured poisoning, label poisoning
etc.). In other words, malicious behaviour of the locally
trained models should be tracked before incorporating them
into the global aggregation in order to avoid malicious
peers compromising and manipulating the global model.
In order to address poisoning attacks [10, 11] in which a
malicious edged device manipulates its local training data
or the model after training it on benign data using care-
fully designed techniques to malfunction the performance
of global model, we proposed a novel poisoning attacks
detection framework in federated learning for healthcare ap-
plications. Furthermore, we analyze the performance of the
proposed framework using two healthcare applications i.e.,
electrocardiogram (ECG) classification and human activity
recognition (HAR) in a federated setting. The contributions
of this work can be summarised as follows.
1) We propose a novel framework to detect poisoning
attacks by malicious-edged devices in FL for health-
care applications.
2) We evaluate the performance of the proposed
framework against state-of-the-art model poisoning
and data poisoning attacks, in two different health-
care applications.
3) With performance analysis we show that the pro-
posed framework can not only detect poisoning
attacks but can also increase the performance of the
global model.
The rest of the article is organised as follows: Section 2
the background and related work. Section 3 presents the
proposed framework, section 4 and section 5 provides
performance evaluation an comparison of the proposed
framework, respectively. Section 6 presents discussion and
limitations. Finally, section 7 concludes the article.
2 BACKG ROUND A ND REL ATE D WORK
2.1 Federated Learning
Federated learning (FL) [1] is a concept of distributed ma-
chine learning in which different edge devices (hospitals,
companies, etc.) collaborate to train a joint model known
as a global model. FL trains the global model without
collecting the data from participating edge devices in a
centralized facility, i.e., the local data of edge devices is
never shared with others in the network. Instead, an edge
device trains local models using its local data and shares
the trained parameters with a central server. The central
server aggregates the received parameters according to a
given aggregation algorithm to produce parameters of the
global model [12], among which FedAvg [13] is the most
commonly used aggregation algorithm [2]. Mathematically,
FedAvg is given by the following equation:
AW =nk
nK
K
X
k=1
Wt+1
k,(1)
where AW is the global aggregated weights, nkand nKare
the number of samples of an individual edge and the total
number of samples of all the edge devices taking part in
the global round, respectively. Wt
kare the locally trained
weights of the k-th edged device in t-th round, and Kis
the number of total edged devices taking part in the global
round. This process of updating the global model repeats
until the desired level of the global model’s performance is
achieved.
2.2 Poisoning attacks
Poisoning attacks [14, 15] can substantially reduce the per-
formance (classification accuracy, precision, and recall) of
FedAvg, even in the presence of a very small percentage
of adversarial participants in the network. Such attacks can
be classified as targeted attacks, i.e., they negatively impact
only the target classes under attack, and untargeted attacks
impact all the classes negatively. Furthermore, poisoning
attacks are mainly classified into categories: data poison-
ing [14] and model poisoning [3] attacks depending on
the phase where the attacks have been launched. If the
attacker manipulates the training data then this is called
data poisoning attacks and if the attacker manipulates the
trained model’s parameters then such attacks are called
model poisoning attacks. Further details of each type of
poisoning are given as follows:
Data poisoning attacks: Data poisoning attacks are those
attacks in which the attacker manipulates the training data
according to a given strategy and then trains the model
using the manipulated dataset. In this study we consider
the following four types of data poisoning attacks:
1) Random label flipping poisoning attack (RL): In
this attack the attacker flips the true labels of the
training instance randomly.
2) Random Label and Feature poisoning attack (RLF):
In this attack in addition to flipping the label ran-
domly the attacker adds random Gaussian noise to
the input features of the training instances.
3) Label Swapping poisoning attack (LS): In this
attack the attacker swaps the label of a given class
with another.
4) Feature poisoning attack (FP): In this attack the
attacker adds white Gaussian noise (enough to ma-
nipulate the global model) to the features of the
training data.
Model poisoning attacks: In the model, poisoning at-
tacks the attacker trains the model using the legitimate
datasets and then manipulates the learned parameters be-
fore sending it to the global server. In this study, we consider
the following four types of model poisoning attacks:
1) Sign-flipping attack (SF): In this attack the attacker
trains the model using the legitimate data and then
flips the sign of trained parameters and enlarges
their magnitude.
2) Same Value Attack (SV): In this attack the attacker
sets the parameter values as C1, where 1corre-
sponds to an all-one vector and c is a constant with
a value equal to 100.
3) Additive Gaussian Noise Attack (AGA): In this
attack the attacker trains the model as expected
with legitimate data but adds Gaussian noise before
sharing the updates with the global server.
3
4) Gradient Ascent Attack (GA): In this attack the
attacker trains the models using gradient ascent
instead of gradient descent optimizer.
2.3 Stitching connectivity
Stitching connectivity [16] is a method to measure the simi-
larity of internal representations of different models trained
using different but similar data. Consider two models Aand
B, which have same architecture. For Aand Bto be stitching
connected, they can be stitched at all the layers to each other.
In other words, two models, let’s say Aand Bwith identical
architecture but trained using stochastic gradient descent
using independent random seeds and independent training
sets taken from the same distribution. Then the two trained
models are stitched connected for natural architectures and
data distributions. Hence, we expect that the models trained
on similar but different training sets of the same distribution
will behave similarly. Based on this property and our exper-
iments, we adopt Proposition 1 that will be explained later.
2.4 Memorization in deep networks
Deep neural networks are capable of memorizing the train-
ing data in a fashion such that they prioritize learning
simple patterns first [17] using the lower-level layers in
the model, while the higher-level layers tend to learn more
specific data characteristics. Furthermore, when a model is
trained on noisy data, the first half of the layers are similar
to a model trained on good-quality data [16]. Based on this
and our experiments, we adopt Proposition 2, that will be
discussed later.
3 PROPOSED FRAMEWORK
3.1 Threat Model
Attacker’s goal: Similar to many other studies [7, 10, 11],
we consider an attacker whose goal is to manipulate the
global model in such a way that it has low performance
and high error rate for test samples. Such attacks make the
global model underperform. For example, an attacker can
attack competitor FL systems. We consider both targeted
and untargeted [18] attacks, as discussed previously.
Assumptions: We consider the following assumption
about the threat model:
1) We consider that attacker edge(s) follow the FL
algorithm i.e., they train their local models using
their local data and share the parameters with the
global model.
2) We assume the attacker edge(s) can manipulate
their local training data and train the local model
using the manipulated data to malfunction the local
models (shared parameters), which are then shared
with the global model.
3) The attacker edge(s) knows the aggregation rule,
global model architecture, and local training data.
4) We also assume that the global server has a public
dataset that has data representing each class for a
given application.
3.2 Overview
In this section, we describe the proposed framework. An
overview of the proposed method has been shown in Fig-
ure 1. Let us assume Kedged devices (hospitals, organisa-
tions etc.) collaborate to train a joint global model GM. An
edge Ektrains a local model LMkusing it local data Dk,
where k= 1,2, . . . , K and Kis the number of participating
devices in each global round of FL. Global server GS is
responsible for receiving the updates from edge devices and
aggregating them. We assume that GS also has an open-
source dataset which is called public data and we represent
it as DP.DP is supposed to be a representative dataset of
all the classes in a classification problem, i.e., it has samples
from each candidate class. The training of the t-th global
round is given as follows.
1) GS creates two copies of initial global model, repre-
sented as GMt
initial and GMt.
2) GS splits DP into train DPtrain and test DPtest
datasets and trains the GMt
initial using DPtrain.
3) After training, GS makes predictions using the
trained GMt
initial and DPtest. During the predictions,
GS taps the gradients of GMt
initial for each input
sample using Algorithm 1 to create a dataset called
audit data and represented as DAt
initial, and splits it
into DAt
initial,train and DAt
initial,test.
4) GS creates another model called audit model rep-
resented as AMt(a one class classifier) and trains
it using DAt
initial,train. Here, we treat DAt
initial,train as a
single class.
5) Each Eitrains LMiusing Di.
6) Each Eisends the updated parameters Wt
iof trained
LMito GS.
7) GS sets Wt
ias the parameters of GMt
initial and makes
predictions using DPtest. During the predictions, GS
taps the gradients of GMt
initial for each input sample
using Algorithm 1 to create a dataset DAt
i.
8) GS makes predictions using AM and DAt
ias input
data. For every input sample X∈DAt
i,AM outputs
yi∈ {1,−1}and creates a set Y={y1, y2, . . . , yz},
where zis the total number of samples in DAi.
9) GS computes poisoned rate ht
i=o×100
z, where ois
the count of −1in Y.
10) GS includes Wt
iin global aggregation if ht
i≤Pt,
otherwise discards Wt
i. Here, Pt=ht
test +σ, and it
is the percentage of poison that we want to tolerate.
σis called deviation tolerance, an can be initial-
ized at development phase. It actually defines the
upper-bound of divergence between distribution of
training set of edge(s) and the initial public set that
we want to incorporate into global model. We set
σ= 10.
The proposed framework works based on two proposi-
tions that we discuss below.
Proposition 1. When a model is trained on noisy data (ma-
licious/poisoned), the first half of the layers are similar
to a model trained on good-quality data (benign).
Since the information specific to the data is learned by
the higher-level layers of the model, we take the activation
of the last layer (in the case of CNN last convolutional
4
Hospital-based Healthcare
Smart Healthcare
Home-based Healthcare
Home-based Healthcare
Compromised by
Attacker
Public Dataset (DP)
GMti
Train using DPtrain
GMti Trained with
DPTrain
Test using DPtest
Make predictions using DPtest and DPtrain.
Tape down activations of last Convolution layer
of GMti for each sample of DPtest and DPtrain to
make audit dataset
DAtinitial,test and DAtinitial,train ,
respectively.
Wt1
Global Model (GMti )
Global Model (GMt)
Train Audit
Model using
DAtinitial,train
Wt2
Wt3
Wt4
Set the shared parameters of
each local model LMi and
tape down
gradients(activations) of last
Convolution layer of model
for each Test sample and
make audit dataset (DAti)
Audit Model
Make
Predictions
using Audit and
DAti
hti > Pt?
NO
YES
Wt4
Wt1Wt2Wt3
Include in Global Aggregation
LM2
LM2
LM3
LM4
Exclude from
Global
Aggregation
Global Server
Shared model's parameters of
locally trained LMi
hti
1
2
3
8
9
10
11
12
1
4
Test Audit Model
using DAtinitial,test
Audit Model
httest
httest+σPt
5
6
7
Fig. 1: Overview of the proposed framework
Algorithm 1: Creation of Audit Dataset(s) DAt
i
Input: A dataset Diand a trained model M
Output: a new dataset DAt
i
1for each sample Xin Dido
2Transform Xinto batch size
3Make Prediction using Xas test sample in M
4Get activation maps Ak
l,w of the last
convolutional layer. where kis the number of
activation maps with length land width weach.
5Get probability score yc, where cis the class label
of X.
6Reshape Akas an array of 1×j, where
j=l×w×k.
7Reshape Xas an array of 1×l, where lis
product of height and width of X
8compute s=XkAkkyc
9Append sin DAt
i
10 Output DAt
i
layer) to capture more information (features) related to the
training data. We only consider the activations of the last
as it provides sufficient information about the underly-
ing training data to detect poisoning attacks. Activations
from other higher-level layers can also be incorporated for
this purpose, nevertheless, this increases the computational
costs with no significant improvement in the results for
detecting poisoning attacks. Hence, we only consider the
activations of the last convolutional hidden layer.
Proposition 2. Different models with the same architecture
but random initial seeds, trained on different training
sets of a similar distribution have similar internal rep-
resentation and thereafter similar activations for a given
test input sample.
Based on Proposition 2, we expect that the models
trained on different datasets of similar distributions will
behave similarly. We capture this property by training an-
other model (audit model) to learn the behaviour of a model
trained on the benign dataset (Initial global model in our
case). Hence, models behaving similar to the audit model
are probably free from poisoning attacks, and at least they
do not degrade the performance of the global model.
4 PERFORMANCE EVALUATIO N
We evaluated the proposed framework using two healthcare
applications, i.e., ECG classification and HAR.
4.1 Experimental Setup
Datasets: For ECG classification, we use the widely known
MIT-BIT arrhythmia dataset [19]. The dataset contains 48-
half-hour two-channel ECG recordings. These recordings
were obtained from 47 subjects. The dataset contains 109,446
5
samples, sampled at a frequency of 125 Hz. Further, the
dataset contains five classes of ECG: non-ecotic beats (nor-
mal beat), supraventricular ectopic beats, ventricular ectopic
beats, fusion beats, and unknown beats.
For the HAR, we used the dataset in [20, 21]. The dataset
contains time-series data related to 14 different human ac-
tivities (Standing, sitting, walking, jogging, up-stairs walk,
down-stairs walk, eating, writing, using a laptop, washing
face, washing hands, swiping, vacuuming, dusting, and
brushing teeth) collected using sensors such as accelerom-
eter, magnetometer, and gyroscope.
Classifiers: We developed a convolution neural net-
works (CNN) based classifier for each application. The
developed classifiers do not achieve the best classification
for the considered datasets, because our objective is to
show that our proposed framework can detect anomalous
(poisoning attacks), not to achieve the best performance in
terms of classification. For ECG classification we developed
a five-class classifier and the aim of FL here is to learn a
global five-class classifier, and for HAR we developed a
fourteen-class classifier, and the aim of FL here is to learn
a global fourteen-class classifier.
Federated Setting: To simulate the federated setting we
simulate four edge devices and a global server using a
Dell workstation with 32 GB RAM and an Intel® Core™ i-
6700HQ CPU. Furthermore, we divided the dataset equally,
but randomly among the participating edge devices for each
application, i.e., ECG classification and HAR.
4.2 Performance Evaluation
In order to evaluate the performance of the proposed frame-
work. First, we test the proposed framework to check its
ability to differentiate DAiof benign edge(s) from DAiof
malicious edge(s). We combine the DAigenerated using
local models of benign edge(s) and DAigenerated using
poisoned local models (both data and model poisoned)
malicious edge(s) and label them as 1 and -1, respectively.
Table 1 shows the classification accuracy of the proposed
framework to differentiate samples generated using shared
parameters of benign edged devices and the samples gener-
ated using shared parameters of malicious edge devices for
HAR application. Similarly, Table 2 shows the classification
accuracy of the proposed framework to differentiate sam-
ples generated using shared parameters of benign edged
devices and the samples generated using shared parameters
of malicious edge devices for ECG classification application
in healthcare. For both types of applications, it can be seen
that the proposed framework can differentiate samples of
benign and malicious edged devices very well, with an
overall accuracy of 94% and 99% for HAR and ECG clas-
sification, respectively.
TABLE 1: Classification accuracy of OCSVM for HAR
Class Precision Recall F1-Score Support
Benign (1) 96 95 95 22,280
Malicious (-1) 90 91 91 11,140
Accuracy 94 33,420
Micro average 93 93 93 33,420
Weighted average 94 94 94 33,420
TABLE 2: Classification accuracy of OCSVM for ECG classi-
fication
Class Precision Recall F1-Score Support
Benign (1) 100 97 98 9,000
Malicious (-1) 97 100 99 9,000
Accuracy 99 18,000
Micro average 99 99 99 18,000
Weighted average 99 99 99 18,000
CN GM LS RLF RL FP
0
20
40
60
80
100
83 87
43
37
26
76
Model
Accuracy(%)
Fig. 2: Performance of the Global Model with and without
proposed framework under data poisoning attacks for ECG
classification.
We tested the performance of the proposed framework
under different types of model and data poisoning attacks,
as discussed in section 2.2. First, we present the compar-
ison of the accuracy of the global model after one global
round with and without our proposed framework, under
poisoning attacks. Figure 2 presents a comparison of the
accuracy of the global model under different data poisoning
attacks and with and without the proposed framework for
ECG classification. Where, CN is centralized model trained
using DP, GM is global model with proposed framework in
federated setting, LS is Global model under label swapping
attack without proposed framework in federated setting,
RLF is global model under radom label and feature poison-
ing attack without proposed framework in federated setting,
RL is global model under random label attack without
proposed framework in federated setting, and FP is global
model under feature poisoning attack without proposed
framework in federated setting. It can be seen that the
performance of the global model deteriorates significantly
under the data poisoning attacks. It can also be seen that
the performance of the global model has been improved
by adopting the proposed framework which eliminates the
poisoned updates from the malicious edge(s).
Similarly, Figure 3 shows the performance of global
model compared under data poisoning attacks with and
without adopting the proposed framework for HAR appli-
cation. Moreover, Figure 4 presents a comparison of the ac-
curacy of the global model under different model poisoning
attacks and with and without the proposed framework for
6
CN GM LS RLF RL FP
0
20
40
60
80
100
86 90
45
23 26
62
Model
Accuracy(%)
Fig. 3: Performance of the Global Model with and without
proposed framework under data poisoning attacks for HAR.
.
CM GM SF SV AGA GA
0
20
40
60
80
100
84 88
18
7
67
21
Model
Accuracy(%)
Fig. 4: Performance of the Global Model with and without
proposed framework under model poisoning attacks for
ECG.
ECG classification. Where, CN represents the performance
of ECG classification model in centralised setting using DP
data, GM shows the performance of global model with
proposed framework in federated setting, SF shows the
performances of global model under sign flip attack without
proposed framework in federated setting, SV shows the per-
formance of global model under same value attack without
proposed framework in federated setting, AGA shows the
performance additive Gaussian noise attack without pro-
posed framework in federated setting, and GA shows the
performance of global model under gradient ascent attack
without proposed framework in federated setting. Similarly,
Figure 5 presents comparison of the accuracy of the global
model under different model poisoning attacks and with
and without the proposed framework for HAR application
Table 3 shows the accuracy of the proposed framework
CM GM SF SV AGA GA
0
20
40
60
80
100
86 90
14 12
71
17
Model
Accuracy(%)
Fig. 5: Performance of the Global Model with and without
proposed framework under model poisoning attacks for
HAR.
to detect different data poisoning attacks in federated set-
tings for the HAR applications. Ptvalue in the table shows
the threshold, which is calculated using the audit test data
of the initial global model, ht
test is the poisoned amount
detected for audit test data for the initial global model, t
is the global round number, and σis known as tolerance
which is added to avoid miss-classification of benign up-
dates. Any updates (Wi) from an edge device for which
its corresponding DAihas a value of ht
igreater than the
threshold Ptwill be marked as malicious or poisoned and
removed from global aggregation. It can be seen that in
Table 3, ht
1and ht
2for edge 1 and edge 2, respectively, have
a value smaller then Pt. Hence, the updates of Wt
1and Wt
2
will be included in global aggregation, while Wt
3will be
excluded from global aggregation as its corresponding ht
3is
greater then Pt. It can be seen that the proposed framework
can not only detect the attacks but can also provide an
insight into the percentage of poisoned data being used
to train the model. For example, for LS we swapped the
label of some classes (for ECG classification two classes
and for HAR 4 classes) while keeping the rest of labels in
their original form, thereafter it gives a value of 47.7 for
ECG and 81.2 for HAR. Similarly, Table 4, 6, 5 presents the
accuracy of proposed framework to detect data poisoning
attacks in ECG classification, model poisoning attacks in
ECG classification and model poisoning attacks in HAR
application, respectively.
5 COMPARISON
In this section, we compare the proposed framework with
some of the existing works [4, 22, 23] as shown in Table 7.
Blanchard et al. [4] proposed Krum, an aggregation rule
by selecting one of the nlocal models similar to other
models as the global model. The intuition is that even if the
local model is a compromise, its impact may be constrained
since it is similar to other local models which are possibly
benign. Yin et al. [22] aggregates the parameter of local
7
TABLE 3: Detection of data poisoning attacks on HAR
Attack Type Pt(ht
test = 20.0, σ = 10.0)Edge 1 (h1) Edge 2 (ht
2)Edge 3 (Malicious) (ht
3)
RLF 30.0 23.3 11.5 94.5
RL 30.0 23.5 10.0 100
LS 30.0 21.1 19.7 81.2
FP 30.0 18.3 16.1 91.0
TABLE 4: Detection of data poisoning attacks on ECG classification
Attack Type Pt(ht
test = 15.0, σ = 10.0)Edge 1 (ht
1) Edge 2 (ht
2)Edge 3 (Malicious) (ht
3)
RLF 25.0 10.2 14.5 100
RL 25.0 14.5 19.0 100
LS 25.0 12.1 21.3 47.7
FP 25.0 13.3 22.1 100
TABLE 5: Detection of model poisoning attacks on HAR classification
Attack Type Pt(ht
test = 20.0, σ = 10.0)Edge 1 (ht
1) Edge 2 (ht
2)Edge 3 (Malicious) (ht
3)
SF 30.0 20.3 19.5 100
SV 30.0 20.2 20.1 100
AGA 30.0 20.1 19.7 90.1
GA 30.0 20.3 20.1 100
TABLE 6: Detection of model poisoning attacks on ECG classification
Attack Type Pt(ht
test = 15.0, σ = 10.0)Edge 1 (ht
1) Edge 2 (ht
2)Edge 3 (Malicious) (ht
3)
SF 25.0 10.0 11.7 100
SV 25.0 10.2 10.1 100
AGA 25.0 9.9 9.7 96.1
GA 25.0 10.1 10.4 100
models independently i.e., for i-th model parameter the
global server sorts the i-th parameter of the mother local
models. Removes the largest and smallest parameters and
computes the mean of the remaining parameters as the i-
th parameter of the global model. Similarly, El Mhamdi et
al. [23] proposed to combine Krum and a variant of trimmed
mean [22]. However, these approaches are vulnerable to
poisoning attacks even using robust aggregation [9]. Ad-
ditionally, methods such as median, and trimmed median
try to minimize the effect of poisoning attacks by taking
the median and mean of individual parameters of local
models, which degrades (they greatly induce error rate in
the global model learned) the performance of the global
model. Additionally, such methods under-perform with the
increasing number of malicious edge devices [3]. On the
other hand, since our proposed framework checks each local
model individually, and rejects the malicious local model
the performance of the global is not degraded, and the
increasing number of malicious edge devices can not affect
the performance of the proposed framework and thereafter
such attacks have no effect on the performance of the global
model.
6 DISCUSSION AND LIMITATIONS
In this section, we provide a discussion and limitations of
the proposed framework. Generally, in healthcare most of
the health conditions such as different classes of arrhythmia
are well known, each class has similar properties across
different patients. Our proposed model can incorporate
TABLE 7: Comparison with state-of-the-art methods
Scheme
Robust against
increasing
malicious edges
Degraded Global
performance
(increased error
rate)
[4] No Yes
[22] No Yes
[23] No Yes
Proposed Yes No
updates from different edged devices trained with training
sets similar to the training data set of the audit model. The
initial global model which was trained using DP can be
updated using the updated global model after each suc-
cessful aggregation, which makes the initial global model
robust, the process can be repeated for subsequent global
rounds. However, if an edge device trains its local model
using data that represents a class that is not represented in
DP, yet benign the proposed framework will classify it as
malicious. Hence, the proposed framework should not be
used in applications with emerging complete new classes,
its only suitable for applications where the candidate classes
are already known and need to obtain a robust global model
for known classes.
7 CONCLUSIONS
In this paper, we proposed a novel framework to detect
poisoning attacks in FL for healthcare applications. The
proposed framework can efficiently detect state-of-the-art
8
poisoning attacks by observing the activations of the shared
weights of the local models. Unlike, other existing meth-
ods, the proposed framework can detect poisoning attacks
without degrading the global model’s performance. More-
over, the increasing number of attacker edge devices in
the FL network can not compromise the security of the
proposed framework. Furthermore, we tested the proposed
framework under different poisoning attacks for two health-
care applications. The performance analysis shows that the
proposed framework can efficiently detect the malicious
updates and can exclude them from the global aggregation,
which results in an increase in the performance of the global
model.
ACKNOWLEDGEMENTS
This research work was supported by the I-SITE Univer-
sité Lille Nord-Europe 2021 of France under grant No. I-
COTKEN-20-001-TRAN-RAZA.
REFERENCES
[1] B. McMahan, E. Moore, D. Ramage, S. Hampson,
and B. A. y Arcas, “Communication-efficient learning
of deep networks from decentralized data,” in
Proceedings of the 20th International Conference on
Artificial Intelligence and Statistics. ML Research Press,
2017, pp. 1273–1282. [Online]. Available: http://procee
dings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf
[2] T. Sun, D. Li, and B. Wang, “Decentralized federated
averaging,” arXiv:2104.11375 [cs.DC], 2021. [Online].
Available: https://doi.org/10.48550/arXiv.2104.11375
[3] M. Fang, X. Cao, J. Jia, and N. Gong, “Local
model poisoning attacks to Byzantine-robust federated
learning,” in Proceedings of the 29th USENIX Security
Symposium. USENIX Association, 2020, pp. 1605–1622.
[Online]. Available: https://www.usenix.org/confere
nce/usenixsecurity20/presentation/fang
[4] P. Blanchard, E. M. El Mhamdi, R. Guerraoui,
and J. Stainer, “Machine learning with adversaries:
Byzantine tolerant gradient descent,” in Advances in
Neural Information Processing Systems, vol. 30. Curran
Associates, Inc., 2017, pp. 119–129. [Online]. Available:
https://papers.nips.cc/paper/2017/hash/f4b9ec30a
d9f68f89b29639786cb62ef-Abstract.html
[5] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith,
“Federated learning: Challenges, methods, and future
directions,” IEEE Signal Processing Magazine, vol. 37,
no. 3, pp. 50–60, 2020. [Online]. Available: https:
//doi.org/10.1109/MSP.2020.2975749
[6] X. Zhou, M. Xu, Y. Wu, and N. Zheng, “Deep
model poisoning attack on federated learning,” Future
Internet, vol. 13, no. 3, pp. 73:1–14:14, 2021. [Online].
Available: https://doi.org/10.3390/fi13030073
[7] B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks
against support vector machines,” in Proceedings of
the 29th International Conference on Machine Learning.
ICML, 2012, pp. 1467–1474. [Online]. Available:
https://icml.cc/2012/papers/880.pdf
[8] M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-
Rotaru, and B. Li, “Manipulating machine learning:
Poisoning attacks and countermeasures for regression
learning,” in Proceedings of the 2018 IEEE Symposium on
Security and Privacy. IEEE, 2018, pp. 19–35. [Online].
Available: https://doi.org/10.1109/SP.2018.00057
[9] P. Liu, X. Xu, and W. Wang, “Threats, attacks and
defenses to federated learning: issues, taxonomy and
perspectives,” Cybersecurity, vol. 5, no. 1, pp. 4:1–4:19,
2022. [Online]. Available: https://doi.org/10.1186/s4
2400-021-00105-6
[10] J. Chen, L. Zhang, H. Zheng, X. Wang, and
Z. Ming, “DeepPoison: Feature transfer based stealthy
poisoning attack for DNNs,” IEEE Transactions on
Circuits and Systems II: Express Briefs, vol. 68,
no. 7, pp. 2618–2622, 2021. [Online]. Available:
https://doi.org/10.1109/TCSII.2021.3060896
[11] A. Shafahi, W. R. Huang, M. Najibi, O. Suciu,
C. Studer, T. Dumitras, and T. Goldstein, “Poison
frogs! targeted clean-label poisoning attacks on
neural networks,” in Advances in neural information
processing systems, vol. 31. Curran Associates, Inc.,
2018, pp. 6103–6113. [Online]. Available: https:
//proceedings.neurips.cc/paper/2018/hash/22722a3
43513ed45f14905eb07621686-Abstract.html
[12] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao,
“A survey on federated learning,” Knowledge-Based
Systems, vol. 216, pp. 106 775:1–106 775:11, 2021.
[Online]. Available: https://doi.org/10.1016/j.knosys
.2021.106775
[13] B. McMahan, E. Moore, D. Ramage, S. Hampson,
and B. A. y Arcas, “Communication-efficient learning
of deep networks from decentralized data,” in
Proceedings of the 20th International Conference on
Artificial Intelligence and Statistics, ser. Proceedings
of Machine Learning Research, vol. 54. PMLR,
2017, pp. 1273–1282. [Online]. Available: https:
//proceedings.mlr.press/v54/mcmahan17a.html
[14] V. Tolpegin, S. Truex, M. E. Gursoy, and L. Liu, “Data
poisoning attacks against federated learning systems,”
in Computer Security – ESORICS 2020: 25th European
Symposium on Research in Computer Security, ESORICS
2020, Guildford, UK, September 14–18, 2020, Proceedings,
Part I. Springer, 2020, pp. 480–501. [Online]. Available:
https://doi.org/10.1007/978-3-030-58951-6_24
[15] P. M. Mammen, “Federated learning: Opportunities
and challenges,” arXiv:2101.05428 [cs.LG], 2021.
[Online]. Available: https://doi.org/10.48550/arX
iv.2101.05428
[16] Y. Bansal, P. Nakkiran, and B. Barak, “Revisiting
model stitching to compare neural representations,”
in Advances in Neural Information Processing Systems,
vol. 34. Curran Associates, Inc., 2021. [Online].
Available: https://proceedings.neurips.cc/paper/202
1/hash/01ded4259d101feb739b06c399e9cd9c-Abstract
.html
[17] D. Arpit, S. Jastrz˛ebski, N. Ballas, D. Krueger,
E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer,
A. Courville, Y. Bengio, and S. Lacoste-Julien, “A closer
look at memorization in deep networks,” in Proceedings
of the 34th International Conference on Machine Learning,
ser. Proceedings of Machine Learning Research,
vol. 70. PMLR, 2017, pp. 233–242. [Online]. Available:
9
http://proceedings.mlr.press/v70/arpit17a.html
[18] O. Suciu, R. Marginean, Y. Kaya, H. Daume III, and
T. Dumitras, “When does machine learning FAIL?
generalized transferability for evasion and poisoning
attacks,” in Proceedings of the 27th USENIX Security
Symposium. USENIX Association, 2018, pp. 1299–1316.
[Online]. Available: https://www.usenix.org/confere
nce/usenixsecurity18/presentation/suciu
[19] G. B. Moody and R. G. Mark, “The impact of the
MIT-BIH arrhythmia database,” IEEE Engineering in
Medicine and Biology Magazine, vol. 20, no. 3, pp. 45–50,
2001. [Online]. Available: https://doi.org/10.1109/51
.932724
[20] A. Raza, K. P. Tran, L. Koehl, S. Li, X. Zeng,
and K. Benzaidi, “Lightweight transformer in
federated setting for human activity recognition,”
arXiv:2110.00244 [cs.CV], 2021. [Online]. Available:
https://doi.org/10.48550/arXiv.2110.00244
[21] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity
recognition using cell phone accelerometers,” ACM
SIGKDD Explorations Newsletter, vol. 12, no. 2, pp.
74–82, 2011. [Online]. Available: https://doi.org/10.1
145/1964897.1964918
[22] D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-
robust distributed learning: Towards optimal statistical
rates,” in Proceedings of the 35th International Conference
on Machine Learning, ser. Proceedings of Machine
Learning Research, vol. 80. PMLR, 2018, pp. 5650–
5659. [Online]. Available: https://proceedings.mlr.pr
ess/v80/yin18a.html
[23] E. M. El Mhamdi, R. Guerraoui, and S. Rouault,
“The hidden vulnerability of distributed learning in
Byzantium,” in Proceedings of the 35th International
Conference on Machine Learning, ser. Proceedings
of Machine Learning Research, vol. 80. PMLR,
2018, pp. 3521–3530. [Online]. Available: https:
//proceedings.mlr.press/v80/mhamdi18a.html
Ali Raza received his BS degree in Computer
Engineering from the University of Engineering
and Technology, Taxila, Pakistan, in 2017, his
MS degree in Electronics and Computer Engi-
neering from the Hongik University, Korea, in
2020. He is currently pursuing a cotutelle (dual
award) PhD jointly funded by the University of
Lille in France and the University of Kent, UK.
He is currently studying at the institute of cyber-
security for society, university of Kent. His re-
search topic is smart healthcare with federated
learning, security and data privacy. His research interests include cryp-
tography, cyber-security, and machine learning.
Shujun Li (M’08–SM’12) is Professor of Cyber
Security at the School of Computing and Direc-
tor of the Institute of Cyber Security for Society
(iCSS), University of Kent, UK. He has published
over 100 scientific papers, including four winning
a best paper award (the 2022 IEEE Transactions
on Circuits and Systems Guillemin-Cauer Best
Paper Award, and three Best Paper Awards at
HICSS 2021, HAS 2007 and IIEEJ IEVC 2012)
and one winning a Honorable Mention at ICWSM
2020. His research interests are about inter-
disciplinary topics related to cyber security and privacy, human factors,
digital forensics and cybercrime, multimedia computing, and data sci-
ence. Professor Shujun Li is on the editorial boards of a number of
international journals and participated in the organization of more than
100 international conferences and workshops.
Kim Phuc Tran is currently an Associate Pro-
fessor in Automation and Industrial Informatics
at the ENSAIT and the GEMTEX laboratory,
University of Lille, France. He received the En-
gineering degree and the Master of Engineer-
ing degree in Automated Manufacturing. He ob-
tained a Ph.D. in Automation and Applied In-
formatics at the Université de Nantes, France.
His research works deal with Real-time Anomaly
Detection with Machine Learning, Decision sup-
port systems with Artificial Intelligence, and En-
abling Smart Manufacturing with IIoT, Federated learning, and Edge
computing. He has published more than 55 papers in peer-reviewed
international journals and proceedings of international conferences. He
is the Topic Editor for the Sensors journal. In addition, as the project
coordinator, he is conducting one regional research project about Smart
Healthcare System with Federated Learning. He has been or is involved
in three regional research and European projects. He is an expert for
the Research and Innovation program of the Government of the French
Community, Belgium.
Ludovic Koehl is a Professor at Ensait, Uni-
versity of Lille. He has been involved in a great
number of projects dealing with optimization of
the quality and comfort of textiles by integrating
physical measures and human knowledge in the
field of technical textiles, quality of textiles as well
as new usage and consumers’ behaviour studies
for textile industry. Since 1998, published more
than 92 papers (to March 2019). Research inter-
ests include pattern recognition, data mining, big
data management, recommendation systems,
traceability, computer modeling and their applications in textile industry.
