ArticlePDF Available

IoT-Based Remote Pain Monitoring System: From Device to Cloud Platform

Authors:

Abstract and Figures

Facial expressions are among behavioral signs of pain that can be employed as an entry point to develop an automatic human pain assessment tool. Such a tool can be an alternative to the self-report method and particularly serve patients who are unable to self-report like patients in the Intensive Care Unit and minors. In this paper, a wearable device with a bio-sensing facial mask is proposed to monitor pain intensity of a patient by utilizing facial surface electromyogram (sEMG). The wearable device works as a wireless sensor node and is integrated into an Internet of Things system for remote pain monitoring. In the sensor node, up to eight channels of sEMG can be each sampled at 1000 Hz, to cover its full frequency range, and transmitted to the cloud server via the gateway in real-time. In addition, both low energy consumption and wearing comfort are considered throughout the wearable device design for long-term monitoring. To remotely illustrate real-time pain data to caregivers, a mobile web application is developed for real-time streaming of high-volume sEMG data, digital signal processing, interpreting, and visualization. The cloud platform in the system act as a bridge between the sensor node and web browser, managing wireless communication between the server and the web application. In summary, this study proposes a scalable IoT system for real-time biopotential monitoring and a wearable solution for automatic pain assessment via facial expressions.
Content may be subject to copyright.
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 1
IoT-based Remote Pain Monitoring System: from
Device to Cloud Platform
Geng Yang, Member, IEEE, Mingzhe Jiang, Student Member, IEEE, Wei Ouyang,
Guangchao Ji, Student Member, IEEE, Amir M. Rahmani, Senior Member, IEEE, Pasi Liljeberg, Member, IEEE,
and Hannu Tenhunen, Member, IEEE
Abstract—Facial expressions are among behavioural signs of
pain that can be employed as an entry point to develop an
automatic human pain assessment tool. Such a tool can be
an alternative to the self-report method and particularly serve
patients who are unable to self-report like patients in the
Intensive Care Unit and minors. In this paper, a wearable device
with a bio-sensing facial mask is proposed to monitor pain
intensity of a patient by utilizing facial surface electromyogram
(sEMG). The wearable device works as a wireless sensor node and
is integrated into an Internet of Things system for remote pain
monitoring. In the sensor node up to eight channels of sEMG can
be each sampled at 1000 Hz, to cover its full frequency range,
and transmitted to the cloud server via the gateway in real-
time. In addition, both low energy consumption and wearing
comfort are considered throughout the wearable device design
for long term monitoring. To remotely illustrate real-time pain
data to caregivers, a mobile web application is developed for
real-time streaming of high-volume sEMG data, digital signal
processing, interpreting, and visualization. The cloud platform
in the system act as a bridge between sensor node and web
browser, managing wireless communication between server and
the web application. In summary, this study proposes a scalable
IoT system for real-time biopotential monitoring and a wearable
solution for automatic pain assessment via facial expressions.
Index Terms—Pain assessment, Healthcare Internet-of-Things
(IoT), Biopotential sensor node, Wearable sensors, Cloud com-
puting, Web-based UI for IoT applications
I. INTRODUCTION
REMOTE health monitoring systems in hospital or home
is required to reduce the overall healthcare cost and
optimizing healthcare processes and work-flows [1–3]. Pain is
among the key remote monitoring indexes as a vital indicator
in disease diagnosis and relieving discomfort of patients.
Some tele-health research efforts have been done through
This work was supported in part by Fundamental Research Funds for the
Central Universities, and the Science Fund for Creative Research Groupsof
the National Natural Science Foundation of China (GrantNo.51521064)
G. Yang is with State Key Laboratory of Fluid Power and Mechatronic
Systems, College of Mechanical Engineering, Zhejiang University, China.
(Email: yanggeng@zju.edu.cn)
M. Jiang and P. Liljeberg are with Department of Future Technologies,
University of Turku, Finland. (Email: mizhji, pasi.liljeberg@utu.fi)
A. M. Rahmani is with Department of Computer Science, University of
California Irvine, USA and Institute of Computer Technology, TU Wien,
Austria. (Email: amirr1@uci.edu)
W. Ouyang is with Imaging and Modeling Unit, Institut Pasteur, France.
(Email: wei.ouyang@cri-paris.org)
G. Ji is with School of Technology and Health, KTH Royal Institute of
Technology, Stockholm, Sweden. (Email: gji@kth.se)
H. Tenhunen is with Department of Future Technologies, University of
Turku, Finland and Department of Industrial and Medical Electronics, KTH
Royal Institute of Technology, Sweden. (Email: hannu@imit.kth.se)
telephone-based, web-based or mobile application question-
naire answered by patients themselves across various patient
groups [2, 4–7]. The questionnaire includes pain intensity in
visual analog scale, pain location, and medications for patients
at home. Results from those researches indicate that such
remote pain monitoring and feedback system is feasible and
effective in monitoring daily pain state. However, there are
several constrains in the self-report methods. People’s non-
compliance to manual entry of daily information, especially in
long-term monitoring, is one of such issues [2]. The second
limitation is that such methods are not suitable for the group
of people with limited cognition and expressing abilities, such
as neonates, elderly, and sedated patients in Intensive Care
Unit (ICU). The third shortcoming of conventional approaches
lies in the lacking of efficient way providing patients with
continuous and real time pain state monitoring. As a result,
patients are exposed to severe and long lasting pain due to
delayed access to medical treatment and prolonged waiting
time. These were the rationale behind the recent studies on
objective and automatic pain intensity monitoring methods.
Some efforts have been devoted to achieving automatic
pain assessment, either from the approach of facial expression
recognition in face video [8] or physiological signal fusion
[9]. However, there exist limited efforts in integrating auto-
matic pain assessment with remote health monitoring systems
[10, 11]. In this paper, both cloud and Internet of Things
(IoT) technologies are leveraged to propose an automatic pain
assessment tool for remote patient monitoring systems. The
system proposed in this paper is targeted for inpatients and
ICU patients, utilizing a novel pain intensity detection tool as
an alternative to face video.
The pain intensity detection tool proposed in this study
detects and evaluates pain from facial expressions by mon-
itoring facial surface electromyography (sEMG). The facial
muscle activities are monitored by a facial mask embedded
with surface sensors which can be developed into a wearable
device and further integrated into a remote monitoring system.
In addition to the proposed pain assessment tool, a remote
pain monitoring system is also presented in this study having
the function of multi-channel biopotential data acquisition,
wireless data transmission, signal processing and remote data
visualization. The proposed system involves both sensing
devices and a cloud-based back-end supporting patients who
have access to Wi-Fi hotspot and caregivers using a computer
or a smart device with web browsing feature. The cloud-based
back-end enables the system to be extendable with advanced
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 2
data analytics such as big data processing and deep learning.
To summarize, the main contributions of this article are as
follows:
proposing a wearable bio-sensing mask for capturing
facial expression in pain monitoring application.
designing a low-power and miniaturized eight-channel
biopotential sensing and processing module with Wi-Fi
data transmission.
developing a cross-platform interactive mobile web ap-
plication for recording, processing, interpreting, and vi-
sualization of biopotential data stream.
implementing a cloud-based platform with data storage
and web server, customized for high data rate streaming
IoT applications capable of synchronizing data with the
web application.
evaluating the pain states using multi-channel facial
sEMG.
The rest of the paper is organized as follows. In Section II,
literature review and requirement analysis are stated from the
device and system viewpoints. Section III illustrates the ap-
plication scenario, remote pain monitoring in hospital. System
architecture and its elements are also presented in the same
section, covering all the components from wearable sensor
node, gateway, and cloud server to mobile web application.
Section IV then presents the implementation of each compo-
nent in detail. System evaluation is presented in Section V,
while Section VI concludes the paper.
II. MOT IVATIO N AN D REL ATED WO RK
This study is motivated by the fact that a growing demand
for automatic pain assessment arises in hospitals to improve
the quality of care for patients and also to facilitate the
work-flow for medical staff. Facial expression is among the
behavioral signs in pain assessment tool for patients unable
to self-report, and thus it is taken as an index of pain in the
remote pain monitorin system. To meet the requirements of
multi-channel biopotential acquisition and transmission in an
IoT framework, the concept of a wearable sensor node for
pain monitoring is proposed. A corresponding IoT architecture
with a mobile web application is meanwhile customized for
biopotential processing and pain remote monitoring.
In terms of pain related facial expressions, five facial
muscles (corrugator, orbicularis oculi, levator, zygomatic and
risorius) are involved in adults [12]. Existing wearable or
portable sEMG devices are application specific for limb
muscles [13, 14] and bulky in size which are not proper
for tiny muscles. Moreover, sensor is usually designed as
channel independent when having the function of wireless data
transmission. There is a wearable facial EMG device designed
in [15]. It estimates the user’s positive or negative emotion
from two channels of EMG on the side of the face, while the
facial expressions of pain involve more facial muscles than
those on the side of the face. A compact and unobtrusive
multiple-channel sEMG sensing device is therefore designed
in this study to measure the activity of tiny facial muscles for
pain assessment application. Furthermore, the captured sEMG
signal can be then processed and displayed in real-time with
the support from the system.
There are several concerns in designing the wearable sEMG
device and the first one comes with biopotential signal quality.
Electrode to skin contact impedance is one parameter relates
to signal quality, where high impedance could cause weak
conductivity between the electrodes and the skin and therefore
degrade the signal to noise ratio and ekectrolyte gel is usually
used to reduce such impedance [16]. In addition, biopotentials
are prone to be corrupted by electrical interference coupled to
human body and unshielded lead wires particularly when lead
wires are long [17]. This issue is usually solved by applying 50
Hz or 60 Hz notch filter to raw samples. The second technical
concern is the power consumption of the sensor node as a
battery powered device. From the system point of view, data
processing can be done on the sensor and then the results
are sent to the server [18], or alternatively, original data is
transmitted to the fog/cloud server for processing [19–21]. The
latter case may save energy for the device and help reach a
balance between device energy consumption and transmission
delay. Besides those technical concerns, comfortableness is
also one key point in designing a wearable device.
To monitor biosignal stream from the caregiver end user’s
perspective, web service is a solution to real-time or near real-
time data visualization in an IoT platform [22]. Compared
with developing a web application or a mobile application for
a single platform, a hybrid application is more interoperable
and can be used across multiple mobile platforms such as
Android, iOS, Windows Phone and Blackberry where the web
application runs in mobile browsers [23, 24]. Therefore, the
mobile application in this work is built in a Hybrid Mobile
App UI framework.
III. REM OTE PAIN MONITORING SYS TE M
The IoT-based remote pain monitoring application and its
architecture are shown in Fig. 1, which is designed for the
real-time central monitoring of inpatients and intensive care
patients in hospital. It is composed of four parts:
Fig. 1. Remote pain monitoring in hospital for inpatients and ICU patients
1) Wearable sensor node: The sensor node is composed of
a passive sEMG sensor facial mask and sEMG sensing and
processing module. The mask is built on soft and stretchable
material. It is replaceable after use. While the hardware
module is reusable responsible for biosignal conditioning,
digitalization and wireless data transmission. The electrodes
integrated on the inner side surface of the mask are closely
attached to facial skin for reliable sEMG measurement. The
placement of the electrodes is determined by the targeted
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 3
Fig. 2. The structure of IoT-based biopotential measurement system and cloud-based data flow in the automatic pain monitoring system
facial muscles. Due to the soft nature of the implemented
mask, the electrode position and the shape of the mask can be
slightly adjusted accordingly to accommodate individual facial
differences.
2) Gateway: As the intermedia between sensor nodes and
cloud, the gateway can be a general router, personal hotspot
of a cellphone, or a smart gateway supporting with added
features such as heterogeneity, scalability and reliability [25].
The system can benefit from smart gateways especially when
heterogeneous data and communication technologies exist in
the overall healthcare remote monitoring system.
3) Cloud server: The cloud server receives data from the
sensor nodes with UDP or TCP protocol, then forwards to a
data streaming channel which can be connected with another
database server for storage and any device with a HTML5-
enabled web browser. This server model are compatible with
cloud computing models such as Software as a Service (SaaS)
and Platform as a Service (PaaS), which could be a key
advantage for further signal processing and data mining, as
well as implementing flexible data privacy policy.
4) Mobile web application: In this architecture, a HTML5-
based mobile application acts as an interactive interface be-
tween system and caregivers. It can render real-time waveform,
conduct lightweight algorithms, save data to an in-browser
database and synchronize with the remote database servers.
The web browser based application is a client-server software
application, in which the user interface runs in a web browser.
One of noticeable advantages of the implemented web app
is that it can be widely applied on various terminal devices.
In general, it is a cross-platform app, compatible with main-
stream operating systems, such as MS Windows, Mac OS, and
Android. The implemented web app is capable of running on
any terminal devices equipped with a web browser, including
stationary terminals and mobile devices.
IV. DEVICE DES IG N AN D SYS TE M IMPLEMENTATION
The implemented IoT based pain monitoring system can be
divided into three main functional parts, as shown in Fig. 2.
The first part is a wearable sensor for multi-channel sEMG
acquisition, which includes a biopotential acquisition sensor
node and a wearable bio-sensing mask. The sensor node also
works as a UDP client transmitting data to the cloud through
the gateway. The second part of the implementation is the
cloud server, which is in charge of wireless data transmission
from the sensor node to remote caregivers by using UDP
communication and websockets. Moreover, it also contains the
node.js server for the mobile application and a database for
synchronizing the recorded data files selected by end users,
caregivers. The third part is web application embedded with
digital signal processing. It includes dynamic sEMG waveform
display on devices across operating systems. The entire system
implementation and data transmission flow are illustrated in
Fig. 2. The details of each part are presented in the subsections
below.
A. Wearable sensor design for pain monitoring
The sensor node is first designed to meet the data acquisition
and transmission requirement considering the data rate of
sEMG. Then the whole sensor node is integrated including
functional modules, battery power supply and skin-friendly
device packaging. Another part of the design is a facial mask
embedded with passive electrodes for wearable use. The facial
mask together with the sensor node constitutes the wearable
sensor monitoring facial sEMG.
1) Sensor node design for biopotential acquisition: As a
continuously running wearable device, low power consump-
tion is taken into consideration during the design process.
Therefore, a low power analog front end (AFE) ADS1198 [26]
is chosen for biopotential measurement, which is an eight-
channel AFE with 16-bit analog to digital resolution and a
programable gain from 1 to 12. The sampling rate of each
channel is programmable and is set to 1000 samples per
second (SPS) to cover the full bandwidth of sEMG signal
[27] according to Nyquist sampling theorem and the gain is
set to be 12. So the data rate with full channel transmission
is 128 Kbps at a minimum. RTX4140 Wi-Fi module [28]
is utilized due to SPI communicating with ADS1198, its
Wi-Fi functionality and its outstanding power performance
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 4
among low power Wi-Fi Modules [29]. On the basis of these
two modules, the sensor node is implemented by hardware
design and software co-design using Co-Located application
framework in the Micro Controller Unit of RTX4140.
2) Sensor node integration: Since the proposed wearable
device will be finally attached on a users face as a soft mask,
considerable efforts are made to address the user experience
issues. Ideally the skin-attached sensing devices should be
miniaturized, unobtrusive, and long lasting in nature. Consider-
ing the implemented mask-like device is design for continuous
pain status monitoring, several practical issues have been taken
into consideration during the prototype design from the aspects
of users’ comfortableness prototype is implemented on flexible
printed circuit board which is bendable to naturally fit to the
curve of human face. Electronic modules and peripheral com-
ponents are mounted on the front side of the flexible printed
circuit board while a coin battery and electrode interface are
mounted on the back side. The dimension of the implemented
electronic core module is around 30 mm ×60 mm, as shown
in Fig. 3.
Fig. 3. Miniaturized flexible sEMG sensor encapsulated by a thin waterproof
dressing
In addition, in most cases, constant exposure to moisture
and water, e.g. sweat, is one of the leading causes of elec-
tronic failure in wearable devices. The same applies to the
implemented mask sensor proposed in this paper. Further
enhancement is needed during integration process to protect
the inner electronic components against moisture and water.
One straightforward and efficient approach is to encapsulate
the electronic components by using waterproofing thin dress-
ing materials. 3M Tegaderm adhesive dressing is adopted to
encapsulate the electronics modules while leaving openings
for the electrode interface and battery socket on the back side.
The encapsulating layer is very thin (with the sickness of 0.46
mm), yet highly absorbent, with excellent vapour permeability.
The thin dressing is semi-transparent, allowing the observation
of inner condition of the electronics and the running status
of the device, e.g. the blinking of LED for status indication.
By applying the thin dressing layer over the electronic parts,
the devices mechanical and electrical reliability is further
enhanced, in the meantime, making the device waterproof
yet with excellent moisture permeability. As shown in Fig. 3,
the implemented flexible sensing device is thin and bendable
addressing on user comfortableness issues. It will serve as
the core module of the proposed smart mask for remote pain
monitoring.
3) Wearable bio-sensing facial mask: The main facial mus-
cles under monitoring are listed in Table I. Most of the muscles
are involved in one or several facial action units (AU) in Facial
Action Coding System [30] as pain behaviors [12]. To mon-
itor the activities of these facial muscles, passive electrodes
are placed on one side of the face following Fridlund and
Cacioppo Guidelines [31].
TABLE I
FACI AL MU SC LES U ND ER MO NIT OR ING A ND T HE TAR GET ED FAC IAL
ACTION UNITS
Channel Muscular basis AU
1 Frontalis
2 Corrugator Brow lower (AU 4)
3 Orbicularis oculi Lids tighten (AU 6)
Cheek raise (AU 7)
4 Levator Nose wrinkle (AU 9)
Upper lip raiser (AU 10)
Eyes close (AU 43)
5 Zygomatic Lip corner pull (AU 12)
6 Risorius Horizontal mouth stretch (AU 20)
Fig 4(a) shows the concept of the facial mask, which
includes two part, a sensor node and a soft facial mask.
The sensor node part is to, condition, digitalize and wireless
transmit sEMG signal, as mentioned above. While the soft
facial mask part is to capture multiple channel sEMG. To
work compatible with the sensor node, the soft facial mask
integrates electrode leads as the connection between electrodes
and the sensor node in addition to the conductive electrodes
for capturing sEMG signal. Six biopotential channels are
employed on the mask to monitor six facial areas in monopolar
configuration. In monopolar configuration, each channel of
biopotential is captured with respect to the common reference
electrode placed on the bony area behind the ear. In this
way, the amount of electrodes and connecting leads is reduced
comparing with biopolar configuration where both differential
input electrodes to an amplifier are placed near to each other
in the same facial area.
The proposed mask is implemented by integrating the
detecting electrodes into soft polydimethylsiloxane (PDMS)
substrate. As a result, the designed mask is easy-to-apply,
and one-step solution, which can largely save the valuable
time of the care givers when making setting up for sensing
vital bio-signals from patients, in particular in the ICU ward
environment. The implementation of the facial mask design is
presented in Fig. 4(b). The customized facial mask is based
on transparent polydimethylsiloxane (PDMS) material. The
softness of PDMS makes the mask fit well on the curvature
of the user’s face. The thickness of the manufactured mask
is 100 µm. In the process of molding the mask, six pre-
gelled Ag/AgCl electrodes are placed on the molding substrate
and therefore embedded in the mask. The mask in Fig. 4(b)
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 5
(a) Design concept of the facial mask (b) Electrodes-embedded facial mask (c) The complete wearable device
Fig. 4. Design concept and implementation of the pain assessment facial mask
weights 7.81 g. (Reviewer 3.2: Since the PDMS substrate is
soft and stretchable which make it favorable for deforming the
shape of the mask and adjusting the electrode positions to the
corresponding detection points on patients face. )
Finally, the entirety of the pain assessment tool is presented
in Fig 4(c) with the integrated Wi-Fi sensor node attached
behind the ear, the electrode-embedded PDMS facial mask
and the leads connecting these two parts. The overall weight
of the implemented pain assessment tool is 39.08 g, which is
light and causes little burden to the user in long term use.
B. Cloud server
As presented in Fig. 2, the implemented cloud structure
includes UDP server receiving data from gateway and web-
socket server delivering data to end user. The digital output of
ADS1198 is eight channels 16 bit data in two’s complement
representation and is kept in binary format until being inter-
preted as magnitude showing on dashboard in the web client
end, reflecting user’s facial muscle activity information. The
recording of biopotential data is controlled by caregiver end
user in the web application, which is then synchronized with
the cloud database.
For the potential big data analytics in the future, NoSQL
database is employed as the database in cloud [32] which is
specifically implemented as CouchDB database. Static files are
stored in cloud for hosting mobile web client code.
C. Mobile web application design
When choosing the framework and database for the applica-
tion, the capability of holding big healthcare data stream and
real time analytics is considered. The mobile web application
is built upon Cordova and Ionic framework, which support
different mobile platforms including Android and IOS by of-
fering an unified web programing interface based on HTML5,
CSS and Javascript. Cordova also enables hardware access
in Javascript, such that Bluetooth chip can be reached within
the app. A library named Socket.io with WebSocket support
is introduced for fast real-time bi-directional communication
between the web server in cloud and the client in browser.
Data can be locally saved in an in-browser NoSQL database
powered by PouchDB, which also enables offline usage of the
app. Data can also be synchronized to the cloud CouchDB
database for sharing and analysing. The recorded data can be
exported as a text file, and presented as waveforms in the
dashboard, features such as digital filters, interpretation and
down-sampling are also provided.
The web-based graphical user interface (GUI) on a mobile
phone is presented in the middle of Fig. 5, together with
its three setting panels. The achieved functions include: 1).
Data settings, where data source can be chosen to present in
dashboard and start-stop data recording; 2) Log in & display
settings, where user name and password are required to ensure
user privacy; and 3) Digital signal processing (DSP) settings,
which is for digital filter configuration such as sample rate,
filter type and window function.
Fig. 5. The dashboard and its setting panels
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 6
(a) Raw biopotentials (b) With on-line filtering
Fig. 6. Dashboard displaying eight channel biopotentials
The web-based GUI can display the collected bio-signals
in a near real-time manner. Since the web-based GUI is
cross-operational-system, it is stand-alone, and can run on
any internet-connected terminal device equipped with a web
browser, including smart phone, laptop, mobile/tablet PC, etc.
Cloud-storage and cloud-computing functions are also de-
signed and merged into this application, which enable people
with authority get access to the data of interest anywhere in
the world. In addition, algorithms running on the cloud server
facilitate the near-real-time signal processing or post-analysis.
The dashboard of the web application presents the measured
multi-channel sEMG signals in near real time, depending on
the researcher or physicians requirements. Processed signals
can also be displayed after a set of cloud based filters or
algorithms being applied.
There are several benefits applying cloud-based on-line
filtering comparing to implementing signal processing on the
device. First, the process on the device is immune from the
latency and energy consumption caused by onsite processing.
Concentrating on data acquisition and transmission only, the
device can deliver streaming data seamlessly with a few flags
or time marks. Otherwise, it is critical to manage the timing of
data acquiring, processing and transmitting properly to ensure
real-time data delivery. Second, raw data saves data bandwidth
during transmission. The analog biopotentials are digitalized
into binary numbers in a signed number representation. Data
size is retrenched in every sample compared to the numbers
in floating-point format obtained after data decoding and
denoising. Third, it is more flexible to change the filters
or algorithms setting on the user end in the mobile web
application than in the sensing device. This could be helpful
when capturing different or multiple biopotentials.
V. SY ST EM VE RI FIC ATIO N AN D DEVICE SUMMARY
One test subject was involved in the test and was guided to
perform a series of facial expressions. Three facial expressions
were defined in the test and they were 1) neutral face as a blank
expression, 2) frown as slight pain which is also common to
express several other negative emotions and 3) pain expression
with a combination of AUs in Table I such as cheek raise, nose
wrinkle and horizontal mouth stretch. In the measurement,
Channel 8 was utilized for electrocardiogram monitoring from
the upper part of the chest on the left as left arm (LA) in
addition to the first six biopotential channels arranged in Table
I. The electrode of Channel 7 worked as right arm (RA).
Channel 7 and 8 measured the lead I of ECG. Each channel
had one electrode placed on the measurement site, either on
one facial muscular area or on one chest area. The biopotential
was measured by the reference to the common electrode which
was placed on the bony area behind the ear on the same side.
Multi-channel biopotentials are captured, transmitted to the
cloud platform and eventually their waveforms are presented in
the web application in a near real-time manner. The recorded
biopotential samples in the test were then downloaded for post-
analysis.
A. Biopotential acquisition and remote monitoring
The monitored multi-channel biopotentials are shown in
Fig. 6. The waveforms are updated on the dashboard in a
laptop web browser. Waveforms without and with on-line
filtering are presented separately. Fig. 6(a) shows the 8 channel
biopotential waveforms collected from the test subject’s face
without processing. The raw data received in the cloud server
is displayed on the dashboard. It can be observed that a
few concurrent spikes and reversals occur at the same time
from Channel 1 to Channel 6. The eye blink interference in
sEMG are triggered by eyelids movement, where the first two
channels are dominated by upper eyelid and Channel 3 to
Channel 6 reflect lower eyelid movement. As a result, the
eye blinks are captured in Channel 1 and Channel 2 as rising
pulses, while the concurrent signals in Channel 3 till Channel 6
are transient sharp declines. In addition, it can be found that the
dips in Channel 5 and Channel 6 are much smaller than those
in Channel 3 and Channel 4 because of locating further lower
from eyelids. During blink, the muscle groups around eye sides
make simultaneous contractions towards the eye ball, which
means the orientation of the biopotential signals measured
from the forehead is opposite to its counter parts measured
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 7
from the points below the eye. As the result shown in Fig. 6(a),
the eye-blink captured and shown in Channel 1 and Channel
2 are rising pulses, while the concurrent signals captured
in Channel 3 till Channel 6 are transient sharp declines. In
addition, it can be found that the dips in Channel 5 and
Channel 6 are much smaller than those in Channel 3 and
Channel 4 because of locating further lower from the central
muscle group around the eye.
Facial activities in the pain expression can be clearly
differentiated from the neutral expression although no signal
processing is applied. The sEMG signal is dominant in the
frequency range of 50 - 150 Hz [33]. While baseline wander
and motion artifact, as shown along with sEMG signal are
caused by body or muscle movements and in low frequency
range below 20 Hz. Since the offending artifacts obscure the
target sEMG signals in study, digital filters can be applied
to remove the interference. A high pass filter of 20 Hz can
efficiently suppress the base line wander noise and remove
motion artifacts. Besides, the high pass filter removes eye blink
pulses from the signal at the same time. A second digital filter
is added to remove powerline electrical interference, which is
a 50 Hz notch filter. The digital filters were implemented in
the mobile web application as 4th order Butterworth filters.
Butterworth filter was chosen for its flat frequency response
in the passband.
By leveraging cloud resources and web application, digital
signal processing is available in a real-time manner and the
filtered samples are shown in Fig. 6(b). In the first facial
expression, muscle activity of corrugator in Channel 2 is
observed due to frowning. It is followed by a neutral face
and then a facial expression of pain. In Neutral Face section,
the test subject remains calm; as a result, all sEMG channels
keep quiet. In Pain expression section, the test subject mimics
suffering from acute pain where facial muscles groups are
actively simulated for a few seconds. As can be observed in
Fig. 6(b), Channel 2, 3 and 5 record significant facial muscle
response. The time axis in the dashboard is in milliseconds
(ms) and the peak to peak amplitude of sEMG is in a few
millivolts after amplification in sensor node.
B. Post data analysis
Raw sEMG signals in the test are saved in file for further
offline analysis. The pattern of facial expressions in sEMG is
revealed by root mean square (RMS) feature extraction and
feature visualization. The feature RMS indicates the energy
level of the signal. Samples in time series is first filtered and
segmented, then RMS can be calculated from each segment.
Fig. 7 shows the RMS out of the first six channel sEMG in
Fig. 6(b), where the signal is cut into 500 ms segments. The
distribution of the RMS features from the test is shown in
Fig. 8 after dimension reduction with t-Distributed Stochastic
Neighbor Embedding technique [34]. A k-NN classifier (k=5)
was trained and tested with the RMS features. The classifica-
tion accuracy in the 10-fold cross validation is 95.9%. .
C. Device summary
This remote pain monitoring system centers on the sensor
node from the patient end. Cloud and the designed mobile web
5 10 15 20 25 30
1000
2000
3000
CH 1
5 10 15 20 25 30
1000
2000
3000
CH 2
5 10 15 20 25 30
1000
2000
3000
CH 3
RMS value
5 10 15 20 25 30
1000
2000
3000
CH 4
5 10 15 20 25 30
1000
2000
3000
CH 5
5 10 15 20 25 30
1000
2000
3000
CH 6
Segments (500ms/segement)
Pain expression
Neutral faceFrown
Fig. 7. RMS features of collected samples in 500 ms segments
Fig. 8. The distribution of RMS features
application assist in taking over the collected biopotentials
automatically and remotely. The implemented wearable IoT
device together with the back-end cloud architecture as a
whole can provide the users not only the online signal pro-
cessing and data storage functions, but also data visualization
and graphical interacting interface solution. Key features of
the designed sensor are summarized in Table II. The designed
sensor node is aiming at minimized size with full function of
biopotential data collection and Wi-Fi wireless transmission.
It reaches the size of 60mm ×30mm as a flat and soft
module, for 8 channel biopotential collection at 1000 SPS.
Energy efficiency is fully addressed by using ultra low power
electronic components and cutting down the radio frequency
power consumption by triggering wireless data transmission
into burst mode. The applied biopotential measurement AFE
component is at the power of 8.2 mW and Wi-Fi part integrated
with the embedded micro processor consumes only 9.1 mW
when listening interval is 100 ms.
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 8
TABLE II
SUMMARY OF BIOPOTENTIAL MEASUREMENT SENSOR NODE
Attribute Value
Size 60mm×30mm
Sampling parameters 8 channels, 1000 SPS sample rate, 16-bit
Data rate 128 Kbps
Power consumption
Biopotential measurement: 8.2 mW with 5 V
power supply
Wi-Fi module supply voltage: 3.5 V
–Listening interval: 1000 ms
Wi-Fi module supply current: 0.76 mA
–Listening interval: 100 ms
Wi-Fi module supply current: 2.6 mA
VI. CONCLUSION
This paper presented a design of a wearable bio-sensing
device for biopotentials monitoring in up to eight channels.
With a wearable facial mask, the device is capable of collect-
ing sEMG from several facial muscles simultaneously. The
design can be applied in pain assessment when monitoring
facial expressions as a behavioural sign of pain. Both the
sensing facial mask and the sensor node are potential for
long term use because both wearing comfortableness and low
energy consumption are considered and qualified in the design.
Furthermore, the wearable device works as a Wi-Fi sensor
node, integrated into an IoT system for remote monitoring use.
The IoT-based remote monitoring system is scalable in terms
of devices and functionality due to the sensor-gateway-cloud
architecture. In the implemented system, the cloud platform
enables mobile web applications so that caregivers are able to
reach the interactive GUI for biopotential monitoring across
operation systems. Moreover, the cloud platform provides
room for any further implementation on on-line data analysis
and decision-making support algorithm for the pain manage-
ment application. The implemented system is also applicable
to other healthcare applications where biosignals need to be
monitored in the near real-time manner.
ACKNOWLEDGMENT
The authors would like to thank Gaoyang Pang for his work
developing PDMS based mask and Jia Deng for her volunteer
work as a test subject.
REFERENCES
[1] H. C. Noel, D. C. Vogel, J. J. Erdos, D. Cornwall, and F. Levin, “Home
telehealth reduces healthcare costs,” Telemedicine Journal & E-Health,
vol. 10, no. 2, pp. 170–183, 2004.
[2] C. R. Jonassaint, N. Shah, J. Jonassaint, and L. De Castro, “Usability
and feasibility of an mHealth intervention for monitoring and managing
pain symptoms in sickle cell disease: The sickle cell disease mobile
application to record symptoms via technology (SMART),Hemoglobin,
vol. 39, no. 3, pp. 162–168, 2015.
[3] L. Catarinucci, D. De Donno, L. Mainetti, L. Palano, L. Patrono,
M. L. Stefanizzi, and L. Tarricone, “An IoT-aware architecture for smart
healthcare systems,” IEEE Internet Things J., vol. 2, no. 6, pp. 515–526,
2015.
[4] E. Jacob, J. Duran, J. Stinson, M. A. Lewis, and L. Zeltzer, “Remote
monitoring of pain and symptoms using wireless technology in children
and adolescents with sickle cell disease,” Journal of the American
Academy of Nurse Practitioners, vol. 25, no. 1, pp. 42–54, 2013.
[5] S. Kofoed, S. Breen, K. Gough, and S. Aranda, “Benefits of remote
real-time side-effect monitoring systems for patients receiving cancer
treatment,” Oncology reviews, vol. 6, no. 1, 2012.
[6] K. Kroenke, D. Theobald, J. Wu, K. Norton, G. Morrison, J. Carpenter,
and W. Tu, “Effect of telecare management on pain and depression
in patients with cancer: A randomized trial,” Journal of the American
Medical Association, vol. 304, no. 2, pp. 163–171, 2010.
[7] P. W. Peng, M. A. Stafford, D. T. Wong, and M. E. Salenieks, “Use of
telemedicine in chronic pain consultation: A pilot study,” The Clinical
journal of pain, vol. 22, no. 4, pp. 350–352, 2006.
[8] P. Lucey, J. F. Cohn, I. Matthews, S. Lucey, S. Sridharan, J. Howlett, and
K. M. Prkachin, “Automatically detecting pain in video through facial
action units,” IEEE Trans. Syst. Man Cybern. B, bern., vol. 41, no. 3,
pp. 664–674, 2011.
[9] M. K¨
achele, P. Werner, A. Al-Hamadi, G. Palm, S. Walter, and
F. Schwenker, “Bio-visual fusion for person-independent recognition of
pain intensity,” in Multiple Classifier Systems, 2015, pp. 220–230.
[10] M. S. Hossain and G. Muhammad, “Cloud-assisted speech and face
recognition framework for health monitoring,Mobile Networks and
Applications, vol. 20, no. 3, pp. 391–399, 2015.
[11] Y. Zhong and L. Liu, “Remote neonatal pain assessment system based
on internet of things,” in IEEE Int. Conf. on Internet of Things, 2011,
pp. 629–633.
[12] K. M. Prkachin, “Assessing pain by facial expression: Facial expression
as nexus,” Pain Research and Management, vol. 14, no. 1, pp. 53–58,
2009.
[13] W. Guo, X. Sheng, H. Liu, and X. Zhu, “Development of a multi-channel
compact-size wireless hybrid sEMG/NIRS sensor system for prosthetic
manipulation,” IEEE Sensors J., vol. 16, no. 2, pp. 447–456, 2016.
[14] S. Benatti, F. Casamassima, B. Milosevic, E. Farella, P. Sch¨
onle,
S. Fateh, T. Burger, Q. Huang, and L. Benini, “A versatile embedded
platform for emg acquisition and gesture recognition,” IEEE transactions
on biomedical circuits and systems, vol. 9, no. 5, pp. 620–630, 2015.
[15] A. Gruebler and K. Suzuki, “Design of a wearable device for reading
positive expressions from facial emg signals,IEEE Transactions on
Affective Computing, vol. 5, no. 3, pp. 227–237, 2014.
[16] A. Albulbul, “Evaluating major electrode types for idle biological signal
measurements for modern medical technology,Bioengineering, vol. 3,
no. 3, p. 20, 2016.
[17] M. Chimene and R. Pall`
as Areny, “A comprehensive model for power
line interference in biopotential measurements,” IEEE Trans. Instrum.
Meas., vol. 49, no. 3, pp. 535–540, 2000.
[18] K. Maharatna, E. B. Mazomenos, J. Morgan, and S. Bonfiglio, “Towards
the development of next-generation remote healthcare system: Some
practical considerations,” in IEEE Int. Symp. on Circuits and Systems,
2012, pp. 1–4.
[19] M. Jiang, T. N. Gia, A. Anzanpour, A.-M. Rahmani, T. Westerlund,
S. Salanter¨
a, P. Liljeberg, and H. Tenhunen, “IoT-based remote facial
expression monitoring system with sEMG signal,” in IEEE Sensors
Applications Symp., 2016, pp. 211–216.
[20] G. Ji, W. Ouyang, K. Yang, and G. Yang, “Skin-attached sensor and
artifact removal using cloud computing,” in 7th Int. Conf. on e-Health,
2015.
[21] A. M. Rahmani, T. N. Gia, B. Negash, A. Anzanpour, I. Azimi, M. Jiang,
and P. Liljeberg, “Exploiting smart e-health gateways at the edge
of healthcare internet-of-things: A fog computing approach,” Future
Generation Computer Systems, 2017.
[22] D. Guinard, V. Trifa, F. Mattern, and E. Wilde, “From the internet of
things to the web of things: Resource-oriented architecture and best
practices,” in Architecting the Internet of things. Springer, 2011, pp.
97–129.
[23] A. Charland and B. Leroux, “Mobile application development: web vs.
native,Communications of the ACM, vol. 54, no. 5, pp. 49–53, 2011.
[24] C. R. Raj and S. B. Tolety, “A study on approaches to build cross-
platform mobile applications and criteria to select appropriate approach,”
in India Conference (INDICON), 2012 Annual IEEE. IEEE, 2012, pp.
625–629.
[25] A.-M. Rahmani, N. K. Thanigaivelan, T. N. Gia, J. Granados, B. Negash,
P. Liljeberg, and H. Tenhunen, “Smart e-health gateway: Bringing
intelligence to internet-of-things based ubiquitous healthcare systems,”
in IEEE Consumer Communications and Networking Conf., 2015, pp.
826–834.
[26] “ADS1198 low-noise, 8 channel, 16 bit analog front end for ECG/EEG
measurements,” http://www.ti.com/product/ADS1198, [Online; accessed
20-June-2017].
[27] A. Van Boxtel, “Optimal signal bandwidth for the recording of surface
emg activity of facial, jaw, oral, and neck muscles,” Psychophysiology,
vol. 38, no. 01, pp. 22–34, 2001.
[28] “RTX41xx low power modules,” https://www.rtx.dk/, [Online; accessed
20-June-2017].
JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 9
[29] M. S. Mahmoud and A. A. Mohamad, “A study of efficient power
consumption wireless communication techniques/modules for internet of
things (iot) applications,” Advances in Internet of Things, vol. 6, no. 02,
p. 19, 2016.
[30] P. Ekman and W. V. Friesen, “Facial Action Coding System: A
Techniques for the Measurement of Facial Movement,” Consulting
Psychologists Press, Palo Alto, 1978.
[31] A. J. Fridlund and J. T. Cacioppo, “Guidelines for human electromyo-
graphic research,” Psychophysiology, vol. 23, no. 5, pp. 567–589, 1986.
[32] A. Moniruzzaman and S. A. Hossain, “Nosql database: New era of
databases for big data analytics-classification, characteristics and com-
parison,” arXiv preprint arXiv:1307.0191, 2013.
[33] C. J. De Luca, “Surface electromyography: Detection and recording,”
DelSys Incorporated, pp. 1–10, 2002.
[34] L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal
of Machine Learning Research, vol. 9, no. Nov, pp. 2579–2605, 2008.
Geng Yang received his Ph.D. degree from Elec-
tronic and Computer Systems from the Royal In-
stitute of Technology (KTH), Stockholm, Sweden,
in 2013. From 2013 to 2015, he worked as a Post-
Doc researcher in iPack VINN Excellence Center,
the School of Information and Communication Tech-
nology (ICT), KTH, Stockholm, Sweden. Currently,
he is a Research Associate in Zhejiang University,
Hangzhou, China. He developed low power, low
noise bio-electric SoC sensors for m-health. His
research interests include flexible and stretchable
electronics, mixed-mode IC design, low-power biomedical microsystem, wear-
able bio-devices, human-computer interface, intelligent sensors and Internet-
of-Things for healthcare.
Mingzhe Jiang received her B.Eng and M.Eng in
Instrument Science and Technology from Harbin
Institute of Technology, Harbin, China, in the year
2012 and 2014, respectively. She is currently with
Internet-of-Things for Healthcare (IoT4Health) re-
search group as a PhD student, at University of
Turku, Finland, working on biomedical signal pro-
cessing and pain pattern recognition in the project
Smart Pain Assessment (SPA).
Wei Ouyang received his Master’s degree from
Department of Measuring and Optical Engineering,
Nanchang Hangkong University, China, in 2013. He
was a visiting researcher in KTH Royal Institute
of Technology, Sweden, in 2014 and an engineer
in Institut Pasteur, France, in 2015. Currently, he
is a PhD candidate in a joint doctoral program
between Institut Pasteur and Centre for Research
and Interdisciplinary in Paris, working on imaging
and modeling yeast chromosome architecture with
Super-resolution Localization Microscopy and Deep
Learning.
Guangchao Ji received his Master’s degree from
Department of System-on-chip Design, KTH Royal
Institute of Technology, Sweden in 2013. Currently,
he is a PhD candidate in Department of School of
Technology and Health,in KTH Royal Institute of
Technology, working on applied biomedical device.
Amir M. Rahmani received his Master’s degree
from Department of Electrical and Computer En-
gineering, University of Tehran, Iran, in 2009 and
Ph.D. degree from Department of Information Tech-
nology, University of Turku, Finland, in 2012. He
also received his MBA jointly from Turku School
of Economics and European Institute of Innovation
& Technology (EIT) ICT Labs, in 2014. He is
currently Marie Curie Global Fellow at University of
California Irvine (USA) and TU Wien (Austria). He
is also an adjunct professor (Docent) in embedded
parallel and distributed computing at the University of Turku, Finland. He is
the author of more than 140 peer-reviewed publications.
Pasi Liljeberg received the MSc and PhD degrees
in electronics and information technology from the
University of Turku, Turku, Finland, in 1999 and
2005, respectively. He received Adjunct professor-
ship in embedded computing architectures in 2010.
Currently he is working as a professor in University
of Turku in the field of Embedded Systems and
Internet of Things. At the moment his research is
focused on biomedical engineering and health tech-
nology. In that context he has established and leading
the Internet-of-Things for Healthcare (IoT4Health)
research group. Liljeberg is the author of more than 250 peer-reviewed
publications.
Hannu Tenhunen received the diplomas from the
Helsinki University of Technology, Finland, 1982,
and the PhD degree from Cornell University, Ithaca,
NY, 1986. In 1985, he joined the Signal Process-
ing Laboratory, Tampere University of Technology,
Finland, as an associate professor and later served
as a professor and department director. Since 1992,
ha has been a professor at the Royal Institute of
Technology (KTH), Sweden, where he also served as
a dean. He has more than 600 reviewed publications
and 16 patents internationality.
... The application of EMG signal processing is multifold. It can be used to diagnose neuromuscular disease, control artificial organs, and identify pain [1][2][3]. Studies in [4] depict an automated pain assessment using wearables and fog computing. The system is composed of sensors that collect EMG signals, a micro-controller for pre-processing and signal transmission, and fog and cloud computing that run machine-learning algorithms for pain assessment. ...
... Automatic pain assessments for seriously sick patients were studied to assess the sEMG signal possibility for pain detection in comparison to healthy individuals. The goal was to observe the facial muscles' expressions to be able to distinguish pain and consider the certainty of pain strength [2]. ...
Article
Full-text available
The real-time recognition of pain level is required to perform an accurate pain assessment of patients in the intensive care unit, infants, and other subjects who may not be able to communicate verbally or even express the sensation of pain. Facial expression is a key pain-related behavior that may unlock the answer to an objective pain measurement tool. In this work, a machine learning-based pain level classification system using data collected from facial electromyograms (EMG) is presented. The dataset was acquired from part of the BioVid Heat Pain database to evaluate facial expression from an EMG corrugator and EMG zygomaticus and an EMG signal processing and data analysis flow is adapted for continuous pain estimation. The extracted pain-associated facial electromyography (fEMG) features classification is performed by K-nearest neighbor (KNN) by choosing the value of k which depends on the nonlinear models. The presentation of the accuracy estimation is performed, and considerable growth in classification accuracy is noticed when the subject matter from the features is omitted from the analysis. The ML algorithm for the classification of the amount of pain experienced by patients could deliver valuable evidence for health care providers and aid treatment assessment. The proposed classification algorithm has achieved a 99.4% accuracy for classifying the pain tolerance level from the baseline (P0 versus P4) without the influence of a subject bias. Moreover, the result on the classification accuracy clearly shows the relevance of the proposed approach.
... IoT based light weight sensors are used to collect the data and signals which is sent to the health care department using Internet with reduced bandwidth and traffic. In [12] a bio sensing mask is proposed which can be used for recording the facial expressions that can notify the occurrence of pain in the body of user. It connects the patients with their caretakers and health care centers over cloud. ...
Article
Full-text available
With rapid advancements in the technology, almost all the devices around are becoming smart and contribute to the Internet of Things (IoT) network. When a new IoT device is added to the network, it is important to verify the authenticity of the device before allowing it to communicate with the network. Hence, access control is a crucial security mechanism that allows only the authenticated node to become the part of the network. An access control mechanism also supports confidentiality, by establishing a session key that accomplishes secure communications in open public channels. Recently, blockchain has been implemented in access control protocols to provide a better security mechanism. The foundation of this survey article is laid on IoT, where a detailed description on IoT, its architecture and applications is provided. Further, various security challenges and issues, security attacks possible in IoT and their countermeasures are also provided. We emphasize on the blockchain technology and its evolution in IoT. A detailed description on existing consensus mechanisms and how blockchain can be used to overpower IoT vulnerabilities is highlighted. Moreover, we provide a comprehensive description on access control protocols. The protocols are classified into certificate-based, certificate-less and blockchain-based access control mechanisms for better understanding. We then elaborate on each use case like smart home, smart grid, health care and smart agriculture while describing access control mechanisms. The detailed description not only explains the implementation of the access mechanism, but also gives a wider vision on IoT applications. Next, a rigorous comparative analysis is performed to showcase the efficiency of all protocols in terms of computation and communication costs. Finally, we discuss open research issues and challenges in a blockchain-envisioned IoT network.
Article
Full-text available
Health monitoring sensors are widely available to monitor patients' health remotely. The collected sensor data are sent to cloud storage and processed in a remote health monitoring system. The Internet of Things (IoT) with cloud support offers a promising answer to data exploding concerns for the ability to constrain specific gadgets. However, IoT faces many security issues when sharing data between two users because of the cloud's leverage nature. In the case of a private cloud, simple encryption techniques can ensure security. But, in the case of a public cloud, maintaining data privacy is a significant issue. To overcome this concern, an optimized Brakerski‐Gentry‐Vaikuntanathan fully homomorphic encryption (BGV‐FHE) encryption method for secured data mutuality is proposed in this article. In this method, IoT medical data are initially acquired and then encrypted using two key schemes, and the data are stored in cloud storage. The main objective of these key schemes is to select optimal key parameters. Key parameters are chosen using the glow‐worm swarm optimization and used for the encryption phase. Storing data in the cloud using a public key provides access to requested users. Meanwhile, sensitive data are encrypted using a private key and cannot be accessed unless authenticated. This proposed approach has two levels of encryption, and thus it is an efficient data secured protocol for cloud resources. Implementing the optimized BGV‐FHE scheme based encryption scheme in Python, two datasets result in an accuracy level of 77% and 91%, respectively.
Article
Background Population growth and aging have highlighted the need for more effective home and prehospital care. Interconnected medical devices and applications, which comprise an infrastructure referred to as the Internet of Medical Things (IoMT), have enabled remote patient monitoring and can be important tools to cope with these demographic changes. However, developing IoMT platforms requires profound knowledge of clinical needs and challenges related to interoperability and how these can be managed with suitable technologies. Objective The purpose of this scoping review is to summarize the best practices and technologies to overcome interoperability concerns in IoMT platform development for medical emergencies in home and prehospital care. Methods This scoping review will be conducted in accordance with Arksey and O’Malley’s 5-stage framework and adhere to the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols) guidelines. Only peer-reviewed articles published in English will be considered. The databases/web search engines that will be used are IEEE Xplore, PubMed, Scopus, Google Scholar, National Center for Biotechnology Information, SAGE Journals, and ScienceDirect. The search process for relevant literature will be divided into 4 different steps. This will ensure that a suitable approach is followed in terms of search terms, limitations, and eligibility criteria. Relevant articles that meet the inclusion criteria will be screened in 2 stages: abstract and title screening and full-text screening. To reduce selection bias, the screening process will be performed by 2 reviewers. Results The results of the preliminary search indicate that there is sufficient literature to form a good foundation for the scoping review. The search was performed in April 2022, and a total of 4579 articles were found. The main clinical focus is the prevention and management of falls, but other medical emergencies, such as heart disease and stroke, are also considered. Preliminary results show that little attention has been given to real-time IoMT platforms that can be deployed in real-world care settings. The final results are expected to be presented in a scoping review in 2023 and will be disseminated through scientific conference presentations, oral presentations, and publication in a peer-reviewed journal. Conclusions This scoping review will provide insights and recommendations regarding how interoperable real-time IoMT platforms can be developed to handle medical emergencies in home and prehospital care. The findings of this research could be used by researchers, clinicians, and implementation teams to facilitate future development and interdisciplinary discussions. International Registered Report Identifier (IRRID) DERR1-10.2196/40243
Chapter
Applied to health field, Internet of Things (IoT) systems provides continuous and ubiquitous monitoring and assistance, allowing the creation of valuable tools for diagnosis, health empowerment, and personalized treatment, among others. Advances in these systems follow different approaches, such as the integration of new protocols and standards, combination with artificial intelligence algorithms, application of big data processing methodologies, among others. These new systems and applications also should face different challenges when applying this kind of technology into health areas, such as the management of personal data sensed, integration with electronic health records, make sensing devices comfortable to wear, and achieve an accurate acquisition of the sensed data. The objective of this chapter is to present the state of the art, indicating the most current IoT trends applied to the health field, their contributions, technologies applied, and challenges faced.
Article
Integration of the latest technological advancements such as Internet of Things (IoT) and Computational Intelligence (CI) techniques is an active research area for various industrial applications. The rapid urbanization and exponential growth of vehicles has led to crowded traffic in cities. The deployment of IoT infrastructures for building smart and intelligent traffic management system greatly improves the quality and comfort of city dwellers. This work aims at building a cost effective IoT enabled traffic forecasting system using deep learning techniques. The case study experimentation is done in a real time traffic environment. The main contributions of this work include: (i) deploying road side sensor station built with ultrasonic sensor and Arduino Uno controller for obtaining traffic flow data (ii) building an IoT cloud system based on open source Thingspeak cloud platform for monitoring real time traffic (iii) performing short term traffic forecast using Recurrent Neural Network (RNN) models such as Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). The performance of the prediction model is compared with the traditional statistical methods such as Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA (SARIMA) and Convolutional Neural Network (CNN). The results show good performance metrics with RMSE of 5.8, 7.9, 10.2 for LSTM model and 6.7, 8.6, 10.9 for GRU model for three different scenarios such as whole day, morning congested hour and evening congested hour datasets.
Article
Unstructured: Face masks are an important way to fight the COVID-19 pandemic. However, the prolonged pandemic has revealed confounding problems of the current face masks, not only the spread of the disease but also concurrent psychological, social, and economic complications. As face masks have been worn for a long time, people have been interested in expanding the purpose of masks from protection to comfort and health, leading to the release of various "smart" mask products around the world. To envision how the smart masks will be extended, this paper reviewed 25 smart masks (12 from commercial products and 13 from academic prototypes) that emerged after the pandemic. While most smart masks presented in the market focus on solving user breathing discomfort problems that arise from prolonged use, academic prototypes were designed for not just sensing COVID-19 but for general health monitoring aspects. Next, we investigated several specific sensors that can be incorporated into the mask for expanding biophysical features. On a larger scale, we discussed the architecture and possible applications with the help of connected smart masks. Namely, beyond a personal sensing application, a group or community sensing application may share an aggregate version of information with the broader population. In addition, this kind of collaborative sensing will also address the challenges of individual sensing, such as reliability and coverage. Lastly, we identified possible service application fields and further considerations for actual use. Along with daily life health monitoring, smart masks may work as a general respiratory health tool for sports training, emergency room/ambulatory setting, protection for industry workers and firefighters, and soldier safety and survivability. For further considerations, we investigated design aspects in terms of sensor reliability and reproducibility, ergonomic design for user acceptance, and privacy-aware data handling. Overall, we aim to explore new possibilities by examining the latest research, sensor technologies, and application platform perspectives for smart masks as one of the promising wearable devices. By integrating biomarkers of respiration symptoms, a smart mask can be a truly cutting-edge device that expands further knowledge on health monitoring to reach the next level of wearables.
Article
Full-text available
Internet of Things (IoT) based wearable healthcare data monitoring is an emerging application of smart medicines. IoT-based communication and information exchange methods make it adaptable for the recent sixth-generation computing systems. The terahertz and large-scale processing of the 6th generation communication and computation technology is assimilated in the wearable sensor data exchange process. In this article, a fault-tolerant data processing method (FTDPM) is proposed to handle uneven sensor data. The non-uniform and unsynchronised observation interval-based sensor data is analyzed for its impact on healthcare recommendations. The reliability in the recommendation ensures the precise need for sensor data processing, mitigating the faults. In this reliability estimation process, a support vector machine classifier is used. This learning classifier differentiates the uniform and non-uniform sensor data traffic for providing reliable recommendations. The data from the uniform process is replicated in the non-uniform sequence for recommendation filling. This is carried out based on marginal classification and near-to-reliable data as classified using SVM. The IoT wearable alliance is exploited using the 6G communication paradigms in handling monitored data. The proposed method's performance is verified using the metrics processing time, recommendation failure, processing complexity, and recommendation response time.
Article
Full-text available
Current developments in ICTs such as in Internet-of-Things (IoT) and Cyber-Physical Systems (CPS) allow us to develop healthcare solutions with more intelligent and prediction capabilities both for daily life (home/office) and in-hospitals. In most of IoT-based healthcare systems, especially at smart homes or hospitals, a bridging point (i.e., gateway) is needed between sensor infrastructure network and the Internet. The gateway at the edge of the network often just performs basic functions such as translating between the protocols used in the Internet and sensor networks. These gateways have beneficial knowledge and constructive control over both the sensor network and the data to be transmitted through the Internet. In this paper, we exploit the strategic position of such gateways at the edge of the network to offer several higher-level services such as local storage, real-time local data processing, embedded data mining, etc., presenting thus a Smart e-Health Gateway. We then propose to exploit the concept of Fog Computing in Healthcare IoT systems by forming a Geo-distributed intermediary layer of intelligence between sensor nodes and Cloud. By taking responsibility for handling some burdens of the sensor network and a remote healthcare center, our Fog-assisted system architecture can cope with many challenges in ubiquitous healthcare systems such as mobility, energy efficiency, scalability, and reliability issues. A successful implementation of Smart e-Health Gateways can enable massive deployment of ubiquitous health monitoring systems especially in clinical environments. We also present a prototype of a Smart e-Health Gateway called UT-GATE where some of the discussed higher-level features have been implemented. We also implement an IoT-based Early Warning Score (EWS) health monitoring to practically show the efficiency and relevance of our system on addressing a medical case study. Our proof-of-concept design demonstrates an IoT-based health monitoring system with enhanced overall system intelligence, energy efficiency, mobility, performance, interoperability, security, and reliability.
Article
Full-text available
Biological signals such as electrocardiogram (ECG) and electromyography (EMG) that can be measured at home can reveal vital information about the patient’s health. In today modern technology, the measured ECG or EMG signals at home can be monitored by medical staff from long distance through the use of internet. Biopotential electrodes are crucial in monitoring ECG, EMG, etc., signals. Applying the right type of electrode that lasts for a long time and assists in recording high signal quality is desirable in medical devices industry. Three types of electrodes (Silver/Silver Chloride (Ag/AgCl) electrodes, Orbital electrodes and Stainless steel electrodes) were tested to identify the most appropriate one for recording biological signals. The evaluation was based on determining the electrode circuit model components and having high capacitance value or high capacitor value of electrode circuit model (Cd) and low electrode-skin impedance value or low resistor value of electrode circuit model (Rd). The results revealed that Ag/AgCl is the best type of electrodes, followed by Orbital electrodes. Stainless steel electrodes had performed poorly. However, Orbital electrodes material can last longer than Ag/AgCl and hence perform similar to Ag/AgCl electrodes, which can be idle for monitoring biological signals at home without the need for medical staff to replace the electrodes in a short period of time.
Article
Full-text available
A study of wireless technologies for IoT applications in term of power consumption has been presented in this paper. The study focuses on the importance of using low power wireless techniques and modules in IoT applications by introducing a comparative between different low power wireless communication techniques such as ZigBee, Low Power Wi-Fi, 6LowPAN, LPWA and their modules to conserve power and longing the life for the IoT network sensors. The approach of the study is in term of protocol used and the particular module that achieve that protocol. The candidate protocols are classified according to the range of connectivity between sensor nodes. For short ranges connectivity the candidate protocols are ZigBee, 6LoWPAN and low power Wi-Fi. For long connectivity the candidate is LoRaWAN protocol. The results of the study demonstrate that the choice of module for each protocol plays a vital role in battery life due to the difference of power consumption for each module/protocol. So, the evaluation of protocols with each other depends on the module used.
Article
Full-text available
Wearable devices offer interesting features, such as low cost and user friendliness, but their use for medical applications is an open research topic, given the limited hardware resources they provide. In this paper, we present an embedded solution for real-time EMG-based hand gesture recognition. The work focuses on the multi-level design of the system, integrating the hardware and software components to develop a wearable device capable of acquiring and processing EMG signals for real-time gesture recognition. The system combines the accuracy of a custom analog front end with the flexibility of a low power and high performance microcontroller for on-board processing. Our system achieves the same accuracy of high-end and more expensive active EMG sensors used in applications with strict requirements on signal quality. At the same time, due to its flexible configuration, it can be compared to the few wearable platforms designed for EMG gesture recognition available on market. We demonstrate that we reach similar or better performance while embedding the gesture recognition on board, with the benefit of cost reduction. To validate this approach, we collected a dataset of 7 gestures from 4 users, which were used to evaluate the impact of the number of EMG channels, the number of recognized gestures and the data rate on the recognition accuracy and on the computational demand of the classifier. As a result, we implemented a SVM recognition algorithm capable of real-time performance on the proposed wearable platform, achieving a classification rate of 90%, which is aligned with the state-of-the-art off-line results and a 29.7 mW power consumption, guaranteeing 44 hours of continuous operation with a 400 mAh battery.
Article
Full-text available
It is evident that surface electromyography (sEMG)-based approaches have inherent difficulty in coping with modern dominant applications of clinical diagnosis and human machine interface such as prosthetic manipulation. This paper presents a hybrid sensor to attentively overcome the difficulty with a clinical purpose of simultaneously acquiring electrophysiological, hemodynamic, and oxidative metabolic information of muscle activity, also with front-end conditioning circuit and Bluetooth module integrated and packaged. A multi-channel compact-size wireless hybrid sEMG/near-infrared spectroscopy (NIRS) acquisition system is developed, forming a platform to demonstrate individual sEMG and NIRS measurement capabilities, and their combination. Extensive experiments are carried out to explore sensor functionality based on the sEMG, NIRS, and their combination, convincingly addressing the capabilities meeting their commercial or state-of-the-art counterparts. Future work is targeted to extract the sEMG/NIRS sensor-based muscular fatigue which plays a crucial role in biomedical and clinical applications.
Article
Full-text available
The increasing demand for the remote monitoring of patients combined with the promising potential of cloud computing has enabled the design and development of a number of cloud-based systems and services for healthcare. The cloud computing, in combination with the popularity of smart handheld devices, has inspired healthcare professionals to remotely monitor patients’ health while the patient is at home. To this end, this paper proposes a cloud-assisted speech and face recognition framework for elderly health monitoring, where handheld devices or video cameras collect speech along with face images and deliver to the cloud server for possible analysis and classification. In the framework, a patient’s state such as pain, tensed, and so forth is recognized from his or her speech and face images. The patient state recognition system extracts local features from speech, and texture descriptors from face images. Then it classifies using support vector machines. The recognized state is later sent to the remote care center, healthcare professionals and providers for necessary services in order to provide seamless health monitoring. Experiments have been performed to validate the approach and to evaluate the suitability of this framework in terms of accuracy and time requirements. The results demonstrate the effectiveness of the proposed approach with regards to face and speech processing.
Article
The authors designed an interactive telecare system (ITCS) enhanced by Internet of Things (IoT) technology that enables direct communication between patients' medical devices and caregivers' smartphones to improve the quality of care for chronically ill patients. This system can remotely activate hardware components of medical devices in real time to access current information and smartphones via a telecare application. A case study was constructed with 2.5G blood glucose monitors (BGMs) integrated with a cloud platform and with Android and iOS telecare applications. Overseas medical institutions have confirmed the system's potential value in chronic-illness treatment regimens and have provided useful feedback.
Conference Paper
In this work, multi-modal fusion of video and biopotential signals is used to recognize pain in a person-independent scenario. For this purpose, participants were subjected to painful heat stimuli under controlled conditions. Subsequently, a multitude of features have been extracted from the available modalities. Experimental validation suggests that the cues that allow the successful recognition of pain are highly similar across different people and complementary in the analysed modalities to an extent that fusion methods are able to achieve an improvement over single modalities. Different fusion approaches (early, late, trainable) are compared on a large set of state-of-the art features for the biopotentials and video channels in multiple classification experiments.
Article
Over the last few years, the convincing forward steps in the development of Internet of Things (IoT)-enabling solutions are spurring the advent of novel and fascinating applications. Among others, mainly radio frequency identification (RFID), wireless sensor network (WSN), and smart mobile technologies are leading this evolutionary trend. In the wake of this tendency, this paper proposes a novel, IoT-aware, smart architecture for automatic monitoring and tracking of patients, personnel, and biomedical devices within hospitals and nursing institutes. Staying true to the IoT vision, we propose a smart hospital system (SHS), which relies on different, yet complementary, technologies, specifically RFID, WSN, and smart mobile, interoperating with each other through a Constrained Application Protocol (CoAP)/IPv6 over low-power wireless personal area network (6LoWPAN)/representational state transfer (REST) network infrastructure. The SHS is able to collect, in real time, both environmental conditions and patients' physiological parameters via an ultra-low-power hybrid sensing network (HSN) composed of 6LoWPAN nodes integrating UHF RFID functionalities. Sensed data are delivered to a control center where an advanced monitoring application (MA) makes them easily accessible by both local and remote users via a REST web service. The simple proof of concept implemented to validate the proposed SHS has highlighted a number of key capabilities and aspects of novelty, which represent a significant step forward compared to the actual state of the art.